o iu@sdZddlmZdZzddlZddlmZddlmZdZ Wn e y)dZ Ynwdd l m Z gd Z Gd d d e ZGd ddZGdddZdS)z@Lexical analysis of formal languages (i.e. code) using Pygments.) annotationsreStructuredTextN)get_lexer_by_name)_get_ttype_classTF)ApplicationError)tokentextc@s eZdZdS) LexerErrorN)__name__ __module__ __qualname__rrb/var/www/html/karishye-ai-python/venv/lib/python3.10/site-packages/docutils/utils/code_analyzer.pyr sr c@s,eZdZdZd d ddZddZd d Zd S)LexeraParse `code` lines and yield "classified" tokens. Arguments code -- string of source code to parse, language -- formal language the code is written in, tokennames -- either 'long', 'short', or 'none' (see below). Merge subsequent tokens of the same token-type. Iterating over an instance yields the tokens as ``(tokentype, value)`` tuples. The value of `tokennames` configures the naming of the tokentype: 'long': downcased full token type name, 'short': short name defined by pygments.token.STANDARD_TYPES (= class argument used in pygments html output), 'none': skip lexical analysis. shortreturnNonecCsj||_||_||_d|_|dvs|dkrdStstdz t|j|_WdStjj y4td|w)zE Set up a lexical analyzer for `code` in `language`. N)r rnonez0Cannot analyze code. Pygments package not found.z6Cannot analyze code. No Pygments lexer found for "%s".) codelanguage tokennameslexer with_pygmentsr rpygmentsutil ClassNotFound)selfrrrrrr__init__4szLexer.__init__ccsjt|}t|\}}|D]\}}||ur||7}q ||fV||}}q |d}|r3||fVdSdS)zrMerge subsequent tokens of same token-type. Also strip the final newline (added by pygments).  N)iternext removesuffix)rtokenslasttypelastvalttypevaluerrrmergeOs      z Lexer.mergeccs|jdurg|jfVdSt|j|j}||D]$\}}|jdkr.t|d}nt |g}dd|D}||fVqdS)z7Parse self.code and yield "classified" tokens. Nlong.cSsg|]}|tvr|qSr)unstyled_tokens).0clsrrr lsz"Lexer.__iter__..) rrrlexr(rstrlowersplitr)rr# tokentyper'classesrrr__iter__`s     zLexer.__iter__N)rrr)r r r __doc__rr(r5rrrrr s   rc@s"eZdZdZd ddZddZdS) NumberLinesaqInsert linenumber-tokens at the start of every code line. Arguments tokens -- iterable of ``(classes, value)`` tuples startline -- first line number endline -- last line number Iterating over an instance yields the tokens with a ``(['ln'], '')`` token added for every code line. Multi-line tokens are split.rrcCs&||_||_dtt|d|_dS)N%zd )r# startlinelenr0fmt_str)rr#r:endlinerrrr}szNumberLines.__init__ccs|j}dg|j|fV|jD]-\}}|d}|ddD]}||dfV|d7}dg|j|fVq||dfVqdS)Nlnr)r:r<r#r2)rlinenor&r'lineslinerrrr5s zNumberLines.__iter__Nr6)r r r r7rr5rrrrr8ps r8)r7 __future__r __docformat__rpygments.lexersrpygments.formatters.htmlrr ImportErrordocutilsrr+r rr8rrrrs      P