관리-도구
편집 파일: _inputstream.cpython-35.pyc
��Rea~ � @ s� d d l m Z m Z m Z d d l m Z d d l m Z m Z d d l Z d d l Z d d l m Z m Z d d l m Z d d l m Z m Z m Z m Z d d l m Z d d l m Z e d d � e D� � Z e d d � e D� � Z e d d � e D� � Z e e d d g � BZ d Z e j r�e dE d k rWe j d � d k s]t � e j! e d dF � e" d � d � Z# n e j! e � Z# d d d d d d d d d d d d d d! d" d# d$ d% d& d' d( d) d* d+ d, d- d. d/ d0 d1 d2 d3 h Z$ e j! d4 � Z% i Z& Gd5 d6 � d6 e' � Z( d7 d8 � Z) Gd9 d: � d: e' � Z* Gd; d< � d< e* � Z+ Gd= d> � d> e, � Z- Gd? d@ � d@ e' � Z. GdA dB � dB e' � Z/ dC dD � Z0 d S)G� )�absolute_import�division�unicode_literals)� text_type)� http_client�urllibN)�BytesIO�StringIO)� webencodings� )�EOF�spaceCharacters� asciiLetters�asciiUppercase)�_ReparseException)�_utilsc C s g | ] } | j d � � q S)�ascii)�encode)�.0�item� r ��/builddir/build/BUILDROOT/alt-python35-pip-20.2.4-5.el8.x86_64/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/html5lib/_inputstream.py� <listcomp> s r c C s g | ] } | j d � � q S)r )r )r r r r r r s c C s g | ] } | j d � � q S)r )r )r r r r r r s � >� <u� [- - --]�]z"\uD800-\uDFFF"i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� z[ - -/:-@\[-`{-~]c @ sj e Z d Z d Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d S)�BufferedStreamz�Buffering for streams that do not have buffering of their own The buffer is implemented as a list of chunks on the assumption that joining many strings will be slow since it is O(n**2) c C s% | | _ g | _ d d g | _ d S)Nr r ���)�stream�buffer�position)�selfr r r r �__init__: s zBufferedStream.__init__c C sP d } x2 | j d | j d � D] } | t | � 7} q! W| | j d 7} | S)Nr r )r r �len)r! �pos�chunkr r r �tell? s !zBufferedStream.tellc C sx | | j � k s t � | } d } x> t | j | � | k rd | t | j | � 8} | d 7} q' W| | g | _ d S)Nr r )�_bufferedBytes�AssertionErrorr# r r )r! r$ �offset�ir r r �seekF s zBufferedStream.seekc C sp | j s | j | � S| j d t | j � k r_ | j d t | j d � k r_ | j | � S| j | � Sd S)Nr r r )r � _readStreamr r# �_readFromBuffer)r! �bytesr r r �readO s zBufferedStream.readc C s t d d � | j D� � S)Nc S s g | ] } t | � � q Sr )r# )r r r r r r Y s z1BufferedStream._bufferedBytes.<locals>.<listcomp>)�sumr )r! r r r r' X s z BufferedStream._bufferedBytesc C sJ | j j | � } | j j | � | j d d 7<t | � | j d <| S)Nr r )r r/ r �appendr r# )r! r. �datar r r r, [ s zBufferedStream._readStreamc C s" | } g } | j d } | j d } x� | t | j � k r� | d k r� | d k s\ t � | j | } | t | � | k r� | } | | | g | _ n/ t | � | } | t | � g | _ | d 7} | j | | | | � � | | 8} d } q) W| r| j | j | � � d j | � S)Nr r � )r r# r r( r1 r, �join)r! r. �remainingBytes�rv� bufferIndex� bufferOffset� bufferedData� bytesToReadr r r r- b s& $ z BufferedStream._readFromBufferN) �__name__� __module__� __qualname__�__doc__r" r& r+ r/ r' r, r- r r r r r 3 s r c K s� t | t j � s< t | t j j � rE t | j t j � rE d } n9 t | d � ro t | j d � t � } n t | t � } | r� d d � | D� } | r� t d | � � t | | � St | | � Sd S)NFr/ r c S s% g | ] } | j d � r | � q S)� _encoding)�endswith)r �xr r r r � s z#HTMLInputStream.<locals>.<listcomp>z3Cannot set an encoding with a unicode input, set %r) � isinstancer � HTTPResponser �response�addbase�fp�hasattrr/ r � TypeError�HTMLUnicodeInputStream�HTMLBinaryInputStream)�source�kwargs� isUnicode� encodingsr r r �HTMLInputStream} s rO c @ s� e Z d Z d Z d Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d d � Z d d � Z d d � Z d d d � Z d d � Z d S)rI z�Provides a unicode stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. i ( c C s� t j s d | _ n- t d � d k r6 | j | _ n | j | _ d g | _ t d � d f | _ | j | � | _ | j � d S)a� Initialises the HTMLInputStream. HTMLInputStream(source, [encoding]) -> Normalized stream from source for use by html5lib. source can be either a file-object, local filename or a string. The optional encoding parameter must be a string that indicates the encoding. If specified, that encoding will be used, regardless of any BOM or later declaration (such as in a meta element) Nu r r zutf-8�certain) r �supports_lone_surrogates�reportCharacterErrorsr# �characterErrorsUCS4�characterErrorsUCS2�newLines�lookupEncoding� charEncoding� openStream� dataStream�reset)r! rK r r r r" � s z HTMLUnicodeInputStream.__init__c C sC d | _ d | _ d | _ g | _ d | _ d | _ d | _ d S)N� r )r% � chunkSize� chunkOffset�errors� prevNumLines� prevNumCols�_bufferedCharacter)r! r r r rZ � s z HTMLUnicodeInputStream.resetc C s( t | d � r | } n t | � } | S)zvProduces a file object from source. source can be either a file object, local filename or a string. r/ )rG r )r! rK r r r r rX � s z!HTMLUnicodeInputStream.openStreamc C st | j } | j d d | � } | j | } | j d d | � } | d k r\ | j | } n | | d } | | f S)N� r r r )r% �countr_ �rfindr` )r! r) r% �nLines� positionLine� lastLinePos�positionColumnr r r � _position� s z HTMLUnicodeInputStream._positionc C s& | j | j � \ } } | d | f S)z:Returns (line, col) of the current position in the stream.r )ri r] )r! �line�colr r r r � s z HTMLUnicodeInputStream.positionc C sI | j | j k r" | j � s" t S| j } | j | } | d | _ | S)zo Read one character from the stream or queue if available. Return EOF when EOF is reached. r )r] r\ � readChunkr r% )r! r] �charr r r rm � s zHTMLUnicodeInputStream.charNc C sI | d k r | j } | j | j � \ | _ | _ d | _ d | _ d | _ | j j | � } | j r� | j | } d | _ n | s� d St | � d k r� t | d � } | d k s� d | k o� d k n r� | d | _ | d d � } | j r | j | � | j d d � } | j d d � } | | _ t | � | _ d S)Nr[ r Fr � i � i�� z rb � Tr r r )�_defaultChunkSizeri r\ r_ r` r% r] rY r/ ra r# �ordrR �replace)r! r\ r2 �lastvr r r rl � s0 ( z HTMLUnicodeInputStream.readChunkc C s: x3 t t t j | � � � D] } | j j d � q Wd S)Nzinvalid-codepoint)�ranger# �invalid_unicode_re�findallr^ r1 )r! r2 �_r r r rS s "z*HTMLUnicodeInputStream.characterErrorsUCS4c C s d } x� t j | � D]� } | r% q t | j � � } | j � } t j | | | d � � r� t j | | | d � � } | t k r� | j j d � d } q | d k r� | d k r� | t | � d k r� | j j d � q d } | j j d � q Wd S)NF� zinvalid-codepointTi � i�� r ) ru �finditerrq �group�startr �isSurrogatePair�surrogatePairToCodepoint�non_bmp_invalid_codepointsr^ r1 r# )r! r2 �skip�match� codepointr$ �char_valr r r rT # s z*HTMLUnicodeInputStream.characterErrorsUCS2Fc C se y t | | f } Wn� t k r� x& | D] } t | � d k s+ t � q+ Wd j d d � | D� � } | sy d | } t j d | � } t | | f <Yn Xg } x� | j | j | j � } | d k r� | j | j k r$PnB | j � } | | j k r$| j | j | j | � � | | _ P| j | j | j d � � | j � s� Pq� Wd j | � } | S)z� Returns a string of characters from the stream up to but not including any character in 'characters' or EOF. 'characters' must be a container that supports the 'in' method and iteration over its characters. � r[ c S s g | ] } d t | � � q S)z\x%02x)rq )r �cr r r r H s z5HTMLUnicodeInputStream.charsUntil.<locals>.<listcomp>z^%sz[%s]+N)�charsUntilRegEx�KeyErrorrq r( r4 �re�compiler� r% r] r\ �endr1 rl ) r! � characters�opposite�charsr� �regexr6 �mr� �rr r r � charsUntil: s2 & z!HTMLUnicodeInputStream.charsUntilc C sl | t k rh | j d k r= | | j | _ | j d 7_ n+ | j d 8_ | j | j | k sh t � d S)Nr r )r r] r% r\ r( )r! rm r r r �ungeti s z HTMLUnicodeInputStream.unget)r; r< r= r> rp r" rZ rX ri r rm rl rS rT r� r� r r r r rI � s & /rI c @ s e Z d Z d Z d d d d d d d d � Z d d � Z d d � Z d d d � Z d d � Z d d � Z d d � Z d S)rJ z�Provides a unicode stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. Nz windows-1252Tc C s� | j | � | _ t j | | j � d | _ d | _ | | _ | | _ | | _ | | _ | | _ | j | � | _ | j d d k s� t � | j � d S)a� Initialises the HTMLInputStream. HTMLInputStream(source, [encoding]) -> Normalized stream from source for use by html5lib. source can be either a file-object, local filename or a string. The optional encoding parameter must be a string that indicates the encoding. If specified, that encoding will be used, regardless of any BOM or later declaration (such as in a meta element) i �d r N)rX � rawStreamrI r" � numBytesMeta�numBytesChardet�override_encoding�transport_encoding�same_origin_parent_encoding�likely_encoding�default_encoding�determineEncodingrW r( rZ )r! rK r� r� r� r� r� � useChardetr r r r" � s z HTMLBinaryInputStream.__init__c C s3 | j d j j | j d � | _ t j | � d S)Nr rr )rW � codec_info� streamreaderr� rY rI rZ )r! r r r rZ � s "zHTMLBinaryInputStream.resetc C s` t | d � r | } n t | � } y | j | j � � Wn t k r[ t | � } Yn X| S)zvProduces a file object from source. source can be either a file object, local filename or a string. r/ )rG r r+ r& � Exceptionr )r! rK r r r r rX � s z HTMLBinaryInputStream.openStreamc C s | j � d f } | d d k r& | St | j � d f } | d d k rO | St | j � d f } | d d k rx | S| j � d f } | d d k r� | St | j � d f } | d d k r� | d j j d � r� | St | j � d f } | d d k r| S| r�y d d l m } Wn t k r5Yn� Xg } | � } xX | j s�| j j | j � } t | t � s{t � | s�P| j | � | j | � qHW| j � t | j d � } | j j d � | d k r�| d f St | j � d f } | d d k r | St d � d f S)NrP r � tentativezutf-16)�UniversalDetector�encodingz windows-1252)� detectBOMrV r� r� �detectEncodingMetar� �name� startswithr� �%pip._vendor.chardet.universaldetectorr� � ImportError�doner� r/ r� rB r. r( r1 �feed�close�resultr+ r� )r! �chardetrW r� �buffers�detectorr r� r r r r� � sR ' z'HTMLBinaryInputStream.determineEncodingc C s� | j d d k s t � t | � } | d k r5 d S| j d k re t d � } | d k s� t � nr | | j d k r� | j d d f | _ nF | j j d � | d f | _ | j � t d | j d | f � � d S) Nr rP �utf-16be�utf-16lezutf-8r z Encoding changed from %s to %s)r� r� )rW r( rV r� r� r+ rZ r )r! � newEncodingr r r �changeEncoding s z$HTMLBinaryInputStream.changeEncodingc C s� t j d t j d t j d t j d t j d i } | j j d � } t | t � sZ t � | j | d d � � } d } | s� | j | � } d } | s� | j | d d � � } d } | r� | j j | � t | � S| j j d � d Sd S) z�Attempts to detect at BOM at the start of the stream. If an encoding can be determined from the BOM return the name of the encoding otherwise return Nonezutf-8zutf-16lezutf-16bezutf-32lezutf-32be� N� rx r )�codecs�BOM_UTF8� BOM_UTF16_LE� BOM_UTF16_BE� BOM_UTF32_LE� BOM_UTF32_BEr� r/ rB r. r( �getr+ rV )r! �bomDict�stringr� r+ r r r r� s$ z HTMLBinaryInputStream.detectBOMc C s} | j j | j � } t | t � s* t � t | � } | j j d � | j � } | d k ry | j d k ry t d � } | S)z9Report the encoding declared by the meta element r N�utf-16be�utf-16lezutf-8)r� r� ) r� r/ r� rB r. r( �EncodingParserr+ � getEncodingr� rV )r! r �parserr� r r r r� 3 s z(HTMLBinaryInputStream.detectEncodingMeta) r; r< r= r> r" rZ rX r� r� r� r� r r r r rJ z s ( > "rJ c @ s� e Z d Z d Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z e e e � Z d d � Z e e � Z e d d � Z d d � Z d d � Z d d � Z d S) � EncodingBytesz�String-like object with an associated position and various extra methods If the position is ever greater than the string length then an exception is raisedc C s+ t | t � s t � t j | | j � � S)N)rB r. r( �__new__�lower)r! �valuer r r r� F s zEncodingBytes.__new__c C s d | _ d S)Nr r )ri )r! r� r r r r" J s zEncodingBytes.__init__c C s | S)Nr )r! r r r �__iter__N s zEncodingBytes.__iter__c C sS | j d } | _ | t | � k r/ t � n | d k rA t � | | | d � S)Nr r )ri r# � StopIterationrH )r! �pr r r �__next__Q s zEncodingBytes.__next__c C s | j � S)N)r� )r! r r r �nextY s zEncodingBytes.nextc C sY | j } | t | � k r$ t � n | d k r6 t � | d | _ } | | | d � S)Nr r )ri r# r� rH )r! r� r r r �previous] s zEncodingBytes.previousc C s( | j t | � k r t � | | _ d S)N)ri r# r� )r! r r r r � setPositionf s zEncodingBytes.setPositionc C s9 | j t | � k r t � | j d k r1 | j Sd Sd S)Nr )ri r# r� )r! r r r � getPositionk s zEncodingBytes.getPositionc C s | | j | j d � S)Nr )r )r! r r r �getCurrentByteu s z EncodingBytes.getCurrentBytec C sf | j } xM | t | � k rX | | | d � } | | k rK | | _ | S| d 7} q W| | _ d S)z Skip past a list of charactersr N)r r# ri )r! r� r� r� r r r r z s zEncodingBytes.skipc C sf | j } xM | t | � k rX | | | d � } | | k rK | | _ | S| d 7} q W| | _ d S)Nr )r r# ri )r! r� r� r� r r r � skipUntil� s zEncodingBytes.skipUntilc C s4 | j | | j � } | r0 | j t | � 7_ | S)z�Look for a sequence of bytes at the start of a string. If the bytes are found return True and advance the position to the byte after the match. Otherwise return False and leave the position alone)r� r r# )r! r. r6 r r r � matchBytes� s zEncodingBytes.matchBytesc C sI y* | j | | j � t | � d | _ Wn t k rD t � Yn Xd S)z�Look for the next sequence of bytes matching a given sequence. If a match is found advance the position to the last byte of the matchr T)�indexr r# ri � ValueErrorr� )r! r. r r r �jumpTo� s * zEncodingBytes.jumpToN)r; r< r= r> r� r" r� r� r� r� r� r� �propertyr r� � currentByte�spaceCharactersBytesr r� r� r� r r r r r� B s r� c @ s� e Z d Z d Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d S)r� z?Mini parser for detecting character encoding from meta elementsc C s t | � | _ d | _ d S)z3string - the data to work on for encoding detectionN)r� r2 r� )r! r2 r r r r" � s zEncodingParser.__init__c C s d | j k r d Sd | j f d | j f d | j f d | j f d | j f d | j f f } x� | j D]� } d } y | j j d � Wn t k r� PYn XxP | D]H \ } } | j j | � r� y | � } PWq� t k r� d } PYq� Xq� W| sk Pqk W| j S) Ns <metas <!--s </s <!s <?r TF) r2 � handleComment� handleMeta�handlePossibleEndTag� handleOther�handlePossibleStartTagr� r� r� r� )r! �methodDispatchrw � keepParsing�key�methodr r r r� � s2 zEncodingParser.getEncodingc C s | j j d � S)zSkip over commentss -->)r2 r� )r! r r r r� � s z EncodingParser.handleCommentc C s< | j j t k r d Sd } d } x| j � } | d k rA d S| d d k r� | d d k } | r4| d k r4| | _ d Sq% | d d k r� | d } t | � } | d k r4| | _ d Sq% | d d k r% t t | d � � } | j � } | d k r% t | � } | d k r% | r.| | _ d S| } q% Wd S) NTFr s http-equivr s content-types charsets content) r2 r� r� � getAttributer� rV �ContentAttrParserr� �parse)r! � hasPragma�pendingEncoding�attr�tentativeEncoding�codec� contentParserr r r r� � s: zEncodingParser.handleMetac C s | j d � S)NF)�handlePossibleTag)r! r r r r� � s z%EncodingParser.handlePossibleStartTagc C s t | j � | j d � S)NT)r� r2 r� )r! r r r r� � s z#EncodingParser.handlePossibleEndTagc C s� | j } | j t k r6 | r2 | j � | j � d S| j t � } | d k r^ | j � n+ | j � } x | d k r� | j � } qm Wd S)NTr )r2 r� �asciiLettersBytesr� r� r� �spacesAngleBracketsr� )r! �endTagr2 r� r� r r r r� � s z EncodingParser.handlePossibleTagc C s | j j d � S)Nr )r2 r� )r! r r r r� s zEncodingParser.handleOtherc C s� | j } | j t t d g � B� } | d k sI t | � d k sI t � | d k rY d Sg } g } x� | d k r~ | r~ Pnz | t k r� | j � } Pn^ | d k r� d j | � d f S| t k r� | j | j � � n | d k r� d S| j | � t | � } qh W| d k r1| j � d j | � d f St | � | j � } | d k r�| } x� t | � } | | k r�t | � d j | � d j | � f S| t k r�| j | j � � q\| j | � q\Wn^ | d k r�d j | � d f S| t k r| j | j � � n | d k r!d S| j | � xw t | � } | t k red j | � d j | � f S| t k r�| j | j � � q1| d k r�d S| j | � q1Wd S) z_Return a name,value pair for the next attribute in the stream, if one is found, or None� /Nr r � =r3 � '� ")r N)r� r )r� r� ) r2 r r� � frozensetr# r( r4 �asciiUppercaseBytesr1 r� r� r� r� )r! r2 r� �attrName� attrValue� quoteCharr r r r� sh $ zEncodingParser.getAttributeN) r; r< r= r> r" r� r� r� r� r� r� r� r� r r r r r� � s $ r� c @ s( e Z d Z d d � Z d d � Z d S)r� c C s" t | t � s t � | | _ d S)N)rB r. r( r2 )r! r2 r r r r" a s zContentAttrParser.__init__c C sN y1| j j d � | j j d 7_ | j j � | j j d k sH d S| j j d 7_ | j j � | j j d k r� | j j } | j j d 7_ | j j } | j j | � r� | j | | j j � Sd Sn] | j j } y+ | j j t � | j | | j j � SWn# t k r/| j | d � SYn XWn t k rId SYn Xd S)Ns charsetr r� r� r� )r� r� )r2 r� r r r� r� r� r� )r! � quoteMark� oldPositionr r r r� e s. zContentAttrParser.parseN)r; r< r= r" r� r r r r r� ` s r� c C s| t | t � r; y | j d � } Wn t k r: d SYn X| d k rt y t j | � SWqx t k rp d SYqx Xn d Sd S)z{Return the python codec name corresponding to an encoding or None if the string doesn't correspond to a valid encoding.r N)rB r. �decode�UnicodeDecodeErrorr �lookup�AttributeError)r� r r r rV � s rV r r )1� __future__r r r Zpip._vendor.sixr �pip._vendor.six.movesr r r� r� �ior r � pip._vendorr � constantsr r r r r r[ r r� r� r� r� r� � invalid_unicode_no_surrogaterQ rc r( r� �evalru r~ �ascii_punctuation_rer� �objectr rO rI rJ r. r� r� r� rV r r r r �<module> sJ " + J ��b�'