METADATA 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552
  1. Metadata-Version: 2.1
  2. Name: html5lib
  3. Version: 1.1
  4. Summary: HTML parser based on the WHATWG HTML specification
  5. Home-page: https://github.com/html5lib/html5lib-python
  6. Maintainer: James Graham
  7. Maintainer-email: james@hoppipolla.co.uk
  8. License: MIT License
  9. Platform: UNKNOWN
  10. Classifier: Development Status :: 5 - Production/Stable
  11. Classifier: Intended Audience :: Developers
  12. Classifier: License :: OSI Approved :: MIT License
  13. Classifier: Operating System :: OS Independent
  14. Classifier: Programming Language :: Python
  15. Classifier: Programming Language :: Python :: 2
  16. Classifier: Programming Language :: Python :: 2.7
  17. Classifier: Programming Language :: Python :: 3
  18. Classifier: Programming Language :: Python :: 3.5
  19. Classifier: Programming Language :: Python :: 3.6
  20. Classifier: Programming Language :: Python :: 3.7
  21. Classifier: Programming Language :: Python :: 3.8
  22. Classifier: Programming Language :: Python :: Implementation :: CPython
  23. Classifier: Programming Language :: Python :: Implementation :: PyPy
  24. Classifier: Topic :: Software Development :: Libraries :: Python Modules
  25. Classifier: Topic :: Text Processing :: Markup :: HTML
  26. Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*
  27. Requires-Dist: six (>=1.9)
  28. Requires-Dist: webencodings
  29. Provides-Extra: all
  30. Requires-Dist: genshi ; extra == 'all'
  31. Requires-Dist: chardet (>=2.2) ; extra == 'all'
  32. Requires-Dist: lxml ; (platform_python_implementation == 'CPython') and extra == 'all'
  33. Provides-Extra: chardet
  34. Requires-Dist: chardet (>=2.2) ; extra == 'chardet'
  35. Provides-Extra: genshi
  36. Requires-Dist: genshi ; extra == 'genshi'
  37. Provides-Extra: lxml
  38. Requires-Dist: lxml ; (platform_python_implementation == 'CPython') and extra == 'lxml'
  39. html5lib
  40. ========
  41. .. image:: https://travis-ci.org/html5lib/html5lib-python.svg?branch=master
  42. :target: https://travis-ci.org/html5lib/html5lib-python
  43. html5lib is a pure-python library for parsing HTML. It is designed to
  44. conform to the WHATWG HTML specification, as is implemented by all major
  45. web browsers.
  46. Usage
  47. -----
  48. Simple usage follows this pattern:
  49. .. code-block:: python
  50. import html5lib
  51. with open("mydocument.html", "rb") as f:
  52. document = html5lib.parse(f)
  53. or:
  54. .. code-block:: python
  55. import html5lib
  56. document = html5lib.parse("<p>Hello World!")
  57. By default, the ``document`` will be an ``xml.etree`` element instance.
  58. Whenever possible, html5lib chooses the accelerated ``ElementTree``
  59. implementation (i.e. ``xml.etree.cElementTree`` on Python 2.x).
  60. Two other tree types are supported: ``xml.dom.minidom`` and
  61. ``lxml.etree``. To use an alternative format, specify the name of
  62. a treebuilder:
  63. .. code-block:: python
  64. import html5lib
  65. with open("mydocument.html", "rb") as f:
  66. lxml_etree_document = html5lib.parse(f, treebuilder="lxml")
  67. When using with ``urllib2`` (Python 2), the charset from HTTP should be
  68. pass into html5lib as follows:
  69. .. code-block:: python
  70. from contextlib import closing
  71. from urllib2 import urlopen
  72. import html5lib
  73. with closing(urlopen("http://example.com/")) as f:
  74. document = html5lib.parse(f, transport_encoding=f.info().getparam("charset"))
  75. When using with ``urllib.request`` (Python 3), the charset from HTTP
  76. should be pass into html5lib as follows:
  77. .. code-block:: python
  78. from urllib.request import urlopen
  79. import html5lib
  80. with urlopen("http://example.com/") as f:
  81. document = html5lib.parse(f, transport_encoding=f.info().get_content_charset())
  82. To have more control over the parser, create a parser object explicitly.
  83. For instance, to make the parser raise exceptions on parse errors, use:
  84. .. code-block:: python
  85. import html5lib
  86. with open("mydocument.html", "rb") as f:
  87. parser = html5lib.HTMLParser(strict=True)
  88. document = parser.parse(f)
  89. When you're instantiating parser objects explicitly, pass a treebuilder
  90. class as the ``tree`` keyword argument to use an alternative document
  91. format:
  92. .. code-block:: python
  93. import html5lib
  94. parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("dom"))
  95. minidom_document = parser.parse("<p>Hello World!")
  96. More documentation is available at https://html5lib.readthedocs.io/.
  97. Installation
  98. ------------
  99. html5lib works on CPython 2.7+, CPython 3.5+ and PyPy. To install:
  100. .. code-block:: bash
  101. $ pip install html5lib
  102. The goal is to support a (non-strict) superset of the versions that `pip
  103. supports
  104. <https://pip.pypa.io/en/stable/installing/#python-and-os-compatibility>`_.
  105. Optional Dependencies
  106. ---------------------
  107. The following third-party libraries may be used for additional
  108. functionality:
  109. - ``lxml`` is supported as a tree format (for both building and
  110. walking) under CPython (but *not* PyPy where it is known to cause
  111. segfaults);
  112. - ``genshi`` has a treewalker (but not builder); and
  113. - ``chardet`` can be used as a fallback when character encoding cannot
  114. be determined.
  115. Bugs
  116. ----
  117. Please report any bugs on the `issue tracker
  118. <https://github.com/html5lib/html5lib-python/issues>`_.
  119. Tests
  120. -----
  121. Unit tests require the ``pytest`` and ``mock`` libraries and can be
  122. run using the ``py.test`` command in the root directory.
  123. Test data are contained in a separate `html5lib-tests
  124. <https://github.com/html5lib/html5lib-tests>`_ repository and included
  125. as a submodule, thus for git checkouts they must be initialized::
  126. $ git submodule init
  127. $ git submodule update
  128. If you have all compatible Python implementations available on your
  129. system, you can run tests on all of them using the ``tox`` utility,
  130. which can be found on PyPI.
  131. Questions?
  132. ----------
  133. There's a mailing list available for support on Google Groups,
  134. `html5lib-discuss <http://groups.google.com/group/html5lib-discuss>`_,
  135. though you may get a quicker response asking on IRC in `#whatwg on
  136. irc.freenode.net <http://wiki.whatwg.org/wiki/IRC>`_.
  137. Change Log
  138. ----------
  139. 1.1
  140. ~~~
  141. UNRELEASED
  142. Breaking changes:
  143. * Drop support for Python 3.3. (#358)
  144. * Drop support for Python 3.4. (#421)
  145. Deprecations:
  146. * Deprecate the ``html5lib`` sanitizer (``html5lib.serialize(sanitize=True)`` and
  147. ``html5lib.filters.sanitizer``). We recommend users migrate to `Bleach
  148. <https://github.com/mozilla/bleach>`. Please let us know if Bleach doesn't suffice for your
  149. use. (#443)
  150. Other changes:
  151. * Try to import from ``collections.abc`` to remove DeprecationWarning and ensure
  152. ``html5lib`` keeps working in future Python versions. (#403)
  153. * Drop optional ``datrie`` dependency. (#442)
  154. 1.0.1
  155. ~~~~~
  156. Released on December 7, 2017
  157. Breaking changes:
  158. * Drop support for Python 2.6. (#330) (Thank you, Hugo, Will Kahn-Greene!)
  159. * Remove ``utils/spider.py`` (#353) (Thank you, Jon Dufresne!)
  160. Features:
  161. * Improve documentation. (#300, #307) (Thank you, Jon Dufresne, Tom Most,
  162. Will Kahn-Greene!)
  163. * Add iframe seamless boolean attribute. (Thank you, Ritwik Gupta!)
  164. * Add itemscope as a boolean attribute. (#194) (Thank you, Jonathan Vanasco!)
  165. * Support Python 3.6. (#333) (Thank you, Jon Dufresne!)
  166. * Add CI support for Windows using AppVeyor. (Thank you, John Vandenberg!)
  167. * Improve testing and CI and add code coverage (#323, #334), (Thank you, Jon
  168. Dufresne, John Vandenberg, Sam Sneddon, Will Kahn-Greene!)
  169. * Semver-compliant version number.
  170. Bug fixes:
  171. * Add support for setuptools < 18.5 to support environment markers. (Thank you,
  172. John Vandenberg!)
  173. * Add explicit dependency for six >= 1.9. (Thank you, Eric Amorde!)
  174. * Fix regexes to work with Python 3.7 regex adjustments. (#318, #379) (Thank
  175. you, Benedikt Morbach, Ville Skyttä, Mark Vasilkov!)
  176. * Fix alphabeticalattributes filter namespace bug. (#324) (Thank you, Will
  177. Kahn-Greene!)
  178. * Include license file in generated wheel package. (#350) (Thank you, Jon
  179. Dufresne!)
  180. * Fix annotation-xml typo. (#339) (Thank you, Will Kahn-Greene!)
  181. * Allow uppercase hex chararcters in CSS colour check. (#377) (Thank you,
  182. Komal Dembla, Hugo!)
  183. 1.0
  184. ~~~
  185. Released and unreleased on December 7, 2017. Badly packaged release.
  186. 0.999999999/1.0b10
  187. ~~~~~~~~~~~~~~~~~~
  188. Released on July 15, 2016
  189. * Fix attribute order going to the tree builder to be document order
  190. instead of reverse document order(!).
  191. 0.99999999/1.0b9
  192. ~~~~~~~~~~~~~~~~
  193. Released on July 14, 2016
  194. * **Added ordereddict as a mandatory dependency on Python 2.6.**
  195. * Added ``lxml``, ``genshi``, ``datrie``, ``charade``, and ``all``
  196. extras that will do the right thing based on the specific
  197. interpreter implementation.
  198. * Now requires the ``mock`` package for the testsuite.
  199. * Cease supporting DATrie under PyPy.
  200. * **Remove PullDOM support, as this hasn't ever been properly
  201. tested, doesn't entirely work, and as far as I can tell is
  202. completely unused by anyone.**
  203. * Move testsuite to ``py.test``.
  204. * **Fix #124: move to webencodings for decoding the input byte stream;
  205. this makes html5lib compliant with the Encoding Standard, and
  206. introduces a required dependency on webencodings.**
  207. * **Cease supporting Python 3.2 (in both CPython and PyPy forms).**
  208. * **Fix comments containing double-dash with lxml 3.5 and above.**
  209. * **Use scripting disabled by default (as we don't implement
  210. scripting).**
  211. * **Fix #11, avoiding the XSS bug potentially caused by serializer
  212. allowing attribute values to be escaped out of in old browser versions,
  213. changing the quote_attr_values option on serializer to take one of
  214. three values, "always" (the old True value), "legacy" (the new option,
  215. and the new default), and "spec" (the old False value, and the old
  216. default).**
  217. * **Fix #72 by rewriting the sanitizer to apply only to treewalkers
  218. (instead of the tokenizer); as such, this will require amending all
  219. callers of it to use it via the treewalker API.**
  220. * **Drop support of charade, now that chardet is supported once more.**
  221. * **Replace the charset keyword argument on parse and related methods
  222. with a set of keyword arguments: override_encoding, transport_encoding,
  223. same_origin_parent_encoding, likely_encoding, and default_encoding.**
  224. * **Move filters._base, treebuilder._base, and treewalkers._base to .base
  225. to clarify their status as public.**
  226. * **Get rid of the sanitizer package. Merge sanitizer.sanitize into the
  227. sanitizer.htmlsanitizer module and move that to sanitizer. This means
  228. anyone who used sanitizer.sanitize or sanitizer.HTMLSanitizer needs no
  229. code changes.**
  230. * **Rename treewalkers.lxmletree to .etree_lxml and
  231. treewalkers.genshistream to .genshi to have a consistent API.**
  232. * Move a whole load of stuff (inputstream, ihatexml, trie, tokenizer,
  233. utils) to be underscore prefixed to clarify their status as private.
  234. 0.9999999/1.0b8
  235. ~~~~~~~~~~~~~~~
  236. Released on September 10, 2015
  237. * Fix #195: fix the sanitizer to drop broken URLs (it threw an
  238. exception between 0.9999 and 0.999999).
  239. 0.999999/1.0b7
  240. ~~~~~~~~~~~~~~
  241. Released on July 7, 2015
  242. * Fix #189: fix the sanitizer to allow relative URLs again (as it did
  243. prior to 0.9999/1.0b5).
  244. 0.99999/1.0b6
  245. ~~~~~~~~~~~~~
  246. Released on April 30, 2015
  247. * Fix #188: fix the sanitizer to not throw an exception when sanitizing
  248. bogus data URLs.
  249. 0.9999/1.0b5
  250. ~~~~~~~~~~~~
  251. Released on April 29, 2015
  252. * Fix #153: Sanitizer fails to treat some attributes as URLs. Despite how
  253. this sounds, this has no known security implications. No known version
  254. of IE (5.5 to current), Firefox (3 to current), Safari (6 to current),
  255. Chrome (1 to current), or Opera (12 to current) will run any script
  256. provided in these attributes.
  257. * Pass error message to the ParseError exception in strict parsing mode.
  258. * Allow data URIs in the sanitizer, with a whitelist of content-types.
  259. * Add support for Python implementations that don't support lone
  260. surrogates (read: Jython). Fixes #2.
  261. * Remove localization of error messages. This functionality was totally
  262. unused (and untested that everything was localizable), so we may as
  263. well follow numerous browsers in not supporting translating technical
  264. strings.
  265. * Expose treewalkers.pprint as a public API.
  266. * Add a documentEncoding property to HTML5Parser, fix #121.
  267. 0.999
  268. ~~~~~
  269. Released on December 23, 2013
  270. * Fix #127: add work-around for CPython issue #20007: .read(0) on
  271. http.client.HTTPResponse drops the rest of the content.
  272. * Fix #115: lxml treewalker can now deal with fragments containing, at
  273. their root level, text nodes with non-ASCII characters on Python 2.
  274. 0.99
  275. ~~~~
  276. Released on September 10, 2013
  277. * No library changes from 1.0b3; released as 0.99 as pip has changed
  278. behaviour from 1.4 to avoid installing pre-release versions per
  279. PEP 440.
  280. 1.0b3
  281. ~~~~~
  282. Released on July 24, 2013
  283. * Removed ``RecursiveTreeWalker`` from ``treewalkers._base``. Any
  284. implementation using it should be moved to
  285. ``NonRecursiveTreeWalker``, as everything bundled with html5lib has
  286. for years.
  287. * Fix #67 so that ``BufferedStream`` to correctly returns a bytes
  288. object, thereby fixing any case where html5lib is passed a
  289. non-seekable RawIOBase-like object.
  290. 1.0b2
  291. ~~~~~
  292. Released on June 27, 2013
  293. * Removed reordering of attributes within the serializer. There is now
  294. an ``alphabetical_attributes`` option which preserves the previous
  295. behaviour through a new filter. This allows attribute order to be
  296. preserved through html5lib if the tree builder preserves order.
  297. * Removed ``dom2sax`` from DOM treebuilders. It has been replaced by
  298. ``treeadapters.sax.to_sax`` which is generic and supports any
  299. treewalker; it also resolves all known bugs with ``dom2sax``.
  300. * Fix treewalker assertions on hitting bytes strings on
  301. Python 2. Previous to 1.0b1, treewalkers coped with mixed
  302. bytes/unicode data on Python 2; this reintroduces this prior
  303. behaviour on Python 2. Behaviour is unchanged on Python 3.
  304. 1.0b1
  305. ~~~~~
  306. Released on May 17, 2013
  307. * Implementation updated to implement the `HTML specification
  308. <http://www.whatwg.org/specs/web-apps/current-work/>`_ as of 5th May
  309. 2013 (`SVN <http://svn.whatwg.org/webapps/>`_ revision r7867).
  310. * Python 3.2+ supported in a single codebase using the ``six`` library.
  311. * Removed support for Python 2.5 and older.
  312. * Removed the deprecated Beautiful Soup 3 treebuilder.
  313. ``beautifulsoup4`` can use ``html5lib`` as a parser instead. Note that
  314. since it doesn't support namespaces, foreign content like SVG and
  315. MathML is parsed incorrectly.
  316. * Removed ``simpletree`` from the package. The default tree builder is
  317. now ``etree`` (using the ``xml.etree.cElementTree`` implementation if
  318. available, and ``xml.etree.ElementTree`` otherwise).
  319. * Removed the ``XHTMLSerializer`` as it never actually guaranteed its
  320. output was well-formed XML, and hence provided little of use.
  321. * Removed default DOM treebuilder, so ``html5lib.treebuilders.dom`` is no
  322. longer supported. ``html5lib.treebuilders.getTreeBuilder("dom")`` will
  323. return the default DOM treebuilder, which uses ``xml.dom.minidom``.
  324. * Optional heuristic character encoding detection now based on
  325. ``charade`` for Python 2.6 - 3.3 compatibility.
  326. * Optional ``Genshi`` treewalker support fixed.
  327. * Many bugfixes, including:
  328. * #33: null in attribute value breaks XML AttValue;
  329. * #4: nested, indirect descendant, <button> causes infinite loop;
  330. * `Google Code 215
  331. <http://code.google.com/p/html5lib/issues/detail?id=215>`_: Properly
  332. detect seekable streams;
  333. * `Google Code 206
  334. <http://code.google.com/p/html5lib/issues/detail?id=206>`_: add
  335. support for <video preload=...>, <audio preload=...>;
  336. * `Google Code 205
  337. <http://code.google.com/p/html5lib/issues/detail?id=205>`_: add
  338. support for <video poster=...>;
  339. * `Google Code 202
  340. <http://code.google.com/p/html5lib/issues/detail?id=202>`_: Unicode
  341. file breaks InputStream.
  342. * Source code is now mostly PEP 8 compliant.
  343. * Test harness has been improved and now depends on ``nose``.
  344. * Documentation updated and moved to https://html5lib.readthedocs.io/.
  345. 0.95
  346. ~~~~
  347. Released on February 11, 2012
  348. 0.90
  349. ~~~~
  350. Released on January 17, 2010
  351. 0.11.1
  352. ~~~~~~
  353. Released on June 12, 2008
  354. 0.11
  355. ~~~~
  356. Released on June 10, 2008
  357. 0.10
  358. ~~~~
  359. Released on October 7, 2007
  360. 0.9
  361. ~~~
  362. Released on March 11, 2007
  363. 0.2
  364. ~~~
  365. Released on January 8, 2007