htmls
Usage
Requirements
Htmls requires lxml <http://lxml.de/>. lxml requires libxml2 and libxslt,
so you have to install those libraries. You do not need to install lxml,
htmls will pull it in as a dependency.
Tested with Python 3.8 and 3.10.
Tip for Mac OSX
How to install libxml2 and libxslt is described in their docs, but their docs does not mention that libxml2 and libxslt is available in Homebrew for OSX. You can install them with::
$ brew install libxml2 libxslt
Install
When you have install libxml2 and libxslt, you can install htmls with::
$ pip install htmls
Usage example
import unittest
import htmls
class ExampleTest(unittest.TestCase):
def test_example(self):
selector = htmls.S("""
<html>
<body>
<a href="#hello">Hello</a>
<a href="#cruel" class="btn btn-default btn-lg">
Cruel
World
</a>
</body>
</html>
""")
# Various ways of finding elements
self.assertEqual(selector.count('a'), 2)
button = selector.one('a.btn') # Single htmls.Element
with self.assertRaises(htmls.NotExactlyOneMatchError):
button = selector.one('a') # Fails because the a selector matches 2 elements.
buttons = selector.list('a.btn') # List of htmls.Element
# Debugging
print selector.prettify() # Or selector.prettyprint()
print button.prettify() # Or button.prettyprint()
# Find the text of an element with normalized whitespace
self.assertEqual(selector.one('a.btn').text_normalized, 'Cruel World')
# Working with CSS classes
self.assertTrue(selector.one('a.btn').hasclass('btn-default'))
self.assertTrue(selector.one('a.btn').hasclasses(['btn-default', 'btn-lg']))
self.assertFalse(selector.one('a.btn').hasclasses(['btn-default', 'btn-lg', 'invalidclass']))
# Working with attributes
self.assertEqual(selector.one('a.btn')['href'], '#cruel')
self.assertTrue(selector.one('a.btn').hasattribute('href'), '#cruel')The example above is also available in tests/test_example.py.
Develop
Use conventional commits for GIT commit messages
See https://www.conventionalcommits.org/en/v1.0.0/. You can use this git commit message format in many different ways, but the easiest is:
- Use commitizen: https://commitizen-tools.github.io/commitizen/commit/
- Use an editor extension, like https://marketplace.visualstudio.com/items?itemName=vivaxy.vscode-conventional-commits for VScode.
- Just learn to write the format by hand (can be error prone to begin with, but it is fairly easy to learn).
Install hatch and commitizen
NOTE: You only need hatch if you need to build releases, and you only need commitizen for releases OR to make it easy to follow conventional commits for your commit messages (see Use conventional commits for GIT commit messages above).
First install pipx with:
$ brew install pipx
$ pipx ensurepath
Then install hatch and commitizen:
$ pipx install hatch
$ pipx install commitizen
See https://github.com/pypa/pipx, https://hatch.pypa.io/latest/install/ and https://commitizen-tools.github.io/commitizen/ for more install alternatives if needed, but we really recommend using pipx since that is isolated.
Install development dependencies
Install a local python version with pyenv:
$ pyenv install 3.10
$ pyenv local 3.10
Create virtualenv:
$ ./recreate-virtualenv.sh
Alternatively, create virtualenv manually (this does the same as recreate-virtualenv.sh):
$ python -m venv .venv
Install dependencies in a virtualenv:
$ .venv/bin/pip install -v ".[dev,test]"
Tests
$ pytest
Documentation
We have no HTML API docs at this time. This is a very small library,
it is less than 200 lines of well documented code. So just read htmls.py,
or use pydoc htmls after you have installed the library.
Also check out the code in tests/test_htmls.py. We have 100% test coverage,
and the tests are good examples.
How to release htmls
Release (create changelog, increment version, commit and tag the change) with:
$ cz bump
$ git push && git push --tags
NOTE:
-
cz bumponly works if conventional commits (see section about that above) is used. -
cz bumpcan take a specific version etc, but it automatically select the correct version if conventional commits has been used correctly. See https://commitizen-tools.github.io/commitizen/. - The
czcommand comes fromcommitizen(install documented above).
Release to pypi:
$ hatch build -t sdist
$ hatch publish
$ rm dist/* # optional cleanup
