urlfetch

urlfetch is a simple, lightweight and easy to use HTTP client for Python. It is distributed as a single file module and has no depencencies other than the Python Standard Library.

Highlights

• Distributed as a single file module, has no depencencies other than the Python Standard Library.
• Pure Python, works fine with gevent.
• Small codebase, about 1000 lines and 30% are comments and blank lines. Only 10 minutes you can know every detail.
• Random user-agent.
• Support streaming.

Installation

$ pip install urlfetch

Hello, world

import urlfetch

response = urlfetch.get('http://python.org/')
print response.status, response.reason
print len(response.content)

Uploading files

import urlfetch

response = urlfetch.post(
    'http://127.0.0.1:8888/upload',
    headers = {
        'Referer': 'http://127.0.0.1/',
    },
    files = {
        'fieldname1': open('/path/to/file', 'rb'),
        #'fieldname2': 'file content', # file must have a filename
        'fieldname3': ('filename', open('/path/to/file2', 'rb')),
        'fieldname4': ('filename', 'file content'),
    },
    data = {
        'foo': 'bar'
    },
)

print response.status, response.content

Testing

To run the tests, urlfetch relies on bottle and gunicorn. If the tests are run by calling python setup.py test, the dependencies will be handled automatically (via tests_require). So, if you want to run the tests directly, that is, python tests/testall.py, make sure bottle and gunicorn are installed under the PYTHONPATH.