bqlib - BigQuery Python Library

BigQuery Python Library.

See: Google BigQuery

Requirements

Python 2.6 or later (not support for 3.x)

Setup

$ pip install bqlib

How to use

• Single Query - BQJob

  BQJob is a class for starting the BigQuery job and fetching the result.
  You can use either run_sync(synchronous) or run_async(asynchronous) method.

  from bqlib import BQJob
  
  project_id = 'example_project'
  query = 'SELECT foo FROM bar'
  http = authorized_http
  
  bqjob = BQJob(http, project_id, query=query)
  
  # run synchronously
  job_result = bqjob.run_sync()
  
  # or run asynchronously
  bqjob.run_async()
  # ... do other things ...
  job_result = bqjob.get_result()
  
  print job_result # [{u'foo': 10}, {u'foo': 20}, ...]

• Multiple Queries - BQJobGroup

  BQJobGroup is a class for putting multiple BQJobs into an one group.
  Each BQJob in that group are executed concurrently.

  from bqlib import BQJob, BQJobGroup
  
  bqjob1 = BQJob(http, project_id, query=query)
  bqjob2 = BQJob(http, project_id, query=query)
  
  job_group = BQJobGroup([bqjob1, bqjob2])
  # synchronously
  results = job_group.run_sync()
  # or asynchronously
  job_group.run_async()
  results = job_group.get_results()
  
  print results # [[{'foo': 10}, {'foo': 20}], [{'bar': 'test'}]]

How to test

$ wget http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
$ pip install -U setuptools
$ python bootstrap.py -v 1.7.1
$ bin/buildout
$ bin/py.test -v tests/bqlib_test.py

Note

• Concurrent Requests to BigQUery
  • Concurrent requests to BigQuery is restricted to 20 requests by Quota Policy.
  • If you want to set up concurrent requests to 20, you also have to set up at traffic controls in api-console page

License

This library is disributed as MIT license.

History

• 2013-10-22 bqlib 0.0.1
  • First release
• 2014-03-18 bqlib 0.0.2
  • Bug fixes