Sinbad is a software library that makes it very easy to uniformly access online data sources provided in standard formats (CSV, JSON, XML).
Sinbad - (Automated Structure Inference and Binding of Data) ============================================================ Sinbad is a software library that makes it very easy to uniformly access online data sources provided in standard formats (CSV, JSON, XML). For more info, please visit: http://berry-cs.github.io/sinbad (Each of the examples below is intended to be run in Intermediate Student language level. In #lang racket define the structures #:transparent to inspect results.) Example 1 --------- Consider the following program, which connects to the FAA live feed of airport status information (XML format): (require sinbad) (define faa (sail-to "http://services.faa.gov/airport/status/ATL?format=application/xml" (cache-timeout 300) ; refresh every 5 minutes (manifest) ; display schema for available data upon load (load))) (define-struct port (name loc condition delay?)) (fetch faa (make-port "Name" "State" "Weather/Weather" "Delay")) Running this program should produce a value like: (make-port "Hartsfield-Jackson Atlanta International" "Georgia" "Fair" #false) Example 2 --------- The following program connects to the USGS live feed of earthquake events (JSON format): (require sinbad) (define Q (sail-to "http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson" (cache-timeout 180))) (define-struct quake (place time mag)) (load Q) ; can (re)load on demand anytime (fetch Q (make-quake "place" "time" "mag") (base-path "features/properties")) (fetch-first Q "features/properties") Note that the first fetch produces a list of instances of a *user-defined* quake structure. The second form produces an association list. Example 3 --------- An example of a very large data set (CSV + zip format): (require sinbad) (define vehicle-data-source (sail-to "http://www.fueleconomy.gov/feg/epadata/vehicles.csv.zip" (load) (manifest))) (define-struct auto (make model year trans city-mpg hwy-mpg)) ;; pull a random data record... (fetch-random vehicle-data-source (make-auto "make" "model" "year" "trany" "city08" "highway08")) ;; get all ~40,000 (!) of them... (define all-vehicles (fetch vehicle-data-source (make-auto "make" "model" "year" "trany" "city08" "highway08"))) (length all-vehicles) Have fun!