ETL2OSM

Extract, Transform and Load to OpenStreetMap

Features

• Extracts & Loads data from multiple formats:

  • Shapefile
  • GeoJSON
  • OSM
  • KML (Coming Soon)

• Replace direction fields (NE > Northeast, SW > Southwest)

• Replace street suffix fields (AVE > Avenue, ST > Street)
• Change & remove attribute names
• Convert text fields into proper titlecase (ottawa ONTARIO > Ottawa Ontario)
• Transform data into WGS84 (EPSG:4326)

API Overview

Step by Step doing a typical Extract Transform Load processing.

>>> import etl2osm
>>> data = etl2osm.extract("infile.shp")
>>> data.transform("config.json")
>>> data.save("outfile.osm")

Doing the entire process in a single line

>>> import etl2osm
>>> etl2osm.process("infile.shp", "config.json", "outfile.osm")

Command Line Interface

Reading the a file, the standard output will be in a GeoJSON format.

$ etl2osm "infile.shp" --config "config.json" --outfile "outfile.osm"

See Examples for more information.

Making a Config.json

Whenever you want to perform a transformation, include a config file in a JSON format.

Road Data

{
    "conform": {
        "addr:street": [
            {"function": "direction", "field":"DIRECTION"},
            {"function": "title", "field": "ST_NAME"},
            {"function": "suffix", "field": "ST_EXT"}
        ],
        "maxspeed": {"function": "mph", "field": "Speed_Limi"}
    }
}

Address Data

{
    "conform": {
        "addr:number": {"int": "True", "field": "NUMBER_"},
        "addr:street": [
            {"function": "direction", "field": "PREDIR"},
            {"function": "title", "field": "ST_NAME"},
            {"function": "suffix", "field": "STSUFFIX"}
        ],
        "addr:postcode": {"int": "True", "field": "ZIP_CODES"},
        "addr:unit": "UNIT"
    }
}

Installation

GDAL & Fiona

ETL2OSM depends on GDAL, Fiona & LXML; to install those packages using Windows. Download the appropriate library from Unofficial Windows Binaries for Python Extension Packages matching your Python version (Python27/34) & 64/32bit.

$ cd packages
$ pip install lxml‑3.4.4‑cp27‑none‑win32.whl
$ pip install Fiona-1.6.2-cp27-none-win32.whl
$ pip install GDAL-1.11.3-cp27-none-win32.whl

PyPi Install

Install ETL2OSM from PyPi's online packages.

$ pip install etl2osm

GitHub Install

Install the latest version directly from Github.

$ git clone https://github.com/osmottawa/etl2osm/
$ cd etl2osm
$ pip install .