The pyModeS interface for OpenSky Impala database

Introduction

This Python library connects the pyModeS decoder and OpenSky-network raw Mode-S data. It aims at making the Enhance Mode-S information form OpenSky network more accessible for researchers.

The library can automatically retrieve and download data in rollcall_replies_data4 table from the OpenSky Imapala database, and then decodes several common Mode-S Comm-B message types. Currently, follows Mode-S downlink reports are supported:

Enhanced Mode-S:

• BDS40: Selected vertical intention report
• BDS50: Track and turn report
• BDS60: Heading and speed report

Mode-S meteorological information:

• BDS44: Meteorological routine air report
• BDS45: Meteorological hazard report

In addition, the library can also be used to query decoded ADS-B information from state_vectors_data4 table from the OpenSky Impala database.

To further explore the Mode-S decoding and aircraft trajectory processing using open-source Python libraries, you may have a look at:

• junzis/pyModeS: https://github.com/junzis/pyModeS
• xoolive/traffic: https://github.com/xoolive/traffic

Install

In order to successfully use this library, you need:

1. Get the ``pyModeS`` library

Install the up-to-date pyModeS version from PyPI:

$ pip install pyModeS --upgrade

Install this library:

$ pip install pymodes-opensky
or
$ pip install git+https://github.com/junzis/pymodes-opensky

2. Obtain access to OpenSky Impala database

Apply access at: https://opensky-network.org/data/impala. The user name and password will be used for the following configuration.

3. Configure OpenSky Impala login

The first time you use this library, the following configuration file will be created:

~/.config/pymodes_opensky/secret.conf

with the following content:

[default]
server = data.opensky-network.org
port = 2230
username =
password =

Fill in the empty username and password field with your OpenSky login.

Use the library

EHSHelper

The EHSHelper class allows the users to download and decode Enhanced Mode-S messages automatically.

To get the messages, the query requires an ICAO address (or a list of ICAO addresses), the start time, and the end time for the messages. By default, all BDS40, BDS50, and BDS60 messages are decoded. The results is represented in a pandas DataFrame.

An example is shown as follows:

from pymodes_opensky import EHSHelper

ehs = EHSHelper()

df = ehs.get(
    icao24="4844C6",
    start="2019-10-01 08:00:00",
    end="2019-10-01 08:10:00",
)

It is also possible to decode a subset of EHS message types, by specify the BDS codes using require_bds() function. For example:

ehs.require_bds(["BDS50", "BDS60"])

df = ehs.get(
    icao24="4844C6",
    start="2019-10-01 08:00:00",
    end="2019-10-01 08:10:00",
)

MeteoHelper

The MeteoHelper class allows the users to download and decoded meteorological messages automatically. By default it provides information from BDS44 messages. Information from BDS45 messages can also be enable with include45=Ture switch.

The interface is similar to EHSHelper, for example:

from pymodes_opensky import MeteoHelper

meteo = MeteoHelper()
df = meteo.get(
    icao24=["49d304", "4007f9"],
    start="2018-07-19 15:00:00",
    end="2018-07-19 15:10:00",
    include45=True,
)

OpenskyImpalaWrapper

All previous queries are based on the OpenskyImpalaWrapper class from the library. The wrapper class can also be used independently to query OpenSky Imapala database. It can be used for raw messages, as wells as decoded ADS-B data by OpenSky.

Be aware! The number of records can be massive without the ICAO filter. Thus the query can take a long time. To increase the query efficiency, please consider using a ICAO filter when possible.

By defined the query type as type="raw", the raw Mode-S message can be obtained. For example:

from pymodes_opensky import OpenskyImpalaWrapper

opensky = OpenskyImpalaWrapper()

# Perform a simple and massive query (~20k records for 1 second here!)
df = opensky.query(
    type="raw", start="2018-07-01 13:00:00", end="2018-07-01 13:00:01"
)

# Perform a query with ICAO filter
df = opensky.query(
    type="raw",
    start="2018-07-01 13:00:00",
    end="2018-07-01 13:00:10",
    icao24=["424588", "3c66a9"],
)

By switching the query type from type="raw" to type="adsb", you can obtained the history ADS-B information in a similar way. You can also add a boundary (with format of [lat1, lon1, lat2, lon2]) to the queries. For example:

from pymodes_opensky import OpenskyImpalaWrapper

opensky = OpenskyImpalaWrapper()

# Perform a simple and massive query (~25k records for 5 second here!)
df = opensky.query(
    type="adsb", start="2018-08-01 13:00:00", end="2018-08-01 13:00:10"
)

# Perform a query with ICAO address filter
df = opensky.query(
    type="adsb",
    start="2018-07-01 13:00:00",
    end="2018-07-01 13:00:10",
    icao24=["424588", "3c66a9"],
    bound=[30, -20, 65, 20],
)

More examples

More complete examples can be found in the test directory of this library.

Other information

If you find this project useful for your research, please consider citing the following works:

@inproceedings{sun2019integrating,
  title={Integrating pyModeS and OpenSky Historical Database},
  author={Sun, Junzi and Hoekstra, Jacco M},
  booktitle={Proceedings of the 7th OpenSky Workshop},
  volume={67},
  pages={63--72},
  year={2019}
}

@article{sun2019pymodes,
    title={pyModeS: Decoding Mode-S Surveillance Data for Open Air Transportation Research},
    author={J. {Sun} and H. {V\^u} and J. {Ellerbroek} and J. M. {Hoekstra}},
    journal={IEEE Transactions on Intelligent Transportation Systems},
    year={2019},
    doi={10.1109/TITS.2019.2914770},
    ISSN={1524-9050},
}

@inproceedings{schafer2014opensky,
  title={Bringing up OpenSky: A large-scale ADS-B sensor network for research},
  author={Sch{\"a}fer, Matthias and Strohmeier, Martin and Lenders, Vincent and Martinovic, Ivan and Wilhelm, Matthias},
  booktitle={Proceedings of the 13th international symposium on Information processing in sensor networks},
  pages={83--94},
  year={2014},
  organization={IEEE Press}
}