stedsans: A Geoparsing Toolkit for Pythonk

Exam Project for Spatial Analysis - Cultural Data Science Elective 2021

Oveview

Description | Demonstration | Installation | Usage | References | Metadata | Contact

Description

stedsans is an innovative Danish and English geoparsing toolkit for Python that facilitates efficient retrieval of the geospatial information embedded in textual data. The package is designed to handle textual input ranging from a single sentence to a large corpus. To broaden the appeal of the package and further set it apart from existing software, we decided to integrate user-friendly functions for visualising and statistically analysing the extracted geospatial data. stedsans neatly compresses highly complex workflows into a single package with predefined functionalities that are tailored to the most common research questions and is aimed at the novice to intermediate geospatial analyst who has basic familiarity with coding and quantitative analyses. stedsans utilizes Transformer-based models (Vaswani et al, 2017), including the Danish Ælæctra and the English BERT fine-tuned for Named Entity Recognition, to geotag entities. The GIS software Nominatim (v3.7.2) is used as the geocoding engine. Nominatim obtains data from the OpenStreetMap database (OpenStreetMap contributors, 2017). See metadata tables for licenses and documentation.

Demonstration

For a demonstration of the current tools, we strongly suggest that use the following Google Colab notebook. This provides a guided demonstration of the key functionalities of the package:

Installation

stedsans is developed and tested on Python 3.7+.

Windows

Since stedsans requires the package geopandas, and the dependencies can cause several issues when using Windows, it is recommended to first install Anaconda, which comes with pre-built binaries for geopandas.

After having installed Anaconda, create a conda environment and then install stedsans using pip and geopandas using conda.

pip install stedsans

conda install geopandas==0.9.0

MacOS and Linux

For MacOS and Linus stedsans and geopandas can be installed directly by using pip.

pip install stedsans

pip install geopandas==0.9.0

Usage

To use stedsans start by importing stedsans.

from stedsans import stedsans

We can then initialize a stedsans instance on a pre-defined example text, and print the extracted entities.

>>> example_text = "Hello my name is Malte and i live in Aarhus C. I love watching Randers FC, a football team from Randers, beat Brøndby IF, a football team from the devils island of Sjælland."

>>> my_stedsans = stedsans(example_text, language="english")

>>> my_stedsans.print_entities()

[   ('Aarhus C', 'LOC'),
    ('Randers FC', 'ORG'),
    ('Randers', 'LOC'),
    ('Brøndby IF', 'ORG'),
    ('Sjælland', 'LOC')]

These locations can then be vizualised using the plot_coordinates() function, and we can specify it to limit the locations to only coming from Denmark. This results in an interactive map.

>>> my_stedsans.plot_locations(limit="country", limit_area="Denmark")

You can also plot an interactive heatmap with plot_heatmap().

>>> my_stedsans.plot_heatmap(limit="country", limit_area="Denmark")

stedsans provides lots of other features and tools and these are very thoroughly demonstrated in the Google Colab (if it wasn't obvious already - we really want you to use the colab😀):

References

OpenStreetMap contributors. (2017). Planet dump retrieved from https://planet.osm.org [Retrieved on 09-07-2021]

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is All you Need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 30 (pp. 5998–6008). Curran Associates, Inc. http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

Metadata

Metadata for software

Metadata for data

Contact

For help or further information feel free to connect with either of the main developers:

Malte Højmark-Bertelsen
hjb@kmd.dk

Jakob Grøhn Damgaard
bokajgd@gmail.com