Common code library for Amundsen

amundsen, data-catalog, data-discovery, linuxfoundation, metadata
pip install amundsen-common==0.27.1




Amundsen is a data discovery and metadata engine for improving the productivity of data analysts, data scientists and engineers when interacting with data. It does that today by indexing data resources (tables, dashboards, streams, etc.) and powering a page-rank style search based on usage patterns (e.g. highly queried tables show up earlier than less queried tables). Think of it as Google search for data. The project is named after Norwegian explorer Roald Amundsen, the first person to discover the South Pole.

LF AI & Data

Amundsen is hosted by the LF AI & Data Foundation. It includes three microservices, one data ingestion library and one common library.

  • amundsenfrontendlibrary: Frontend service which is a Flask application with a React frontend.
  • amundsensearchlibrary: Search service, which leverages Elasticsearch for search capabilities, is used to power frontend metadata searching.
  • amundsenmetadatalibrary: Metadata service, which leverages Neo4j or Apache Atlas as the persistent layer, to provide various metadata.
  • amundsendatabuilder: Data ingestion library for building metadata graph and search index. Users could either load the data with a python script with the library or with an Airflow DAG importing the library.
  • amundsencommon: Amundsen Common library holds common codes among microservices in Amundsen.
  • amundsengremlin: Amundsen Gremlin library holds code used for converting model objects into vertices and edges in gremlin. It's used for loading data into an AWS Neptune backend.
  • amundsenrds: Amundsenrds contains ORM models to support relational database as metadata backend store in Amundsen. The schema in ORM models follows the logic of databuilder models. Amundsenrds will be used in databuilder and metadatalibrary for metadata storage and retrieval with relational databases.




  • Python = 3.6 or 3.7
  • Node = v10 or v12 (v14 may have compatibility issues)
  • npm >= 6

User Interface

Please note that the mock images only served as demonstration purpose.

  • Landing Page: The landing page for Amundsen including 1. search bars; 2. popular used tables;

  • Search Preview: See inline search results as you type

  • Table Detail Page: Visualization of a Hive / Redshift table

  • Column detail: Visualization of columns of a Hive / Redshift table which includes an optional stats display

  • Data Preview Page: Visualization of table data preview which could integrate with Apache Superset or other Data Visualization Tools.

Get Involved in the Community

Want help or want to help? Use the button in our header to join our slack channel. Contributions are also more than welcome! As explained in there are many ways to contribute, it does not all have to be code with new features and bug fixes, also documentation, like FAQ entries, bug reports, blog posts sharing experiences etc. all help move Amundsen forward. If you find a security vulnerability, please follow this guide.

Getting Started

Please visit the Amundsen installation documentation for a quick start to bootstrap a default version of Amundsen with dummy data.

Architecture Overview

Please visit Architecture for Amundsen architecture overview.

Supported Entities

  • Tables (from Databases)
  • People (from HR systems)
  • Dashboards

Supported Integrations

Table Connectors

Amundsen can also connect to any database that provides dbapi or sql_alchemy interface (which most DBs provide).

Table Column Statistics

Dashboard Connectors

ETL Orchestration


Please visit Installation guideline on how to install Amundsen.


Please visit Roadmap if you are interested in Amundsen upcoming roadmap items.

Blog Posts and Interviews


  • Disrupting Data Discovery {slides, recording} (Strata SF, March 2019)
  • Amundsen: A Data Discovery Platform from Lyft {slides} (Data Council SF, April 2019)
  • Disrupting Data Discovery {slides} (Strata London, May 2019)
  • ING Data Analytics Platform (Amundsen is mentioned) {slides, recording } (Kubecon Barcelona, May 2019)
  • Disrupting Data Discovery {slides, recording} (Making Big Data Easy SF, May 2019)
  • Disrupting Data Discovery {slides, recording} (Neo4j Graph Tour Santa Monica, September 2019)
  • Disrupting Data Discovery {slides} (IDEAS SoCal AI & Data Science Conference, Oct 2019)
  • Data Discovery with Amundsen by Gerard Toonstra from Coolblue {slides} and {talk} (BigData Vilnius 2019)
  • Towards Enterprise Grade Data Discovery and Data Lineage with Apache Atlas and Amundsen by Verdan Mahmood and Marek Wiewiorka from ING {slides, talk} (Big Data Technology Warsaw Summit 2020)
  • Airflow @ Lyft (which covers how we integrate Airflow and Amundsen) by Tao Feng {slides and website} (Airflow Summit 2020)
  • Data DAGs with lineage for fun and for profit by Bolke de Bruin {website} (Airflow Summit 2020)
  • Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metadata Platform by Tao Feng (Data+AI summit Europe 2020)
  • Data Discovery at Databricks with Amundsen by Tao Feng and Tianru Zhou (Data+AI summit NA 2021)

Related Articles

Community meetings

Community meetings are held on the first Thursday of every month at 9 AM Pacific, Noon Eastern, 6 PM Central European Time. Link to join

Upcoming meetings & notes

You can the exact date for the next meeting and the agenda a few weeks before the meeting in this doc.

Notes from all past meetings are available here.

Who uses Amundsen?

Here is the list of organizations that are using Amundsen today. If your organization uses Amundsen, please file a PR and update this list.

Currently officially using Amundsen:

  1. Asana
  2. Bagelcode
  3. Bang & Olufsen
  4. Brex
  5. Cameo
  6. Chan Zuckerberg Initiative
  7. Cimpress Technology
  8. Coles Group
  9. Convoy
  10. Databricks
  11. Data Sprints
  12. Dcard
  13. Delivery Hero
  14. Devoted Health
  15. DHI Group
  16. Edmunds
  17. Everfi
  18. Gusto
  19. Hurb
  20. ING
  21. Instacart
  22. iRobot
  23. Lett
  24. LMC
  25. Loft
  26. Lyft
  27. Merlin
  28. PicPay
  29. Plarium Krasnodar
  30. PUBG
  31. Rapido
  32. REA Group
  33. Remitly
  34. Snap
  35. Square
  36. Tile
  37. WePay
  38. WeTransfer
  39. Workday


Thanks goes to these incredible people: