Ego-Exo4D is a large-scale multi-modal multi-view video dataset (including 3D) and benchmark challenge. The dataset consists of time-synchronized videos of participants recorded with at least one first-person (egocentric Aria glasses) and third-person (exocentric GoPro cameras) perspective cameras.

• Please refer to the website, documentation, paper, blog post and video introduction.

Ego4D is the world's largest egocentric (first person) video ML dataset and benchmark suite, including over 3700 hours of annotated first-person video data.

• Please refer to the website, documentation or paper for more information.

• To access the data, please refer to the Documentation:
  • For Ego-Exo4D: refer to the Getting Started page.
  • For Ego4D: refer to the Start Here page.
• To download the data, refer to:
  • For Ego-Exo4D: Ego-Exo4D's Downloader CLI README
  • For Ego4D: Ego4D's CLI README
• Explore Ego4D or Ego-Exo4D here (you'll need a license): Ego4D Visualizer
• Read the Summary below for details about this repository.

The Ego4d repository (ego4d python module) includes:

• Ego-Exo4D Downloader CLI for the Ego-Ego4D dataset (available as the command egoexo)
• Ego4D Downloader CLI for the Ego4D dataset (available as the command ego4d)
• A simple API abstracting common video reading libraries (TorchAudio, PyAV),
• An API for feature extraction, which includes lightweight wrappers for common models, such as: Omnivore and SlowFast
• Notebooks (for Ego4D and Ego-Exo4D) serving as examples/tutorials to analyze & use the dataset
  • Colab notebooks for Ego4D serve as additional examples for the benchmarks (VQ, NLQ and STA) can be found on: https://ego4d-data.org/docs/challenge/
• Research code to train models on the dataset, e.g. clep
  • NOTE: baseline code for Ego-Exo4D is coming soon!
  • NOTE: baseline code for the Ego4D benchmarks exists on separate GitHub repositories, see the EGO4D organization and docs

Please see Structure of the Repository below for details.

Please follow the below instructions to setup the downloader CLI for Ego4d and to install the ego4d python module.

pip install ego4d --upgrade

NOTE: Please ensure you are on at least Python 3.10

Ensure you have cloned or downloaded the code to your local disk. All instructions assume you are the root of the repository.

Create a conda environment to enable pip installation:

conda create -n ego4d python=3.11 -y
conda activate ego4d

If you are using an existing conda (or pyenv) environment: please ensure you have installed at least Python 3.10.

pip install .  # run from the root of Ego4d

Now you should be able to import ego4d:

python3 -c 'import ego4d; print(ego4d)'

You can check that the ego4d module links to the correct file on your file system from the output of the above command.

The repository contains multiple directories covering a specific theme. Each theme contains an associated README.md file, please refer to them.

All python code is located in the ego4d and associated subdirectories. The goal for each subdirectory is to cover one specific theme.

• ego4d: the ego4d python module exists
  • cli: The Ego4D CLI for downloading the dataset
  • features: Feature extraction across the dataset
  • research: Everything related to research and usage of the dataset (dataloaders, etc).
    • research/clep: Contrastive Language Ego-centric video Pre-training
• viz: visualization engine

• For a demo notebook: Annotation Notebook
• For the visualization engine: Viz README

Ego4D is released under the MIT License.