expanse

turn apple health export.xml into parquet


Keywords
apple, apple-health, applehealth, data-visualization, parquet
License
Apache-2.0
Install
pip install expanse==0.2.4

Documentation

Atlas

Atlas

Atlas lets you explore your Apple Health data.


PyPI Tests Changelog License

Installation

Install Atlas using pip:

pip install atlas-db

Upgrade Atlas using pip:

pip install atlas-db --upgrade

Uninstall Expanse (old name) using pipx:

pipx uninstall expanse

Explore

First we create the .parquet file from the export.xml file.

atlas parquet export.xml -o ah.parquet

We can explore the data in many ways.

It is just a table/dataframe/parquet file with 5 columns.

But here we'll use clickhouse local:

clickhouse local

Let's take a look at the table.

DESCRIBE TABLE `ah.parquet`
β”Œβ”€name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐
β”‚ type    β”‚ Nullable(String)        β”‚              β”‚                    β”‚         β”‚                  β”‚                β”‚
β”‚ start   β”‚ Nullable(DateTime64(6)) β”‚              β”‚                    β”‚         β”‚                  β”‚                β”‚
β”‚ end     β”‚ Nullable(DateTime64(6)) β”‚              β”‚                    β”‚         β”‚                  β”‚                β”‚
β”‚ created β”‚ Nullable(DateTime64(6)) β”‚              β”‚                    β”‚         β”‚                  β”‚                β”‚
β”‚ value   β”‚ Nullable(String)        β”‚              β”‚                    β”‚         β”‚                  β”‚                β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

What kind of "types" do we have and how many?

SELECT
    type,
    COUNT(*) AS count
FROM `ah.parquet`
GROUP BY type
ORDER BY count DESC
β”Œβ”€type───────────────────────────┬──count─┐
β”‚ ActiveEnergyBurned             β”‚ 879902 β”‚
β”‚ HeartRate                      β”‚ 451854 β”‚
β”‚ BasalEnergyBurned              β”‚ 289031 β”‚
β”‚ DistanceWalkingRunning         β”‚ 260500 β”‚
β”‚ StepCount                      β”‚ 217384 β”‚
β”‚ PhysicalEffort                 β”‚  69747 β”‚
β”‚ AppleExerciseTime              β”‚  61363 β”‚
β”‚ AppleStandTime                 β”‚  58309 β”‚
β”‚ EnvironmentalAudioExposure     β”‚  44535 β”‚
β”‚ SleepAnalysis                  β”‚  36599 β”‚
β”‚ WalkingStepLength              β”‚  28281 β”‚
β”‚ WalkingSpeed                   β”‚  28281 β”‚
β”‚ RespiratoryRate                β”‚  27829 β”‚
β”‚ AppleStandHour                 β”‚  25877 β”‚
β”‚ FlightsClimbed                 β”‚  22690 β”‚
β”‚ WalkingDoubleSupportPercentage β”‚  21900 β”‚
β”‚ WalkingAsymmetryPercentage     β”‚  13820 β”‚
β”‚ HeartRateVariabilitySDNN       β”‚  11961 β”‚
β”‚ OxygenSaturation               β”‚   4912 β”‚
β”‚ StairDescentSpeed              β”‚   4718 β”‚
β”‚ StairAscentSpeed               β”‚   4249 β”‚
β”‚ DistanceCycling                β”‚   2890 β”‚
β”‚ TimeInDaylight                 β”‚   2403 β”‚
β”‚ HeadphoneAudioExposure         β”‚   2323 β”‚
β”‚ RestingHeartRate               β”‚   1399 β”‚
β”‚ WalkingHeartRateAverage        β”‚   1176 β”‚
β”‚ DistanceSwimming               β”‚    455 β”‚
β”‚ SwimmingStrokeCount            β”‚    455 β”‚
β”‚ AppleSleepingWristTemperature  β”‚    442 β”‚
β”‚ RunningSpeed                   β”‚    391 β”‚
β”‚ VO2Max                         β”‚    366 β”‚
β”‚ RunningPower                   β”‚    173 β”‚
β”‚ DietaryCaffeine                β”‚    171 β”‚
β”‚ AppleWalkingSteadiness         β”‚    138 β”‚
β”‚ SixMinuteWalkTestDistance      β”‚    122 β”‚
β”‚ HeartRateRecoveryOneMinute     β”‚     76 β”‚
β”‚ RunningVerticalOscillation     β”‚     74 β”‚
β”‚ RunningGroundContactTime       β”‚     67 β”‚
β”‚ RunningStrideLength            β”‚     54 β”‚
β”‚ MindfulSession                 β”‚     34 β”‚
β”‚ HighHeartRateEvent             β”‚     18 β”‚
β”‚ AudioExposureEvent             β”‚     14 β”‚
β”‚ BodyMass                       β”‚     14 β”‚
β”‚ Height                         β”‚      5 β”‚
β”‚ Fatigue                        β”‚      1 β”‚
β”‚ HKDataTypeSleepDurationGoal    β”‚      1 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”˜

What's our total step count?

Note

The value column is type Nullable(String) so we have to cast toFloat64 to sum up the step values.

SELECT sum(toFloat64(value))
FROM `ah.parquet`
WHERE type = 'StepCount'
β”Œβ”€sum(toFloat64(value))─┐
β”‚              30295811 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

30.295.811 (30.29 million) steps. That's a lot of steps!

How to get the Apple Health export.xml file

group-figma-small

  • open the Apple Health app on iOS
  • tap on your profile picture (or initials) at the top right
  • tap on Export All Health Data
  • tap on Export
  • wait a few seconds to a few minutes (~3min for 10 years of data)
  • get the export.zip archive via Airdrop to a Mac (or save to Files)

Note

The export.xml file is in the export.zip archive.

You can expand the export.zip file by double-clicking on it.

This creates a directory named apple_health_export and in it is the export.xml file.

See: Apple Support on how to export Apple Health and Fitness in XML format

Usage

atlas parquet export.xml

Features

  • turn export.xml into a simple parquet file