T-Digest provides a mergeable summary of a distribution, enabling approximate quantiles and CDF with strong tail accuracy. tdigest-rs delivers a production-ready Rust core with Python and Polars APIs plus Java (JNI), combining high performance, stable accuracy, and minimal memory overhead.

• 🦀 Single Rust core shared across Rust, Polars, Python, and Java
• 🚀 Mergeable digests for large / streaming data — fast union with consistent accuracy and guaranteed unique centroids
• 🔁 Cross-surface coherence: Consistent, verified behavior across all bindings
• ⚡ Quantile & CDF — optimized evaluation loops with half-weight bracketing and singleton-aware interpolation
• 🧠 Heap-stream k-way digest merge in Rust core for lower peak memory on large digest unions
• 🧵 Streaming two-way raw-ingest merge path in Rust core (centroids + values) to avoid extra merge buffers
• 🧊 TDigest Precision: Centroids as f64 or f32 — auto-selected by input dtype
• ⚖️ Weighted ingest across Rust/Python/Polars/Java (add_weighted, add_weighted_values, Java weighted adds)
• 🔄 Explicit precision casting across surfaces (cast_precision / castPrecision)
• 📦 TDIG v3 wire default (flags + header length + precision code + checksum), with v1/v2 decode compatibility
• 🧭 Explicit wire-version encode controls (to_bytes(version=1|2|3), toBytes(version))
• 🎚️ Scale families: Quad, K1, K2, K3
• 🔩 Singleton handling policy: edge-precision (keep N), respect singletons, or uniform merge

Apache-2.0

• Contributing guide: CONTRIBUTING.md
• Code of conduct: CODE_OF_CONDUCT.md
• Security policy: SECURITY.md
• Issue tracker: https://github.com/ingolfured/gr-tdigest/issues

make setup    # toolchains + Python deps
make build    # Rust lib+CLI, Python ext, Java classes (dev)
make test     # Rust + Python tests
make release  # release CLI + wheel + JARs

Release workflows are in .github/workflows/ and trigger on tags matching v*:

• release_pypi.yml
• release_cargo.yml
• release_maven.yml

Minimum GitHub setup:

1. PyPI (release_pypi.yml):

• Create GitHub environment pypi.
• Configure PyPI Trusted Publisher for this repo/workflow in PyPI.

2. Cargo (release_cargo.yml):

• Create GitHub environment crates-io.
• Add secret CARGO_REGISTRY_TOKEN.

3. Maven (release_maven.yml):

• Create GitHub environment maven.
• Add secrets MAVEN_REPOSITORY_URL, MAVEN_USERNAME, MAVEN_PASSWORD.
• Add MAVEN_SIGNING_KEY and MAVEN_SIGNING_PASSWORD if your Maven repository requires signed artifacts.

4. Release tag:

• Ensure Cargo.toml version equals the release tag without v (for example v0.2.3).
• Push tag: git tag v0.2.3 && git push origin v0.2.3

5. Repository protection (recommended):

• Apply rulesets from version-controlled specs:
  • ./scripts/apply_github_rulesets.sh
  • details: .github/REPO_SETTINGS.md

make publish publishes to PyPI, crates.io, and Maven from local credentials.

Dry run (recommended first):

PUBLISH_DRY_RUN=1 make publish

Real publish:

MATURIN_PYPI_TOKEN=... \
CARGO_REGISTRY_TOKEN=... \
MAVEN_REPOSITORY_URL=... \
MAVEN_USERNAME=... \
MAVEN_PASSWORD=... \
make publish

Optional Maven signing variables:

• MAVEN_SIGNING_KEY
• MAVEN_SIGNING_PASSWORD

Python

import gr_tdigest as td
d = td.TDigest.from_array([0,1,2,3], max_size=100, scale="k2")
print("p50 =", d.quantile(0.5))
print("cdf  =", d.cdf([0.0, 1.5, 3.0]))
d.add_weighted([10.0, 20.0], [2.0, 3.0])
blob_v1 = d.to_bytes(version=1)
d32 = d.cast_precision("f32")

Polars

import polars as pl
from gr_tdigest import tdigest, quantile

df = pl.DataFrame({"g": ["a"]*5, "x": [0,1,2,3,4]})
out = (
    df.lazy()
      .group_by("g")
      .agg(tdigest(pl.col("x"), max_size=100, scale="k2").alias("td"))
      .select(quantile("td", 0.5))
      .collect()
)
print(out)

Rust CLI

echo '0 1 2 3' | target/release/tdigest --stdin --cmd quantile --p 0.5 --no-header

Java (AutoCloseable)

import gr.tdigest.TDigest;
import gr.tdigest.TDigest.Precision;
import gr.tdigest.TDigest.Scale;
import gr.tdigest.TDigest.SingletonPolicy;

import java.util.Arrays;

public class Example {
  public static void main(String[] args) {
    try (TDigest digest = TDigest.builder()
        .maxSize(100)
        .scale(Scale.K2)
        .singletonPolicy(SingletonPolicy.EDGES).keep(4)
        .precision(Precision.F32)
        .build(new float[]{0, 1, 2, 3})) {
      double[] c = digest.cdf(new double[]{0.0, 1.5, 3.0});
      double p50 = digest.quantile(0.5);
    }
  }
}

├── src/                                  # Rust core, CLI entrypoint, algorithm modules
│   ├── bin/                              # Command-line app (tdigest CLI)
│   ├── tdigest/                          # Core T-Digest implementation (centroids, merge, scale)
│   └── quality/                          # Accuracy helpers & scoring utilities
├── bindings/                             # Language bindings
│   ├── python/                           # Python wheel (maturin)
│   │   ├── gr_tdigest/                   # Python package (abi3 native extension)
│   │   └── tests/                        # Python API + Polars tests
│   └── java/                             # Java API (Gradle project) + JNI shims
│       └── src/
│           └── gr/
│               └── tdigest/              # Public Java API + native bridge
├── integration/
│   └── api_coherence/                    # Cross-API contract tests (CLI ↔ Python ↔ Polars ↔ Java)
├── benches/                              # Rust benchmarks (quantile/CDF/codecs)
├── crates/
│   └── testdata/                         # Small datasets & fixtures for tests/benches
└── dist/                                 # Build artifacts (wheels/JARs) after release

• Rust: stable (2021 edition)
• Python: CPython 3.12; packaged with maturin
• Polars: current 1.x (Python); Rust crate versions tracked in Cargo.toml

• See CHANGELOG.md for release notes and unreleased changes.

• Allow scaling of weights and guard against centroid weight overflow
• Auto suggest a scaling function based on distribution