Sparse, provenance-bearing conversational evidence recovery for Pi sessions.


Keywords
pi-package, rarebit, pi, session
License
MIT
Install
npm install @hypercarrier/rarebit@0.2.0

Documentation

Rarebit

@hypercarrier/rarebit recovers decision-bearing conversational evidence from one persisted Pi Session. It deterministically selects readable user messages and assistant continuation or stop prose on the active branch. It excludes tool payloads, tool results, and hidden reasoning.

Rarebit is public alpha software. Its CLI output, exports, sidecar protocol, and visual language can change. It is not a Task, Project, runtime, priority, attention, or delivery authority. Native Pi Session JSONL remains the evidence authority.

Rarebit is released from an immutable version tag by .github/workflows/publish.yml. The workflow runs the package gates, verifies one packed artifact, scans that artifact, and publishes it with npm provenance. Install the exact version required by your Pi host.

The old alpha.1, alpha.2, and alpha.3 tag graphs remain public and are not privacy-clean. Existing package versions, tags, and releases remain immutable. The source-only candidate derivation and vendored-scanner binding is release/privacy-lineage.v1.json. It is excluded from npm and does not claim publication completion.

Install

Use Node 22 or later and Pi 0.83 or later. Install it in Pi from npm:

pi install npm:@hypercarrier/rarebit@0.2.0
pi list

For a local checkout, Pi loads a directory without copying it:

pi install /absolute/path/to/rarebit
pi list

The package imports Pi's bundled @earendil-works/pi-ai as a peer dependency. Do not install or bundle a second Pi core. Pi packages execute with your user permissions, so review source before installation.

To use the CLI in a normal Node project, install it and invoke the local bin with npm:

npm install @hypercarrier/rarebit@0.2.0
npm exec -- rarebit --help

First query and extract

Give the CLI an exact Session JSONL path or supported Pi Session identifier. query returns metadata only. extract returns selected raw Session prose.

npm exec -- rarebit query --session /absolute/path/to/session.jsonl --json
npm exec -- rarebit extract --session /absolute/path/to/session.jsonl --json

The Pi extension adds one /rarebit parent command. In the TUI, the bare command opens an action palette. /rarebit settings opens a tabbed editor with Actions, Summary, Recap, and Session sections. Choose global settings or a trusted Project override; each field shows its effective value and source. Edits require Save confirmation. Blank input or Remove override restores inheritance. Settings apply to future operations; disabling automatic Recap also clears its pending or visible widget.

Direct help, status, settings, config, auto-title, title, recap, summarize, recall, and fork subcommands remain available. Use /rarebit settings global or /rarebit settings project to choose the editing scope directly. recap reads the current Summary receipt and renders the complete multiline Summary in the TUI. It never starts synthesis.

/rarebit fork validates the newest contiguous Rarebit suffix, writes a new Session in Pi's Session store for the current working directory, and switches to it without starting a model turn. Use /rarebit fork --max-token-length 64000 to set the imported prose budget. Imported messages keep their roles and source outcomes, use zero usage, and carry machine-only lineage. The generated opening message records source ID/path/leaf, target directory, coverage, and the read-only PiQ recovery command. Legacy source entries retain (sessionId, sourceOrder) when native entry IDs are absent; Rarebit does not migrate those sources.

The CLI performs the same operation without changing the caller's Session:

rarebit fork /absolute/source.jsonl
rarebit fork /absolute/source.jsonl --max-token-length 64000 --no-launch

The first form launches Pi in the invocation working directory. The second returns JSON for automation. Fork mode requires an installed Pi coding-agent peer; read-only query, extract, and PiQ commands do not. Use piq entries --session <path> to read omitted native evidence without writing or migrating the source.

Model setup and optional derivations

Summary and Title use a dedicated configured model. They never inherit Pi's interactive defaultModel. Start with the example config. Merge its rarebit object into global Pi settings (~/.pi/agent/settings.json, or $PI_CODING_AGENT_DIR/settings.json when set), or a trusted project's .pi/settings.json. Preserve other settings and select a model available through your authenticated provider. The example includes all Pi extension settings with their defaults; its model is an example choice.

The configuration shape is:

{
  "rarebit": {
    "model": "provider/model",
    "min_total_length": 80000,
    "max_rarebit_ratio": 0.4,
    "auto_title": true,
    "max_input_tokens": 64000,
    "summary_prompt": "The summary is free-form prose. State when evidence is uncertain, confusing, contradictory, or importantly missing instead of inventing a coherent account.",
    "diagnostics": {
      "summary_triggered": false,
      "summary_updated": false
    },
    "recap": {
      "enabled": true,
      "delay_ms": 60000,
      "timezone": "host"
    }
  }
}

After a successful automatic or explicit Summary materialization, the Pi TUI offers the current Summary above the editor after one minute. The offer stays visible while you type and clears when Pi sends input, starts a new turn, changes Session or branch, or shuts down. Set rarebit.recap.enabled to false to disable the offer, or change rarebit.recap.delay_ms to adjust the delay. Set rarebit.recap.timezone to host or an IANA time zone such as Asia/Hong_Kong to format the Recap header. The header identifies the host zone or selected IANA zone and its offset at the observation time. The widget reads an existing receipt and never adds a Session message. In the Pi extension, max_input_tokens limits the Summary prompt by the existing ceil(prompt characters / 4) estimate. It defaults to 64,000 estimated input tokens and is not a provider output limit. Summary trigger and update notifications stay off unless rarebit.diagnostics.summary_triggered or rarebit.diagnostics.summary_updated is enabled.

rarebit.summary_prompt is one scalar guidance string. It uses the default guidance shown in the example when omitted. The editor accepts multiline text, preserves its line breaks, and trims only outer whitespace. A blank value removes the override. The fixed evidence and JSON status contract remains owned by Rarebit. Keep custom guidance to a few short bullets so it leaves room for evidence in the configured Summary input budget. The editor rejects unsafe control characters and guidance longer than 64,000 UTF-16 characters. This guidance bound is separate from the 8,000-character Summary output safety limit.

Then request explicit model work when you want it:

npm exec -- rarebit summarize --session /absolute/path/to/session.jsonl --json --force
npm exec -- rarebit title --session /absolute/path/to/session.jsonl --json

A Summary is a lossy assessment of an identified selection. It can report only Session-scoped appearance, not completed Project or Task work. A Title is a mutable label proposal, not Session identity. Automatic Summary work runs only at persisted direct-input or settled-agent boundaries and only when its policy permits it.

Privacy, local data, and provider egress

Rarebit reads Pi Session JSONL locally and does not modify it. extract prints selected raw prose, so handle its output as sensitive.

Summary and Title send selected Rarebit prose to the provider and model that you configure. When input exceeds the fixed limit, Rarebit sends a newest suffix with an explicit omission marker. It does not send tool inputs, tool results, hidden reasoning, provider credentials, or HTTP headers as Rarebit content.

Derived receipts live below ~/.pi/agent/rarebit/materializations-v4/; job leases live below ~/.pi/agent/rarebit/jobs-v4/. These directories use mode 0700 and their files use mode 0600. Receipts retain compact metadata but no selected prose, prompt, provider response body, headers, or credentials. They remain until you remove them. Session JSONL retention is controlled by Pi.

First Recall

Run /rarebit recall <prompt> in a Pi Session. Rarebit writes the exact active branch selection to two private files: a conversation view and detailed lineage evidence. It then sends one atomic, human-readable Markdown user message containing your exact request and absolute pointers to both files.

When Pi is idle, that message starts one turn. When Pi is busy, it queues one steering message. Recall does not send a second follow-up, add a custom Session entry, or create a durable Recall receipt.

The temporary directory is mode 0700 and its JSON files are mode 0600. The files remain after the request so Pi can read them; delete the directory after the turn no longer needs it. Both files and the persisted Pi message are sensitive. Do not put their paths or content in a ticket.

Update, uninstall, and rollback

Pin a version for reproducible installs:

pi install npm:@hypercarrier/rarebit@0.2.0
pi remove npm:@hypercarrier/rarebit

The exact-version install also moves an existing Rarebit npm installation to that pin. Pi skips versioned npm sources during package updates. Restart Pi after installation to load the new extension code.

To roll back, reinstall a known version with pi install npm:@hypercarrier/rarebit@<version>. Removal stops package loading. It does not alter Pi Session JSONL or delete Rarebit sidecars, job leases, or Recall temp files. Remove retained local data only after you review it.

Compatibility and support

Rarebit supports Node 22+ and Pi 0.83.0 or later. Its Pi AI peer has no upper bound. The release gate runs Recall against Pi 0.83.0 and Pi 0.84.2. It works as a deterministic CLI without a model. Summary, Title, and the Pi extension require a compatible Pi installation and configured provider credentials.

Report security issues privately as described in SECURITY.md. Use https://github.com/deephbz/rarebit/issues for normal support. Include no Session prose, credentials, or Recall files in public reports.

See RUNBOOK.md for recovery and sidecar details, and VISUAL-LANGUAGE.md for evidence-mark meaning.