@dhttp/bridge

dhttp bridge: serves RPC commands over HyperDHT and re-originates them as HTTP or WebSocket against one endpoint.


License
Apache-2.0
Install
npm install @dhttp/bridge@0.0.0-rc.1

Documentation

dhttp

A bridge between HyperDHT and HTTP. It runs as a sidecar next to a webserver and makes it reachable by public key.

It is a bridge rather than a proxy: a proxy forwards the protocol it receives, whereas this terminates one protocol — RPC over HyperDHT — and re-originates the traffic as HTTP or WebSocket against one configured upstream. Only the meaning of a request crosses; no bytes are forwarded verbatim.

Based on hello-pear-bare, so it is a standalone Bare process with peer-to-peer OTA updates via pear-runtime, running in a worker thread of their own.

The webserver is identified by its public key, and clients dial that key directly on the DHT — there is no discovery topic and no swarm.

What it does

  • Listens on the DHT under a stable keypair.
  • Speaks bare-rpc over the connection — the same RPC Pear itself uses — and answers three commands:
    • INFO — service description (name, upstream, allowed methods, limits)
    • REQUEST — one HTTP round trip, response streamed with flow control
    • WEBSOCKET — a WebSocket to the upstream, terminated at the bridge and relayed as whole messages
  • Never buffers a response. Whether a client wants the body whole or as a stream is decided on the client, not on the wire.
  • Requests are forwarded to a single pre-configured upstream, e.g. https://api.example.com. Nothing outside that origin and path prefix is reachable through the bridge.
client ──dht.connect(417fe9...)──▶ bridge ──http(s)/ws──▶ server
       ◀──────── response ────────        ◀── response ──

Each server runs its own bridge with its own key; nothing is shared between them.

bare-rpc frames the stream itself and multiplexes requests by id, so one connection carries any number of concurrent requests and streams.

Hyperswarm is used for one thing only: when OTA updates are enabled the updater replicates its drive over a swarm topic. That happens in a worker thread, so replicating and verifying an update is never work the thread answering requests has to do; the price is a DHT node over there as well, since a node cannot be shared across threads. With --no-updates the worker is never started.

Requirements

  • npm via Node.js
  • pearnpx pear (only needed for deployment/updates)

Usage

npm install
npm start -- --upstream https://api.example.com

Output:

Updates: disabled
Upstream: https://api.example.com/
Advertising as: dhttp

Bridge key: 417fe94ac099cc08140ef78ae63be8d3c9402151d155d23b69a284cfcbc1ba05
Clients reach this bridge with:
  npm run hurl -- hyper://417fe94ac099cc08140ef78ae63be8d3c9402151d155d23b69a284cfcbc1ba05/v1/info

Hand out that key — it is the server's address on the DHT.

hurl

hurl is curl for hyper URLs: hyper://<key>/<path> addresses a path on the bridge's upstream. It is its own package with its own binary, and doubles as the worked example of using the client library.

npm run hurl -- hyper://417fe9.../v1/items                       # GET
npm run hurl -- -d '{"name":"thing"}' hyper://417fe9.../v1/items  # POST
npm run hurl -- -i -s hyper://417fe9.../v1/large                 # stream, with head
npm run hurl -- -w -d '{"hello":"world"}' hyper://417fe9.../v1/ws # websocket
npm run hurl -- --info hyper://417fe9.../                        # who is this?
Option Meaning
-X, --request <verb> HTTP method. Defaults to GET, or POST when -d is given.
-d, --data <body> Request body; sets content-type: application/json.
-H, --header <h: v> Extra request header, repeatable.
-i, --include Print the response head.
-s, --stream Stream the response instead of buffering it.
-w, --websocket Open a WebSocket; -d is sent, then messages are printed.
--info Print the bridge's service description.

Build it as a standalone binary — no runtime, nothing to install:

npm run make:hurl              # host platform, into packages/hurl/out/<host>
./packages/hurl/out/darwin-arm64/hurl hyper://417fe9.../v1/items

Flags

Flag Default Description
--upstream, -u <url> — (required) HTTP endpoint to bridge to. Also read from HTTP_BRIDGE_UPSTREAM.
--seed <hex|phrase> generated once Identity seed. Also read from HTTP_BRIDGE_SEED. See Identity.
--name <name> dhttp Name reported by info.
--methods <list> GET,HEAD,POST,OPTIONS Comma separated allowed HTTP methods.
--allow-paths <list> all Comma separated path prefixes, e.g. /v1/.
--timeout <ms> 30000 Time to first byte, then time between chunks.
--no-websocket Refuse WebSocket tunnels.
--storage <dir> platform default Storage directory.
--no-updates Disable OTA updates for this run (npm start sets it).

Everything else the bridge enforces — request body size, in-flight requests per peer, open sockets, message size — is a safety limit rather than a tuning knob, so it has a default and no flag. Embed the bridge API if you need to move one.

Identity

The bridge's key is derived from a 32 byte seed:

  • By default a seed is generated on first run and stored at <storage>/identity.json (mode 0600). Keep that file and the key is stable.
  • --seed <64 hex> uses that seed directly, --seed <passphrase> hashes the passphrase into one. Either way nothing is written to disk, so the same seed reproduces the same key anywhere — useful for redeploying a bridge on a different host.

Treat the seed like a private key: whoever holds it can impersonate the bridge.

Example: expose one API, read and write, nothing else

npm start -- \
  --upstream https://api.example.com \
  --name "example api" \
  --allow-paths /v1/ \
  --methods GET,POST

A client reaches it by key:

npm run hurl -- hyper://417fe9.../v1/items

Protocol

Transport: a HyperDHT connection to the bridge's public key (dht.connect(publicKey); the sidecar runs dht.createServer().listen(keyPair)).

RPC: bare-rpc (librpc ABI), self-framing, requests multiplexed by id. Commands are numbered — INFO = 1, REQUEST = 2, WEBSOCKET = 3 — and only ever appended to.

Two conventions carry the whole protocol:

  1. A command's payload is a request frame — method, path, headers, body. A WebSocket open is one of those with method GET and no body, sent as the first frame of the request stream (bare-rpc lets a request carry inline data or a stream, not both).
  2. A response is a stream whose first frame is a head — status, status message, headers — and whose remaining frames are the body: chunks for a request, whole messages for a WebSocket.

INFO is the exception: no request frame, and a plain value in reply, since it is asked before anything else is agreed.

The protocol string reported by info is dhttp/1.

info request is empty, the response is JSON:

{
  "service": "dhttp",
  "protocol": "dhttp/1",
  "name": "example api",
  "version": "0.0.0-rc.0",
  "publicKey": "417fe94ac099cc08140ef78ae63be8d3c9402151d155d23b69a284cfcbc1ba05",
  "upstream": "https://api.example.com/",
  "methods": ["GET", "POST"],
  "allowedPaths": ["/v1/"],
  "maxBodySize": 1048576,
  "timeout": 30000,
  "maxSockets": 8,
  "maxMessageSize": 1048576,
  "commands": ["INFO", "REQUEST", "WEBSOCKET"]
}

Both frames are compact-encoding structs (see packages/protocol/lib/messages.js):

// request
{ method: string, path: string, headers: [[name, value]], body: buffer | null }

// head — first frame of every response
{ status: uint, statusMessage: string, headers: [[name, value]] }

Error semantics — the bridge behaves like a gateway, so almost everything is a status rather than an RPC error:

  • Refused by the bridge405 method not allowed, 403 path escapes the upstream or falls outside --allow-paths, 413 request body too large, 429 too many in-flight requests, 400 malformed path. The body is { error, code } as JSON.
  • Upstream failed (connection refused, timeout) — 502, or 504 on a timeout, with the same JSON body shape.
  • Upstream answered — status, headers and body are relayed verbatim, including 4xx/5xx from the upstream itself.

RPC errors are reserved for protocol faults — an unknown command, a frame that will not decode. Those reject the call.

WebSockets

WEBSOCKET uses a bidirectional stream pair on one request id: a request frame out, a head frame back — 101 on success, or a refusal status — and whole messages after that in both directions.

The bridge terminates the real WebSocket with bare-ws: it runs the upgrade handshake against the upstream, answers pings, reassembles fragments, and relays message payloads. Clients need no WebSocket implementation of their own. Messages are relayed to the upstream as text frames, which is what JSON protocols over WebSockets use.

Client library

@dhttp/client wraps one DHT connection. It depends on nothing the bridge needs — no HTTP stack, no DHT server, no Pear runtime — so a client app can take it on its own:

npm install @dhttp/client
const DHT = require('hyperdht')
const b4a = require('b4a')
const HttpBridgeClient = require('@dhttp/client')

const key = b4a.from('417fe94ac0...', 'hex') // the bridge's key

const dht = new DHT()
const stream = dht.connect(key)
const client = new HttpBridgeClient(stream)

console.log(await client.info())

const res = await client.get('/v1/items')
console.log(res.status, res.json())

const created = await client.post('/v1/items', { name: 'thing' })
console.log(created.status, created.ok)

client.destroy()
await dht.destroy()

request() buffers the response for you, refusing anything over its own maxBodySize (1 MiB unless you pass one). For a response you do not want in memory at all, stream() resolves as soon as the head arrives and hands back a paused Readable:

const res = await client.stream('GET', '/v1/large')

console.log(res.status, res.headers)

for await (const chunk of res.body) {
  // Backpressure runs all the way back to the upstream socket: stop reading
  // and the bridge stops pulling.
}

res.body.destroy() // hangs up early, aborting the upstream request

For a subscription or any other push protocol, websocket() resolves once the upstream handshake has succeeded and hands back a WebSocket-shaped object:

const socket = await client.websocket('/v1/ws')

socket.addEventListener('message', (event) => {
  const update = JSON.parse(event.data)
})

socket.send(JSON.stringify({ subscribe: 'items' }))

socket.close()

It implements send / close / readyState / addEventListener and the onmessage, onopen, onclose, onerror properties, so a library that takes an injected WebSocket implementation can use it as is — or assign a bound factory to globalThis.WebSocket in a Pear app. Either way the client never learns the upstream's URL.

Requests are multiplexed by id, so one connection can carry many buffered calls, streams and WebSockets at once — HttpBridgeClient holds no per-request state.

For a library written against fetch that lets you swap the transport out, client.fetch() hands back a fetch-shaped function over that one client — a status is never a rejection, a transport failure is a TypeError, a body is read once, and headers behaves like Headers:

const fetch = client.fetch()

const res = await fetch('https://service.invalid/v1/items')

if (res.ok) console.log(await res.json())

Bridge API

@dhttp/bridge is a ready-resource: construct it with a DHT node and a keypair, await bridge.ready(), and await bridge.close() when done.

const HttpBridge = require('@dhttp/bridge')

const bridge = new HttpBridge({
  dht, // a hyperdht node
  keyPair, // DHT.keyPair(seed)
  upstream: 'https://api.example.com',
  allowedPaths: ['/v1/'], // default: all
  methods: ['GET', 'POST'], // default: GET, HEAD, POST, OPTIONS
  maxBodySize: 1024 * 1024, // largest request body forwarded
  timeout: 30000,
  maxConcurrency: 32,
  websocket: true,
  maxSockets: 8,
  maxMessageSize: 1024 * 1024
})

await bridge.ready()

Events:

Event When
listening (publicKey) the DHT server is announced
peer-add / peer-remove a client connection opened or closed
peer-error (err, stream) a connection failed; routine hangups included
request (info) one request finished, whatever the outcome
socket-open / socket-close a WebSocket tunnel opened or closed

A request event carries { method, path, status, websocket, error, elapsed, remotePublicKey }. status is 0 when the upstream never answered. bridge.stats counts connections, requests, sockets, failed and rejected.

Security notes

  • Only the configured upstream origin is reachable. Paths are resolved against the upstream URL and rejected if they leave its origin or base path, so //evil.example.com/x and /v1/../admin are refused.
  • Request headers are allow-listed (accept, accept-language, content-type, authorization, x-request-id). Hop-by-hop headers are stripped in both directions and host is set by the bridge.
  • Request bodies are buffered in memory and capped (1 MiB), and each peer is limited to 32 in-flight requests. Responses are never buffered by the bridge, so nothing there is capped.
  • WebSockets get their own budget — eight open at a time, 1 MiB per message — since they are long lived, and --no-websocket refuses them outright. Their paths go through the same allow list.
  • There is no peer authentication: anyone who knows the key can use the bridge. Since the key is not announced on any topic, it is only as discoverable as you make it. Use --allow-paths if that matters, and put the upstream's own auth in front of anything sensitive.
  • Clients authenticate the bridge, not the other way around: the connection is encrypted and bound to the bridge's public key, so dialling a key you trust cannot be answered by anyone else.

Limitations

  • Request bodies are buffered and capped; only responses stream. API payloads are typically small JSON, so this has not been worth the second state machine.
  • A buffered request() holds the whole response in the client's memory, capped by its own maxBodySize (1 MiB by default). Use stream() for anything larger.
  • WebSocket messages are relayed as text. bare-ws drops the text/binary opcode when it hands up a payload, so a binary WebSocket protocol would need a flag on the wire. JSON protocols are unaffected.
  • Reconnection is the client's job: if the DHT connection drops, open sockets close with it.
  • One upstream per process.

Architecture

The repo is an npm workspace holding three packages plus the app that runs the bridge:

Package Depends on For
@dhttp/protocol compact-encoding, b4a command ids, wire encodings, framing
@dhttp/client bare-rpc, bare-events, + protocol any client
@dhttp/bridge bare-http1/https/ws, ready-resource, + protocol the bridge itself
@dhttp/hurl hyperdht, + client the CLI, and a standalone binary

The protocol package is the only thing both halves share, and it knows nothing about transports, HTTP or policy. A client package that grew an import of the bridge — or of an HTTP stack — fails packages/client/test, which checks the declared dependencies and every require in the package.

dhttp/                       the app: CLI, lifecycle, OTA updates
  bin.mjs                    flags, logging, signals
  app.js                     DHT node, bridge, and (with updates) the worker
  workers/updater.js         the OTA updater, in a thread of its own
  packages/
    protocol/
      lib/commands.js        command ids
      lib/messages.js        wire encodings
      lib/frames.js          read one frame without starting the flow
    client/
      index.js               HttpBridgeClient
      lib/fetch.js           fetch facade, for libraries that take a transport
      lib/websocket.js       WebSocket facade
    bridge/
      index.js               DHT server, RPC handlers, request policy
      lib/upstream.js        HTTP/HTTPS request to the upstream
      lib/errors.js          refusals, rendered as gateway statuses
      lib/identity.js        persistent seed and bridge key
      lib/constants.js       policy defaults
    hurl/
      bin.mjs                the CLI
      lib/args.js            curl-style argument parsing

Neither half assumes how the connection was made: HttpBridgeClient takes a stream, HttpBridge takes a DHT node.

Scripts

  • npm start — run in dev mode with updates disabled
  • npm run hurl — run hurl, the CLI client
  • npm run make:hurl — build hurl as a standalone binary for the host
  • npm test — run every workspace's tests
  • npm run lint / npm run format
  • npm run make — build a standalone binary for the host platform

Deployment and updates

Updates come from the template: set the upgrade field in package.json to a pear:// link created with pear touch, drop --no-updates, and follow the hello-pear-bare deployment flow. Until that link is replaced the app must be run with --no-updates (which npm start does).