@mfl/line-delimited-json-stream

A Transform stream that buffers JSON objects. It will emit a completed object on a newline.


Keywords
hpc, rust, udev, zed, zfs
License
MIT
Install
npm install @mfl/line-delimited-json-stream@1.0.0

Documentation

device-scanner

Build Status

This repo provides:

  • a persistent daemon That holds block devices, ZFS devices, and device mounts in memory.
  • a binary that emits UEvents for block-device changes as they occur.
  • a binary that emits device mount changes as they occur.
  • a proxy that transforms the unix domain socket events to HTTPS POSTs.

Architecture

┌───────────────┐ ┌───────────────┐
│  Udev Script  │ │    ZEDlet     │
└───────────────┘ └───────────────┘
        │                 │
        └────────┬────────┘
                 ▼
      ┌─────────────────────┐
      │ Unix Domain Socket  │
      └─────────────────────┘
                 │
                 ▼
   ┌───────────────────────────┐
   │   Device Scanner Daemon   │
   └───────────────────────────┘
                 │
                 ▼
      ┌─────────────────────┐
      │ Unix Domain Socket  │
      └─────────────────────┘
                 │
                 ▼
       ┌──────────────────┐
       │ Consumer Process │
       └──────────────────┘

Development Dependencies

Development setup

  • (Optional) Install ZFS via OS package manager
  • Install Rust deps: cargo build

Building

Local

  • cargo build

Vagrant

  • Running vagrant up will setup a complete environment. It will build device-scanner, scanner-proxy, and device-aggregator, package them as RPMs and install them on the correct nodes.

    To interact with the device-scanner in real time the following command can be used to keep the stream open such that updates can be seen as the data changes:

    cat - | ncat -U /var/run/device-scanner.sock | jq

    If interaction is not required, device info can be retrieved from the device-scanner by running the following command:

    echo '"Stream"' | socat - UNIX-CONNECT:/var/run/device-scanner.sock | jq

Testing

  • cargo test