node-red-contrib-prib-functions

Node-RED added node functions.


Keywords
node-red, adjoint, append, analysis, average, avg, AVRO, arvo, backward substitution, compare, Complement Minor, compression, CMA, cumulative, Cumulative Moving Average, data, data analysis, decompression, deflate, delta, determinant, distance, euclidian distance, EMA, EWMA, Exponential Moving Average, forward elimination, functions, gauss jordan, Gauss Jordan Inverse, gaussian elimination, gzip, host available, host status, identity, inflate, injector, inverse, inverse matrix, Linear Correlation Coefficient, levenshtein distance, load, load injector, matrix, maximum, mean, median, minimun, model, monitor, moving, Moving Average, normalize, Pearson Product Moment Correlation, PPMC, Process, range, realtime, Reduced Row Echelon Form, Row Echelon Form, sample variance, sample stddev, Simple Moving Average, skew, SMA, snappy, standard deviation, standardize, statistics, stddev, spawn, spawn process, testing, test, translate, transform, transpose, unzip, variance, weighted, Weighted Moving Average, WMA, xlsx, xcel, xml, zip, Z-score
License
GPL-3.0
Install
npm install node-red-contrib-prib-functions@0.19.2

Documentation

node-red-contrib-prib-functions

Node-Red nodes for various functions:

  • Data Analysis - statistical metrics that has real time option
  • Matrix
  • Transform
  • Test
  • Load Injector
  • Monitor Flow
  • append
  • Spawn Process
  • Host Available
  • node.js os metrics
  • Levenshtein Distance

Data Analysis

Real time metrics which are recalculated on single of data point and posted in msg.result. Key and value can be selected from msg.payload. Sending message with topic"@stats" places message with all stats on second port. If realtime metrics then a third port is shown where the message is sent if it is an outlier being outside 3 standard deviations from mean. This can be changed to median and number of deviations.

Data Analysis Realtime Data Analysis Pearson R

A set of data analysis functions that can be run over an array of data

Single value metrics:

  • Average/Mean
  • Maximum
  • Median
  • Minimum
  • Range
  • Standard Deviation
  • Skewness
  • Sum
  • Variance

Array metrics:

  • Deltas
  • Deltas Normalised
  • Moving Average Simple (SMA)
  • Moving Average Cumulative (CMA)
  • Moving Average Weighted (WMA)
  • Moving Average Exponential (EMA/EWMA)
  • Normalise
  • Standardization (Z-score Normalization)

Array data

  • distances
  • minimum distance(s) between points
  • maximum distance(s) between points

Data Analysis

example:

Data Analysis Realtime example

Data Analysis Pearson R example

Data Analysis example


Matrix

Define a matrix and perform various functions

  • Define / Define Empty / Create / Create Like/ clone"
  • Add / Add Row to Row / Add to Cell / Add Row / Subtract Cell
  • Multiple / Multiple Cell / Divide Cell / Divide Row
  • Transpose
  • Adjoint
  • Cofactor
  • Complement Minor
  • Identity
  • Inverse
  • Determinant
  • Backward Substitution
  • Forward Elimination
  • Gaussian Elimination
  • Reduced Row EchelonForm
  • Row Echelon Form
  • Nearly Equals / Is Square / Get Cell
  • Sum Row
  • Swap Rows
  • To Array Object

Matrix


Transform

Translates a selected msg property to a target property. Messages generates a message for each row or record.

Transformations:

  • Array to CSV
  • Array to HTML
  • Array to ISO8385
  • Array to Messages
  • Array to xlsx / xlsx object (excel uses xlsx)
  • AVRO to JSON (uses avsc)
  • Buffer to comprossed
  • Confluence to JSON
  • Compressed to Buffer
  • Compressed to String
  • COmpressed to JSON
  • CSV to Array
  • CSV to HTML
  • CSV to Messages
  • CSVWithHeader to Array
  • CSVWithHeader to HTML
  • CSVWithHeader to JSON
  • ISO8385 to Array
  • ISO8385 to JSON
  • JSON to Array
  • JSON to Confluence
  • JSON to CSV
  • JSON to AVRO (uses avsc)
  • JSON to ISO8385
  • JSON to Messages
  • JSON to String
  • JSON to xlsx / xlsx object (excel uses xlsx)
  • JSON to XML (uses fast-xml-parser)
  • String to JSON
  • path to Basename
  • path to Dirname
  • path to Extname
  • path to Format
  • path to Is Absolute
  • path to Join
  • path to Parse
  • path to Normalize
  • path to Resolve
  • snappy compress (uses snappy, must install separately)
  • snappy uncompress (uses snappy, must install separately)
  • xlsx / xlsx object to array/JSON (excel uses xlsx)
  • XML to JSON (uses fast-xml-parser)

Note, snappy needs to be installed separately as can have issues with auto install as build binaries.

With xlsx object one can use the function in xlsx against the object in functions node.

Example AVRO with schema

Transform AVRO

For Confluence schema contains a list of schemas in form {"",}


Test

Allows a test case for a message to allow simple testing of nodes. Injects a new message via mouse or message. Message sent to first port which can be consumed by other nodes and returned back to node in a loop. The Test node then checks against detailed expected payload result.

Test

example:

Test example

after run

Test example run


append

Append file(s) to payload. Cached to maximise performance. Require can be used to find file.

append

Test example:

append example


Levenshtein Distance

The levenshtein distance between two character strings.

Levenshtein Distance


Load Injector

Inject messages for a set period of time with varying think time. Primary purpose is testing and useful for load/stress testing.

Has 3 extra data types

  1. generated id - Unique id for each message
  2. generated data - string text generated using dummy-json
  3. generated json - json generated using dummy-json

Load Injector

Test example:

Load Injector example


Monitor Flow

Add on wire between two flows to see message rates in status line. Rate sampled every second and provides rate last second / 10 seconds / 1 minute / 5 minutes.

Monitor Flow

Test example:

Monitor Flow example


Spawn Process

Spawn process as per node.js manual with ability to set working directory, environment variables and argument passed to process. STDOUT and STDERR are sent as individual messages. RC port is sent a message on closure. Takes in messages that starts a process with ability to add environment values. Message can be sent to kill the process.

Spawn Process

Test example:

Spawn Process example


Host Available

Test if host is available sending msg to up or down port so action can be taken. Message only sent on state change or if message is sent which doesn't have topic refreshHostAvailable. This topic forces a check rather than time check which can be set.

Host Available

Test example:

Host Available example


os

The metrics from node.js os

os

Test example:

os example


Install

Run the following command in the root directory of your Node-RED install

npm install node-red-contrib-prib-functions

Tests

Test/example flow in test/generalTest.json


Version

0.19.0 Improve load injector, fix bug in test comparing buffers, add compression tranforms

0.18.0 Add matrix node

0.17.0 Add finished wire to load injector

0.16.0 fix data analysis variance and stddev, add sample, add tests

Author

Peter Prib