haro
HarÅ is a modern immutable DataStore built with ES6 features, which can be wired to an API for a complete feedback loop.
It is un-opinionated, and offers a plug'n'play solution to modeling, searching, & managing data on the client, or server
(in RAM). It is a partially persistent data structure, by maintaining version sets of records in versions
(MVCC).
Synchronous commands return an Array
instantly, while asynchronous commands return Promises
which will
resolve or reject in the future. This allows you to build complex applications without worrying about managing async
code.
HarÅ indexes have the following structure Map (field/property) > Map (value) > Set (PKs)
which allow for quick & easy
searching, as well as inspection. Indexes can be managed independently of del()
& set()
operations, for example you
can lazily create new indexes via reindex(field)
, or sortBy(field)
.
Requirements
HarÅ is built with ES6+ features, and requires polyfills for ES5 or earlier environments.
Map
Set
Promise
-
Array.from()
&Array.is()
Object.fromEntries()
How to use
HarÅ takes two optional arguments, the first is an Array
of records to set asynchronously, & the second is a
configuration descriptor.
const storeDefaults = haro();
const storeRecords = haro([{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}]);
const storeCustom = haro(null, {key: 'id'});
Examples
Piping Promises
const store = haro();
console.log(store.size); // 0
store.set(null, {abc: true}).then(function (arg) {
console.log(arg); // [$uuid, {abc: true}];
console.log(store.size); // 1
return store.set(arg[0], {abc: false});
}).then(function (arg) {
console.log(arg); // [$uuid, {abc: false}];
console.log(store.versions.get(arg[0]).size); // 1;
return store.del(arg[0])
}).then(function () {
console.log(store.size); // 0;
}).catch(function (e) {
console.error(e.stack || e.message || e);
});
Indexes & Searching
const store = haro(null, {index: ['name', 'age']}),
data = [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function (records) {
console.log(records[0]); // [$uuid, {name: 'John Doe', age: 30}]
console.log(store.size); // 2
console.log(store.find({age: 28})); // [[$uuid, {name: 'Jane Doe', age: 28}]]
console.log(store.search(/^ja/i, 'name')); // [[$uuid, {name: 'Jane Doe', age: 28}]]
console.log(store.search(function (age) { return age < 30; }, 'age')); // [[$uuid, {name: 'Jane Doe', age: 28}]]
}).catch(function (e) {
console.error(e.stack || e.message || e);
});
MVCC versioning
const store = haro();
store.set(null, {abc: true}).then(function (arg) {
return store.set(arg[0], {abc: false});
}).then(function (arg) {
return store.set(arg[0], {abc: true});
}).then(function (arg) {
store.versions.get(arg[0]).forEach(function (i) { console.log(i[0]); }); // {abc: true}, {abc: false}
}).catch(function (e) {
console.error(e.stack || e.message || e);
});
Benchmarked
A benchmark is included in the repository, and is useful for gauging how haro will perform on different hardware, & software.
Please consider that batch()
, & set()
use Promises
and incur time as a cost.
Batch successful on test
time to batch insert data: 58.500052ms
datastore record count: 1000
name indexes: 1000
testing time to 'find()' a record (first one is cold):
0.194559ms
0.030232ms
0.009265ms
0.006583ms
0.005852ms
testing time to 'search(regex, index)' for a record (first one is cold):
0.720213ms
0.160183ms
0.114591ms
0.110933ms
0.112396ms
time to override data: 5.041485ms
testing time to 'search(regex, index)' on overridden data for a record (first one is cold):
0.129219ms
0.113127ms
0.106789ms
0.105081ms
0.104594ms
Configuration
beforeBatch Function
Event listener for before a batch operation, receives type
, data
.
beforeClear Function
Event listener for before clearing the data store.
beforeDelete Function
Event listener for before a record is deleted, receives key
, batch
.
beforeSet Function
Event listener for before a record is set, receives key
, data
.
index Array
Array of values to index. Composite indexes are supported, by using the default delimiter (this.delimiter
).
Non-matches within composites result in blank values.
Example of fields/properties to index:
const store = haro(null, {index: ['field1', 'field2', 'field1|field2|field3']});
key String
Optional Object
key to utilize as Map
key, defaults to a version 4 UUID
if not specified, or found.
Example of specifying the primary key:
const store = haro(null, {key: 'field'});
logging Boolean
Logs persistent storage messages to console
, default is true
.
onbatch Function
Event listener for a batch operation, receives two arguments ['type', Array
].
onclear Function
Event listener for clearing the data store.
ondelete Function
Event listener for when a record is deleted, receives the record key.
onerror Function
Event listener for errors which occur during common operations, receives two arguments ['type', Error
]
onset Function
Event listener for when a record is set, receives an Array
.
versioning Boolean
Enable/disable MVCC style versioning of records, default is false
. Versions are stored in Sets
for easy iteration.
Example of enabling versioning:
const store = haro(null, {versioning: true});
Properties
data Map
Map
of records, updated by del()
& set()
.
indexes Map
Map of indexes, which are Sets containing Map keys.
registry Array
Array representing the order of this.data
.
size Number
Number of records in the DataStore.
versions Map
Map
of Sets
of records, updated by set()
.
API
batch(array, type) Promise
The first argument must be an Array
, and the second argument must be del
or set
.
const haro = require('haro'),
store = haro(null, {key: 'id', index: ['name']}),
nth = 100,
data = [];
let i = -1;
while (++i < nth) {
data.push({id: i, name: 'John Doe' + i});
}
store.batch(data, 'set').then(function(records) {
// records is an Array of Arrays
}, function (e) {
console.error(e.stack);
});
clear() self
Removes all key/value pairs from the DataStore.
Example of clearing a DataStore:
const store = haro();
// Data is added
store.clear();
del(key) Promise
Deletes the record.
Example of deleting a record:
const store = haro();
store.set(null, {abc: true}).then(function (rec) {
return store.del(rec[0]);
}, function (e) {
throw e;
}).then(function () {
console.log(store.size); // 0
}, function (e) {
console.error(e.stack);
});
dump(type="records") Array or Object
Returns the records or indexes of the DataStore as mutable Array
or Object
, for the intention of reuse/persistent storage without relying on an adapter which would break up the data set.
const store = haro();
// Data is loaded
const records = store.dump();
const indexes = store.dump('indexes');
// Save records & indexes
entries() MapIterator
Returns returns a new Iterator
object that contains an array of [key, value]
for each element in the Map
object in
insertion order.
Example of deleting a record:
const store = haro();
let item, iterator;
// Data is added
iterator = store.entries();
item = iterator.next();
do {
console.log(item.value);
item = iterator.next();
} while (!item.done);
filter(callbackFn[, raw=false]) Array
Returns an Array
of double Arrays
with the shape [key, value]
for records which returned true
to
callbackFn(value, key)
.
Example of filtering a DataStore:
const store = haro();
// Data is added
store.filter(function (value) {
return value.something === true;
});
find(where[, raw=false]) Array
Returns an Array
of double Arrays
with found by indexed values matching the where
.
Example of finding a record(s) with an identity match:
const store = haro(null, {index: ['field1']});
// Data is added
store.find({field1: 'some value'});
forEach(callbackFn[, thisArg]) Undefined
Calls callbackFn
once for each key-value pair present in the Map
object, in insertion order. If a thisArg
parameter is provided to forEach
, it will be used as the this value for each callback.
Example of deleting a record:
const store = haro();
store.set(null, {abc: true}).then(function (rec) {
store.forEach(function (value, key) {
console.log(key);
});
}, function (e) {
console.error(e.stack);
});
get(key[, raw=false]) Array
Gets the record as a double Array
with the shape [key, value]
.
Example of getting a record with a known primary key value:
const store = haro();
// Data is added
store.get('keyValue');
has(key) Boolean
Returns a Boolean
indicating if the data store contains key
.
Example of checking for a record with a known primary key value:
const store = haro();
// Data is added
store.has('keyValue'); // true or false
keys() MapIterator
Returns a new Iterator
object that contains the keys for each element in the Map
object in insertion order.`
Example of getting an iterator, and logging the results:
const store = haro();
let item, iterator;
// Data is added
iterator = store.keys();
item = iterator.next();
do {
console.log(item.value);
item = iterator.next();
} while (!item.done);
limit(offset=0, max=0, raw=false) Array
Returns an Array
of double Arrays
with the shape [key, value]
for the corresponding range of records.
Example of paginating a data set:
const store = haro();
let ds1, ds2;
// Data is added
console.log(store.size); // >10
ds1 = store.limit(0, 10); // [0-9]
ds2 = store.limit(10, 10); // [10-19]
console.log(ds1.length === ds2.length); // true
console.log(JSON.stringify(ds1[0][1]) === JSON.stringify(ds2[0][1])); // false
map(callbackFn, raw=false) Array
Returns an Array
of the returns of callbackFn(value, key)
. If raw
is true
an Array
is returned.
Example of mapping a DataStore:
const store = haro();
// Data is added
store.map(function (value) {
return value.property;
});
override(data[, type="records", fn]) Promise
Returns a Promise
for the new state. This is meant to be used in a paired override of the indexes & records, such that
you can avoid the Promise
based code path of a batch()
insert or load()
. Accepts an optional third parameter to perform the
transformation to simplify cross domain issues.
Example of overriding a DataStore:
const store = haro();
store.override({'field': {'value': ['pk']}}, "indexes").then(function () {
// Indexes have been overridden, no records though! override as well?
}, function (e) {
console.error(e.stack);
});
reduce(accumulator, value[, key, ctx=this, raw=false]) Array
Runs an Array.reduce()
inspired function against the data store (Map
).
Example of filtering a DataStore:
const store = haro();
// Data is added
store.reduce(function (accumulator, value, key) {
accumulator[key] = value;
return accumulator;
}, {});
reindex([index]) Haro
Re-indexes the DataStore, to be called if changing the value of index
.
Example of mapping a DataStore:
const store = haro();
// Data is added
// Creating a late index
store.reindex('field3');
// Recreating indexes, this should only happen if the store is out of sync caused by developer code.
store.reindex();
search(arg[, index=this.index, raw=false]) Array
Returns an Array
of double Arrays
with the shape [key, value]
of records found matching arg
.
If arg
is a Function
(parameters are value
& index
) a match is made if the result is true
, if arg
is a RegExp
the field value must .test()
as true
, else the value must be an identity match. The index
parameter can be a String
or Array
of Strings
;
if not supplied it defaults to this.index
.
Indexed Arrays
which are tested with a RegExp
will be treated as a comma delimited String
, e.g. ['hockey', 'football']
becomes 'hockey, football'
for the RegExp
.
Example of searching with a predicate function:
const store = haro(null, {index: ['name', 'age']}),
data = [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function () {
console.log(store.search(function (age) {
return age < 30;
}, 'age')); // [[$uuid, {name: 'Jane Doe', age: 28}]]
}, function (e) {
console.error(e.stack || e.message || e);
});
set(key, data, batch=false, override=false) Promise
Returns a Promise
for setting/amending a record in the DataStore, if key
is false
a version 4 UUID
will be
generated.
If override
is true
, the existing record will be replaced instead of amended.
Example of creating a record:
const store = haro(null, {key: 'id'});
store.set(null, {id: 1, name: 'John Doe'}).then(function (record) {
console.log(record); // [1, {id: 1, name: 'Jane Doe'}]
}, function (e) {
console.error(e.stack || e.message || e);
});
sort(callbackFn, [frozen = true]) Array
Returns an Array of the DataStore, sorted by callbackFn
.
Example of sorting like an Array
:
const store = haro(null, {index: ['name', 'age']}),
data = [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function () {
console.log(store.sort(function (a, b) {
return a < b ? -1 : (a > b ? 1 : 0);
})); // [{name: 'Jane Doe', age: 28}, {name: 'John Doe', age: 30}]
}, function (e) {
console.error(e.stack || e.message || e);
});
sortBy(index[, raw=false]) Array
Returns an Array
of double Arrays
with the shape [key, value]
of records sorted by an index.
Example of sorting by an index:
const store = haro(null, {index: ['name', 'age']}),
data = [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function () {
console.log(store.sortBy('age')); // [[$uuid, {name: 'Jane Doe', age: 28}], [$uuid, {name: 'John Doe', age: 30}]]
}, function (e) {
console.error(e.stack || e.message || e);
});
toArray([frozen=true]) Array
Returns an Array of the DataStore.
Example of casting to an Array
:
const store = haro(),
data = [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function () {
console.log(store.toArray()); // [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}]
}, function (e) {
console.error(e.stack || e.message || e);
});
values() MapIterator
Returns a new Iterator
object that contains the values for each element in the Map
object in insertion order.
Example of iterating the values:
const store = haro(),
data = [{name: 'John Doe', age: 30}, {name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function () {
const iterator = store.values();
let item = iterator.next();
while (!item.done) {
console.log(item.value);
item = iterator.next();
};
}, function (e) {
console.error(e.stack || e.message || e);
});
where(predicate[, raw=false, op="||"]) Array
Ideal for when dealing with a composite index which contains an Array
of values, which would make matching on a single value impossible when using find()
.
const store = haro(null, {key: 'guid', index: ['name', 'name|age', 'age']}),
data = [{guid: 'abc', name: 'John Doe', age: 30}, {guid: 'def', name: 'Jane Doe', age: 28}];
store.batch(data, 'set').then(function () {
console.log(store.where({name: 'John Doe', age: 30})); // [{guid: 'abc', name: 'John Doe', age: 30}]
}, function (e) {
console.error(e.stack || e.message || e);
});
License
Copyright (c) 2020 Jason Mulligan Licensed under the BSD-3 license