HouseofCat.Serialization

RabbitMQ.Dataflows library helps use RabbitMQ as well as create powerful RabbitMQ powered workflows.


Keywords
C#, Dataflow, HouseofCat, HouseofCat.io, NET8, NetCore, Pipelines, RabbitMQ, TPL, Workflows, csharp, data, dataflows, net, queue, queueing, rabbit, workflow
License
MIT
Install
Install-Package HouseofCat.Serialization -Version 4.1.3

Documentation

RabbitMQ & Dataflows

A RabbitMQ library of .NET tools to help rapidly develop well performant services or to just help manage durable connectivity with the RabbitMQ.Client!

Status

Test Server: Windows 11
RabbitMQ Server: v3.13
Erlang: v26.2.3

Release build
Codacy Badge1
Codacy Badge2

Why Make A RabbitMQ Powered Dataflow?

Dataflows have configurable concurrency, serialization, monitoring, compression, and encryption all as first class citizens. This paradigm allows developers to just focus on the important stuff - getting work done!

Here are some features ready with RabbitMQ.Dataflows.

Workflows

  • Pipelines (v1) are light weight workflow-esque function-as-step executor.
    • All steps process in the order provided allowing you to still control order of execution.
    • Configurable parallelism and concurrency built-in.
    • No automatic OpenTelemetry.
    • No automatic exception catching/async error handling (thrown exceptions kill Pipelines).
  • Dataflows (v2) are the more complex, yet complete, workflow class.
    • All steps process in the order provided allowing you to still control order of execution.
    • Configurable parallelism and concurrency built-in.
    • Async processing, batch processing, consumer cloning and more!
    • Async error handling (simplify functional error handling by allowing functions to throw).
    • AutoPublish (to the next Queue) functionality built-in.
    • OpenTelemetry with native distributed tracing for Publish/Consumer.

Built-Ins

  • A RabbitMQ ConnectionPool and ChannelPool (connection durability).
  • Supports ILogger<T> via LogHelpers static singleton.
  • Contracted IWorkState simplifies functional generic returns and integration.
  • Has ISerializationProvider support for System.Text.Json and MessagePack.
    • Easy to write your own providers.
  • Has ICompressionProvider built-in support for Gzip, Deflate, Brotli, and LZ4.
  • Has IEncryptionProvider built-in support for AesGcm and BouncyCastle AesGcm.
  • RecyclableMemoryStream options for ICompressionProvider and IEncryptionProvider.
  • Publishers AutoPublish seamless encryption/decryption steps (adds headers too).
  • Consumers allow seamless compression/decompression steps (uses headers).
  • Publisher/Consumers have built-in OpenTelemetry support.

Testing

  • All built-in steps will have integration tests that should remove concerns from end-user developer.
  • Future case will include much more complex abstract UnitTesting as time allows.
  • The developer should only need to unit test their functional business code.

Help & Guides

More to come!

You can also find various library examples inside the tests/UnitTests or the tests/RabbitMQ.Console.Test project.

Main RabbitMQ Library

HouseofCat.RabbitMQ

NuGet
NuGet

A library that focuses on RabbitMQ connection and channel management to create fault tolerant Publishers and Consumers.
Formerly called CookedRabbit.Core/Tesseract.

DataFlow Library

HouseofCat.Dataflows

NuGet
NuGet

A library that provides the base magic Dataflows for RabbitMQ.Dataflows.

  • Custom TPL Block - ChannelBock used as a Channel-based BufferBlock<TIn>
  • Has DataFlowEngine and ChannelBlockEngine.
  • Has Pipelines (Dataflow alternative).

Core Productivity Libraries

These libraries are here to help you build powerful Dataflows for your messages.

HouseofCat.Serialization

NuGet
NuGet

A library that has a collection of .NET ISerializationProvider or the interface to make your own.

  • Supports MessagePack and System.Text.Json and Newtonsoft.Json.

HouseofCat.Compression

NuGet
NuGet

A library that has a collection of .NET ICompressionProvider or the interface to make your own.

  • Supports LZ4, Gzip, Brotli, and Deflate.
  • Supports RecyclableMemoryStream variants.

HouseofCat.Hashing

NuGet
NuGet

A library that focuses on implementing hashing.

  • Supports Argon2.

HouseofCat.Encryption

NuGet
NuGet

A library that provides encryption contracts and the base AesGCM/AesCBC .NET IEncryptionProvider as well as the interface to make your own.

  • Supports AesCbc via CryptoStream (good for encrypted file/memorystreams).
  • Supports .NET AesGcm 128, 192, 256 (non-streams).
  • Supports BouncyCastle AesGcm 128/192/256.
  • Supports RecyclableMemoryStream variants.

Non-Critical Library Integrations

HouseofCat.Data

NuGet
NuGet

A library that provides the provides helper classes for data manipulation and transformation.

Also provides Database abstractions, a simple Dapper integration, and SqlKata integration SQL query generation.

Database Connection Factory Support

  • System.Data.SqlClient
  • Microsoft.Data.SqlClient
  • MySql.Data.MySqlClient
  • Npgsq
  • MySql.Data
  • Oracle
  • SQLite

HouseofCat.Utilities

NuGet
NuGet

A library that focuses on general purpose utilities and functions that simplify the coding experience.