KVLite

KVLite is a partition-based key-value cache built for SQL RDBMSs.

KVLite entries can be stored either in persistent or volatile fashion, and each key/value pair can have its own lifetime and refresh mode.

Entries are grouped by partition, where each key must be unique only inside the partition it belongs to.

Following RDBMSs are currently supported by KVLite or will be supported soon:

• MySQL and MariaDB (.NET and .NET Core, with integration tests for MySQL only)
• Oracle (.NET and .NET Core)
• PostgreSQL (.NET and .NET Core, with integration tests)
• SQL Server (.NET and .NET Core, with integration tests)
• SQLite (.NET and .NET Core, with integrations tests)

KVLite implements different caching adapters for various libraries and frameworks:

• ASP.NET Core distributed cache and session.
• IdentityServer4 cache provider.
• Polly cache provider.

Summary

• Build status on GitLab:
• DocFX generated API documentation: https://pommalabs.gitlab.io/kvlite/
• NuGet package(s):
  • PommaLabs.KVLite, caching interfaces, data models and core implementations.
  • PommaLabs.KVLite.Memory, default in-memory driver.
  • PommaLabs.KVLite.Database, SQL code shared by all drivers.
  • PommaLabs.KVLite.MySql, driver for MySQL and MariaDB.
  • PommaLabs.KVLite.Oracle, driver for Oracle.
  • PommaLabs.KVLite.PostgreSql, driver for PostgreSQL.
  • PommaLabs.KVLite.SqlServer, driver for SQL Server.
  • PommaLabs.KVLite.SQLite, driver for SQLite.
  • PommaLabs.KVLite.AspNetCore, distributed caching and session helpers.
  • PommaLabs.KVLite.IdentityServer4, IdentityServer4 cache provider.
  • PommaLabs.KVLite.Polly, Polly cache provider.

Introduction

Let's start with a simple example of what you can do with KVLite:

/// <summary>
///   Learn how to use KVLite by examples.
/// </summary>
internal static class Program
{
    /// <summary>
    ///   Learn how to use KVLite by examples.
    /// </summary>
    public static void Main()
    {
        // Some variables used in the examples.
        var examplePartition1 = "example partition 1";
        var examplePartition2 = "example partition 2";
        var exampleKey1 = "example key 1";
        var exampleKey2 = "example key 2";
        var simpleValue = Math.PI;
        var complexValue = new ComplexValue
        {
            Integer = 21,
            NullableBoolean = null,
            String = "Learning KVLite",
            Dictionary = new Dictionary<short, ComplexValue>
            {
                [1] = new ComplexValue { NullableBoolean = true },
                [2] = new ComplexValue { String = "Nested..." }
            }
        };

        /*
         * KVLite stores its values inside a given partition and each value is linked to a key.
         * KVLite can contain more than one partition and each partition can contain more than one key.
         *
         * Therefore, values are stored according to this logical layout:
         *
         * [partition1] --> key1/value1
         *              --> key2/value2
         * [partition2] --> key1/value1
         *              --> key2/value2
         *              --> key3/value3
         *
         * A key is unique inside a partition, not inside all cache.
         * A partition, instead, is unique inside all cache.
         */

        // You can start using the default caches immediately. Let's try to store some values in
        // a way similar to the figure above, using the default persistent cache.
        ICache persistentCache = PersistentCache.DefaultInstance;
        persistentCache.AddTimed(examplePartition1, exampleKey1, simpleValue, persistentCache.Clock.UtcNow + TimeSpan.FromMinutes(5));
        persistentCache.AddTimed(examplePartition1, exampleKey2, simpleValue, persistentCache.Clock.UtcNow + TimeSpan.FromMinutes(10));
        persistentCache.AddTimed(examplePartition2, exampleKey1, complexValue, persistentCache.Clock.UtcNow + TimeSpan.FromMinutes(10));
        persistentCache.AddTimed(examplePartition2, exampleKey2, complexValue, persistentCache.Clock.UtcNow + TimeSpan.FromMinutes(5));
        PrettyPrint(persistentCache);

        // Otherwise, you can customize you own cache... Let's see how we can use a volatile
        // cache. Let's define the settings that we will use in new volatile caches.
        var volatileCacheSettings = new VolatileCacheSettings
        {
            CacheName = "My In-Memory Cache", // The backend.
        };

        // Then the settings that we will use in new persistent caches.
        var persistentCacheSettings = new PersistentCacheSettings
        {
            CacheFile = "CustomCache.sqlite", // The SQLite DB used as the backend for the cache.
            ChancesOfAutoCleanup = 0.5, // Chance of an automatic a cache cleanup being issued.
        };

        // We create both a volatile and a persistent cache.
        var volatileCache = new VolatileCache(volatileCacheSettings);
        persistentCache = new PersistentCache(persistentCacheSettings);

        // Use the new volatile cache!
        volatileCache.AddTimed(examplePartition1, exampleKey1, Tuple.Create("Volatile!", 123), TimeSpan.FromMinutes(60));
        PrettyPrint(volatileCache);

        // Use the new persistent cache!
        persistentCache.AddTimed(examplePartition2, exampleKey2, Tuple.Create("Persistent!", 123), TimeSpan.FromMinutes(60));
        PrettyPrint(persistentCache);

        /*
         * An entry can be added to the cache in three different ways.
         *
         * "Timed" values last until the specified date and time, or for a specified timespan.
         * Reading them will not extend their lifetime.
         *
         * "Sliding" values last for the specified lifetime, but, if read,
         * their lifetime will be extended by the timespan specified initially.
         */

        // Let's clear the volatile cache and let's a value for each type.
        volatileCache.Clear();
        volatileCache.AddTimed(examplePartition1, exampleKey1, simpleValue, volatileCache.Clock.UtcNow + TimeSpan.FromMinutes(10));
        volatileCache.AddTimed(examplePartition1, exampleKey2, complexValue, TimeSpan.FromMinutes(15));
        volatileCache.AddSliding(examplePartition2, exampleKey2, complexValue, TimeSpan.FromMinutes(15));
        PrettyPrint(volatileCache);
    }

    private static void PrettyPrint(ICache cache)
    {
        Console.WriteLine($"Printing the contents of a {cache.GetType().Name}");

        // When we use "Peek*" methods, the expiration time of entries is left untouched.
        var cacheEntries = cache.PeekEntries<object>();
        foreach (var cacheEntry in cacheEntries.OrderBy(ci => ci.Partition).ThenBy(ci => ci.Key))
        {
            Console.WriteLine($"{cacheEntry.Partition} --> {cacheEntry.Key} --> {cacheEntry.Value}");
        }

        Console.WriteLine();
    }

    private sealed class ComplexValue
    {
        public int Integer { get; set; }
        public bool? NullableBoolean { get; set; }
        public string String { get; set; }
        public IDictionary<short, ComplexValue> Dictionary { get; set; }

        public override string ToString() => nameof(ComplexValue);
    }
}

Polly cache provider

Moreover, KVLite can be used as Polly cache provider:

// Every KVLite cache can be interfaced with Polly: Memory, MySQL, Oracle, ...
var options = new KVLiteCacheProviderOptions();
var cacheProvider = new KVLiteSyncCacheProvider<string>(PersistentCache.DefaultInstance, options);
var cachePolicy = Policy.Cache(cacheProvider, TimeSpan.FromMinutes(10));

var myGuid1 = cachePolicy.Execute(() => Guid.NewGuid().ToString(), new Context("MyGuid"));
var myGuid2 = cachePolicy.Execute(() => Guid.NewGuid().ToString(), new Context("MyGuid"));

// Two GUIDs are equal because they share the same key.
Debug.Assert(myGuid1 == myGuid2);

myGuid1 = cachePolicy.Execute(() => Guid.NewGuid().ToString(), new KVLiteContext("My", "Complex", "Key", 1));
myGuid2 = cachePolicy.Execute(() => Guid.NewGuid().ToString(), new KVLiteContext("My", "Complex", "Key", 2));

// Two GUIDs are not equal because they do not share the same key.
Debug.Assert(myGuid1 != myGuid2);

Examples

Further examples can be found in the following projects:

• ASP.NET Core: It shows how to register KVLite services and how to use it as a proper distributed cache implementation. Moreover, it shows how to use KVLite session extensions.

Storage layout

KVLite stores cache entries in a dedicated table, whose schema is as much tuned as possible for each RDBMS. The logical schema for cache entries table is the following:

If SQL user chosen for KVLite has enough privileges, then cache entries table will be automatically created. Anyway, specialized schemas for supported RDBMS systems are available inside this project repository or at following links:

• MySQL and MariaDB
• Oracle
• PostgreSQL
• SQL Server

Each script might have a few comments suggesting how to further optimize cache entries table storage depending on the actual version of the specific RDBMS system.

Customizing table name or SQL schema

Default name for cache entries table is kvle_cache_entries and default SQL schema is kvlite. However, those values can be easily changed at runtime, as we do in the following snippet:

// Change cache entries table name for Oracle cache.
OracleCache.DefaultInstance.Settings.CacheEntriesTableName = "my_custom_name";

// Change SQL schema name for MySQL cache.
MySqlCache.DefaultInstance.Settings.CacheSchemaName = "my_schema_name";

// Change both table ans schema name for SQL Server cache.
SqlServerCache.DefaultInstance.Settings.CacheEntriesTableName = "my_custom_name";
SqlServerCache.DefaultInstance.Settings.CacheSchemaName = "my_schema_name";

Please perform those customizations as early as your application starts; for example, these are good places where to put the lines:

• Program.cs for console and Windows applications.
• Global.asax.cs for classic web applications.
• Startup.cs for Owin-based web applications or ASP.NET Core.

About this repository and its maintainer

Everything done on this repository is freely offered on the terms of the project license. You are free to do everything you want with the code and its related files, as long as you respect the license and use common sense while doing it :-)

I maintain this project during my spare time, so I can offer limited assistance and I can offer no kind of warranty.

However, if this project helps you, then you might offer me an hot cup of coffee:

Credits

This project heavily relies on the following open source projects:

• FsPicker, used to serialize and deserialize cache entries.
• HellBrick.Diagnostics, used to improve code quality.
• LibLog, library-agnostic log provider.