toolcache

toolcache makes it simple to create and configure caches in python

Features

• save caches to memory or to disk
• memoize functions, instance methods, @classmethods, and @staticmethods
• control cache size with ttl and eviction policies like lru / fifo / lfu
• use thread safety, process safety, or no safety (default = thread safety)
• use custom hash functions
• track cache usage statistics

Install

pip install toolcache

Contents

• Example Usage
• Reference
  • Cache Types
  • Cache Creation
  • Cache Configuration
  • Cache Decorators
  • Cache Methods
• Frequently Asked Questions

Example Usage

Creating Caches

import toolcache

# memoize function with memory cache
@toolcache.cache('memory')
def f(a, b, c):
    return a * b * c

# memoize function with disk cache, stored in a tempdir
@toolcache.cache('disk')
def f(a, b, c):
    return a * b * c

# memoize function with disk cache, stored in a persistent dir
@toolcache.cache('disk', cache_dir='/path/to/cache/dir')
def f(a, b, c):
    return a * b * c
    
# remove cache entries once they reach a specific age
@toolcache.cache('disk', ttl='24 hours')
def f(a, b, c):
    return a * b * c

# remove cache entries once cache reaches a specific size
@toolcache.cache('disk', max_size=3, max_size_policy='fifo')
def f(a, b, c):
    return a * b * c

# specify which args are used to create unique hash of inputs
@toolcache.cache('disk', hash_args=['a', 'b'])
def f(a, b, c):
    return a * b * c

# create standalone cache
standalone_cache = toolcache.MemoryCache()

Using Caches

# get cache size
print(f.cache.get_cache_size())
> 4

# track cache usage statistics
print(f.cache.stats)
> {'n_checks': 6,
>  'n_deletes': 2,
>  'n_hashes': 8,
>  'n_hits': 2,
>  'n_loads': 1,
>  'n_misses': 4,
>  'n_saves': 3,
>  'n_size_evictions': 0,
>  'n_ttl_evictions': 0}

# clear cache
f.cache.delete_all_entries()

More Examples

• Configure Hashing
• Configure Disk Caches
• Use Cache Eviction Policies
• Monitor Cache Usage
• Create Standalone Caches
• Define Custom Cachetypes

Cache Reference

Cache Types

toolcache includes 3 cache types that each inherit from abstract cache class BaseCache:

Cache Creation

Caches can be created in two ways:

1. decorating a function with @toolcache.cache(cachetype) where cachetype is 'memory', 'disk', 'null', or a class inheriting from BaseCache
2. creating a standalone cache by instantiating a class that inherits from BaseCache

Cache Configuration

The configuration options listed below can be passed to toolcache.cache() or passed to a standalone cache during initialization.

General Config

these configuration options are available to every cache

Hash Config

Eviction Config

Statistic Tracking Config

DiskCache-specific Config

Cache Decorators

When using toolcache.cache() to decorate a function, one should consider 1) how function inputs will be hashed, 2) what attributes will be added to the function, and 3) what arguments might be added to the function.

Hashing Function Inputs

To save a function input-output pair within a cache, a unique hash must be taken of the inputs.

Under the default hash configuration, each input arg should either be json-serializable or be a hashable object (i.e. it implements a __hash__() method). By default toolcache uses orjson to create these hashes quickly.

If function inputs do not satisfy these criteria, one or more of the cache config parameters should be used:

toolcache.cache() also works on functions that have *args or **kwargs for inputs

Decorated Function Args

Every time the decorated function is called, it can use the following keyword args to control cache behavior.

You can avoid adding these args to the decorated function by using @toolcache.cache(..., add_cache_args=False).

Decorated Function Attributes

The original decorated function can be acessed as f.__wrapped__.

The cache instance associated with a decorated function f() can be accessed using f.cache.

Cache Methods

These methods are available on every cache instance:

Frequently Asked Questions

How is the performance? What is the overhead for using a cache decorator?

To maximize cache performance, one can disable input name normalization (normalize_hash_inputs=False), statistic tracking (track_basic_stats=False and track_detailed_stats=False), and thread safety (safety=None).

On a somewhat modern machine with the above settings, the toolcache.cache() decorator adds about 3 μs to each function call, whereas running a simple function with no cache decorator takes about 50 ns per function call. Using a disk cache instead of a memory cache adds about 25 μs per function call. To truly know whether toolcache is fast enough for your application you may need to run your own benchmarks.

How does toolcache relate to other similar projects?

A large motivation for developing toolcache was being able to manage memory-based and disk-based caches with a unified interface and feature set. toolcache is currently the only python package to offer this functionality.

There exist many other python packages for caching and memoization. cacheout and python-memoization both provide in-memory caches with many features. Compared to toolcache these libraries provide a wider variety of cache eviction policies and other interesting features. python-diskcache provides a feature-rich disk-based cache with Django integration and extensive benchmark comparisons to other solutions.