Bloom Filters for Golang
Bloom filter for go, backed by redis or in process bitset
If you are not familiar with how Bloom filters work and their usefulness, please read.
Example Usage (in process):
install with
go get gopkg.in/bculberson/bloom.v2
go import ( "gopkg.in/bculberson/bloom.v2" )
This bloom filter is initialized to hold 1000 keys and will have a false positive rate of 1% (.01).
m, k := bloom.EstimateParameters(1000, .01)
b := bloom.New(m, k, bloom.NewBitSet(m))
b.Add([]byte("some key"))
exists, _ := b.Exists([]byte("some key"))
doesNotExist, _ := b.Exists([]byte("some other key"))Example Usage (redis backed):
This bloom filter is initialized to hold 1000 keys and will have a false positive rate of 1% (.01).
This library uses http://github.com/garyburd/redigo/redis
pool := &redis.Pool{
MaxIdle: 3,
IdleTimeout: 240 * time.Second,
Dial: func() (redis.Conn, error) { return redis.Dial("tcp", addr) },
}
conn := pool.Get()
m, k := bloom.EstimateParameters(1000, .01)
bitSet := bloom.NewRedisBitSet("test_key", m, conn)
b := bloom.New(m, k, bitSet)
b.Add([]byte("some key"))
exists, _ := b.Exists([]byte("some key"))
doesNotExist, _ := b.Exists([]byte("some other key"))