VecSim - A unified interface for similarity servers
A standard, light-weight interface to all popular similarity servers.
The problems we are trying to solve:
- Standard API - Different vector similarity servers have different APIs - so switching is not trivial.
- Identifiers - Some vector similarity servers support string IDs, some do not - we keep track of the mapping.
- Partitions - In most cases, pre-filtering is needed prior to querying, we abstract this concept away.
Supported engines:
- Scikit-learn, via NearestNeighbors
- RediSearch
- Faiss
- ElasticSearch
QuickStart example
import numpy as np
# Import a similarity server of your choice:
# SKlearn (best for small datasets or testing)
from vecsim import SciKitIndex
sim = SciKitIndex(metric='cosine', dim=32)
user_ids = ["user_"+str(1+i) for i in range(100)]
user_data = np.random.random((100,32))
item_ids=["item_"+str(101+i) for i in range(100)]
item_data = np.random.random((100,32))
sim.add_items(user_data, user_ids, partition="users")
sim.add_items(item_data, item_ids, partition="items")
# Index the data
sim.init()
# Run nearest neighbor vector search
query = np.random.random(32)
dists, items = sim.search(query, k=10) # returns a list of users and items
dists, items = sim.search(query, k=10, partition="users") # returns a list of users onlyFor more examples, please read our documentation
