A highly scalable Kubernetes application from LUCIT to manage multiple and redundant UNICORN Binance Local Depth Cache Instances on a Kubernetes Cluster for high-frequency access to Binance's DepthCache data (order books).

The cluster can be accessed from any programming language via a REST API, allowing Asks and Bids to be retrieved in JSON format.

Get help!

If you like the project, please it on GitHub!

Licenses will only be publicly available in the store in a few days. If you are interested in a free trial license, please contact us via the chat!

To run the UNICORN DepthCache Cluster for Binance you need a valid license!

The main idea is to deploy the UBDCC on a Kubernetes cluster with, for example, 4 rented servers. After transferring a valid license, you can create and manage DepthCaches within the cluster environment instead of on local servers and access them from multiple clients.

For example, when you configure the system to create 200 DepthCaches with a desired_quantity of 2, UBDCC will deploy 2 DepthCaches for each symbol/market. These DepthCaches are evenly distributed across the nodes of the cluster and can download order book snapshots from the Binance Rest API using their own public IP addresses. On the first run, each server starts 50 DepthCaches, synchronizing the full set of 200 as quickly as possible. Afterward, replicas are initiated, with each node handling 100 DepthCaches.

• Asynchronous Operation: The entire cluster code is built to run asynchronously.
• Load Balancing & Failover: All requests for data (Asks/Bids) are handled via a load balancer with built-in failover, ensuring high availability and quick response times.
  • Local requests for Asks/Bids: ~0.01 seconds
  • Requests via the Internet: ~0.06 seconds
• Flexible Data Retrieval: You can trim the amount of transferred data at the cluster level, either by limiting to a specific amount of top Asks/Bids or by setting a threshold.
• HTTP Access: DepthCache values can be retrieved through HTTP using both synchronous and asynchronous methods provided by UBLDC.
• Top Performance: The entire code base is deployed in the Kubernetes cluster as a compiled C-Extension!
• Supported Architectures: CPython 3.12 on 64-bit (x86_64) and 32-bit (i686): musllinux (based on musl libc 1.1+), manylinux (based on glibc 2.5+ and 2.17+) compatible with manylinux1 and manylinux2014
• Manages Binance Weight Costs: If the weight costs become too high, the cluster throttles the initialization.
• Supported Exchanges:

The first MVP is stable and offers the most critical features for efficient DepthCache management. Future improvements might include switching to websockets instead of REST queries, or implementing simultaneous queries for both Asks and Bids. Vote here for new features!

For more information, check out the GitHub Repository and the Docs.

• Get a Kubernetes cluster with powerful CPUs from a provider of your choice and connect kubectl
• Install dependencies

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

• Install Helm
• Prepare helm

helm repo add lucit-ubdcc https://unicorn-depthcache-cluster-for-binance.docs.lucit.tech/helm
helm repo update

• Install the UNICORN DepthCache Cluster for Binance

helm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc

• Get the "LoadBalancer Ingress" IP, the default Port is TCP 80!

kubectl describe services lucit-ubdcc-restapi

• Find a version to choose

helm search repo lucit-ubdcc

• Then

helm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc --version 0.1.4

helm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc --namespace lucit-ubdcc

helm install lucit-ubdcc lucit-ubdcc/lucit-ubdcc --set publicPort.restapi=8080

• Download the deployment files
• Apply the deployment files with kubectl

kubectl apply -f ./setup/01_namespace_lucit-ubdcc.yaml
kubectl apply -f ./setup/02_role_lucit-ubdcc.yaml
kubectl apply -f ./setup/03_rolebinding_lucit-ubdcc.yaml
kubectl apply -f ./lucit-ubdcc-dcn.yaml  
kubectl apply -f ./lucit-ubdcc-mgmt.yaml
kubectl apply -f ./lucit-ubdcc-mgmt_service.yaml
kubectl apply -f ./lucit-ubdcc-restapi.yaml
kubectl apply -f ./lucit-ubdcc-restapi_service.yaml

• Get the "LoadBalancer Ingress" IP, the default Port is TCP 80:

kubectl describe services lucit-ubdcc-restapi

In any case, you should set the firewall in the web interface of the Kubernetes provider so that only your systems have access to UBDCC.

If you want to do this, you can add HTTPS to the LoadBalancer with most providers.

kubectl delete -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

helm uninstall lucit-ubdcc

• Delete the deployment with kubectl

kubectl delete -f ./setup/01_namespace_lucit-ubdcc.yaml
kubectl delete -f ./setup/02_role_lucit-ubdcc.yaml
kubectl delete -f ./setup/03_rolebinding_lucit-ubdcc.yaml
kubectl delete -f ./lucit-ubdcc-dcn.yaml  
kubectl delete -f ./lucit-ubdcc-mgmt.yaml
kubectl delete -f ./lucit-ubdcc-mgmt_service.yaml
kubectl delete -f ./lucit-ubdcc-restapi.yaml
kubectl delete -f ./lucit-ubdcc-restapi_service.yaml

The UNICORN DepthCache Cluster for Binance is accessed with the Python module UNICORN Binance Local Depth Cache.

Just try this examples!

List of planned features - click if you need one of them or suggest a new feature!

Before you report a bug, try the latest release. If the issue still exists, provide the error trace, OS and Python version and explain how to reproduce the error. A demo script is appreciated.

If you don't find an issue related to your topic, please open a new issue!

Report a security bug!

UNICORN DepthCache Cluster for Binance is an open source project which welcomes contributions which can be anything from simple documentation fixes and reporting dead links to new features. To contribute follow this guide.

We open source!

This project is for informational purposes only. You should not construe this information or any other material as legal, tax, investment, financial or other advice. Nothing contained herein constitutes a solicitation, recommendation, endorsement or offer by us or any third party provider to buy or sell any securities or other financial instruments in this or any other jurisdiction in which such solicitation or offer would be unlawful under the securities laws of such jurisdiction.

Under no circumstances will we be responsible or liable for any claims, damages, losses, expenses, costs or liabilities of any kind, including but not limited to direct or indirect damages for loss of profits.

Do you need a developer, operator or consultant? Contact us for a non-binding initial consultation!