YourBase Python Acceleration

Tests are important. For large monoliths, they're also a major source of drag on velocity.

YourBase is a tool that traces your tests to determine which functions each test depends on. It can later use this information to determine which tests don't need to run because their code paths have not changed. These tests are skipped automatically.

No configuration, setup, or ongoing intervention is required. All you need to do is

pip install yourbase

and then forget it exists.

YourBase works with Python 2.7+ and Python 3.8+. It cooperates with pytest and unittest.

Access

YourBase is an early access tool. It is not free, but it pays for itself. Every YourBase customer saves more in engineer-time and CI costs than they pay to use YourBase.

First run

After installing yourbase, if you are using unittest you must put

import yourbase

in a file that runs before your tests. If you are using pytest this step is not necessary.

Run your tests with the same command you typically use. You should see some output from YourBase similar to

[YB] Starting Python acceleration

The first run will be cold, so if you just want to see YourBase in action and your tests are going to take a while, you can run a subset of tests. Tracing data for the subset will be used correctly even if you later run all tests.

After the run finishes, running again will skip all tests. Modifying a dependency will run only tests whose code paths touched the changed code. You're YourBased! 🚀

Installing for yourself vs. your team

You can install YourBase only for yourself simply by not saving the dependency to your requirements.txt file. YourBase will not pollute your project directory or otherwise make itself known to your team.

You can install YourBase for everyone in your team by adding it to your requirements file and setting YOURBASE_LICENSE_KEY in whatever place your team stores environment variables for tests. If you add it to your requirements file but don't sync the environment variable with your team, their test runs will not be accelerated (but will otherwise run fine).

Synchronizing tracing data

In any of the above scenarios, YourBase's tracing data will not leave the machine it was gathered on without further configuration. This means CI or containerized test runs will not be able to share tracing data forward, removing any opportunity for acceleration.

As a solution, YourBase can be made to share tracing data among any number of machines using an S3 bucket as a collaboration point. To do so, all machines should set

YOURBASE_BUCKET=s3://BUCKETNAME

where BUCKETNAME is an S3 bucket that each machine has Get/Put/List access to.

The tracing data includes test names, file names, function names, and a small amount of metadata like commit hash and build time. Your code will never be uploaded to the bucket. No tracing data will touch YourBase servers.

It is safe to share one bucket between multiple repositories, even if across multiple YourBase-supported languages.

Known issues

Errors

If you run into errors about the _sqlite3 module not being found, first run

# Debian-based
sudo apt-get install libsqlite3-dev
# macOS
brew install sqlite3

then rebuild and reinstall the Python version you are using. If you use pyenv, this will look something like

pyenv install --force <PYTHON_VERSION>

# If that doesn't work, try
PYTHON_CONFIGURE_OPTS="--enable-loadable-sqlite-extensions" pyenv install --force <PYTHON_VERSION>

Conflicts

YourBase has logical conflicts with the following packages:

• pytest-cov: If installed, YourBase will be unable to trace your tests and will be unable to realize it. This will result in nearly-zeroed tracing data, such that tests will appear to have no dependencies. This causes false positive skips. This issue is high priority for us to fix. As a workaround, uninstall pytest-cov then run rm -rf ~/.cache/yourbase-python on any machine with poisoned tracing data. (pytest and coverage independently are fine.)
• moto: If you use YourBase with S3 sync (see "Synchronizing tracing data" above) and use moto mocks in your tests, moto will globally mock the boto3 library YourBase uses to communicate with S3. This will cause errors with stack traces pointing either to your tests or to YourBase internals, depending on initialization order. See spulec/moto#3140 for details. As a workaround you can manually start and stop moto mocks as needed, turn off YourBase's S3 sync feature by unsetting YOURBASE_BUCKET, or remove uses of moto in tests.

Contributing

This open source package is a lightweight wrapper for YourBase proprietary code. We welcome contributions to this wrapper, but at this time we have not built shims or mocks to allow it to be tested end to end without both pieces.

Code style

We use Black for code formatting, which is similar in personality to gofmt -- ruthless consistency, no configuration. Your build will not pass CI if the Black run doesn't come back clean, so we recommend you have your editor automatically run it on save. You can run it manually with

black .