⚠️ This tool has moved to a new home, please see the following repository in the Mend Toolkit for the latest version: SCA Cleanup Tool
- Current version v23.7.1
- The self-hosted CLI tool features cleaning up projects and generating reports before deletion in 2 modes:
- By stating OperationMode=FilterProjectsByUpdateTime and how many days to keep (-r/ DaysToKeep=)
- By stating OperationMode=FilterProjectsByLastCreatedCopies and how many copies to keep (-r/ DaysToKeep=)
- The reports are saved in the designated location as follows: [Output_DIR]/[PRODUCT NAME]/[PROJECT NAME]/[REPORT NAME]
- The default location is the [WORKING DIRECTORY]/Mend/Reports/[PRODUCT NAME]/[PROJECT NAME]/[REPORT NAME]
- To review the outcome before actual deletion use -y true / DryRun=True flag. It will NOT delete any project nor create reports
- By default, the tool generates all possible project-level reports. By specifying ((-t / Reports=/) it is possible to select specific reports
- The full list of available reports is below
- The full parameters list is available below
- There are two ways to configure the tool:
- By configuring params.config on the executed dir or passing a path to the file in the same format
- By setting command line parameters as specified in the usage below
- Linux (Bash): CentOS, Debian, Ubuntu, RedHat
- Windows (PowerShell): 10, 2012, 2016
- Python 3.8+
- The user used to execute the tool has to have "Organization Administrator" or "Product Administrator" on all the maintained products and "Organization Auditor" permissions.
- It is recommended to use a service user.
- Install by executing:
pip install mend-sca-cleanup-tool
- Configure the appropriate parameters either by using the command line or in
params.config
. - Execute the tool (
mend_sca_cleanup_tool ...
). - In order to update the tool please run
pip install mend-sca-cleanup-tool --upgrade
- Download and unzip mend-sca-cleanup-tool.zip from the most recent tagged release.
- Install requirements:
pip install -r requirements.txt
- Configure the appropriate parameters either by using the command line or
params.config
. - Execute:
python sca_cleanup_tool.py <CONFIG_FILE>
Perform dry run check-in to get to know which projects would have been deleted:
mend_sca_cleanup_tool -r 30 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -y true
Keep the last 60 days on each product, omitting a product token <PRODUCT_1> from analyzing:
mend_sca_cleanup_tool -r 60 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -e <PRODUCT_TOKEN_1>
Keep only two of the newest projects in each product token PRODUCT_1 and PRODUCT_2:
mend_sca_cleanup_tool -r 2 -m FilterProjectsByLastCreatedCopies -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2>
Analyze only the projects that have the specified Mend tag and keep the newest project in each product:
mend_sca_cleanup_tool -r 1 -m FilterProjectsByLastCreatedCopies -u <USER_KEY> -k <ORG_TOKEN> -g <KEY>:<VALUE>
Keep the last 2 weeks and analyze only the projects whose match their tag key and the tag value contains the specified value:
mend_sca_cleanup_tool -r 14 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -v <KEY>:<VALUE>
Keep the last 100 days for both PRODUCT_1 and PRODUCT_2, but do not delete the project PROJECT_1 (which is a project in one of the included products):
mend_sca_cleanup_tool -r 100 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2> -x <PROJECT_TOKEN_1>
Keep the last month for both PRODUCT_1 and PRODUCT_2, but do not delete projects that contain provided strings in their names:
mend_sca_cleanup_tool -r 31 -m FilterProjectsByUpdateTime -u <USER_KEY> -k <ORG_TOKEN> -i <PRODUCT_TOKEN_1>,<PRODUCT_TOKEN_2> -n CI_,-test
usage: ws_cleanup_tool [-h] -u MEND_USER_KEY -k MEND_TOKEN [-a MEND_URL] [-t REPORT_TYPES] [-m {FilterProjectsByUpdateTime,FilterProjectsByLastCreatedCopies}] [-o OUTPUT_DIR] [-e EXCLUDED_PRODUCT_TOKENS] [-i INCLUDED_PRODUCT_TOKENS]
[-g ANALYZED_PROJECT_TAG] [-r DAYS_TO_KEEP] [-p PROJECT_PARALLELISM_LEVEL] [-y DRY_RUN]
Mend Cleanup Tool
optional arguments:
-h, --help show this help message and exit
-u MEND_USER_KEY, --userKey
Mend User Key
-k MEND_API_TOKEN, --apiToken, --orgToken
Mend Organization Key (API Key)
-a MEND_URL, --mendUrl, --wsURL
Mend URL. This value defaults to saas.whitesourcesoftware.com.
-t REPORT_TYPES, --reportTypes
Report Types to generate (comma seperated list)
-m OPERATION_MODE, --operationMode {FilterProjectsByUpdateTime,FilterProjectsByLastCreatedCopies}
Cleanup operation mode
-o OUTPUT_DIR, --outputDir
Output directory
-e EXCLUDED_PRODUCT_TOKENS, --excludedProductTokens
List of excluded products
-i INCLUDED_PRODUCT_TOKENS, --includedProductTokens
List of included products
-g ANALYZED_PROJECT_TAG, --AnalyzedProjectTag
Analyze only the projects whose contain the specific Mend tag (key:value). Case sensitive.
-v ANALYZED_PROJECT_TAG_REGEX_IN_VALUE, --AnalyzedProjectTagRegexInValue
Analyze only the projects whose match their tag key and the tag value contains the specified value (key:value). Case sensitive.
Note: This was originally broken in the original ws-cleanup-tool. The functionality was adjusted to work as originally written. The naming convention is a misnomer but was kept to avoid breaking existing integrations.
-r DAYS_TO_KEEP, --DaysToKeep
Number of days to keep in FilterProjectsByUpdateTime or number of copies in FilterProjectsByLastCreatedCopies
-p PROJECT_PARALLELISM_LEVEL, --ProjectParallelismLevel
Project parallelism level
Note: This is currently not used in this version of the mend-sca-cleanup-tool. Was kept to prevent breaking existing integrations.
-y DRY_RUN, --DryRun
Logging the projects that are supposed to be deleted without deleting and creating reports
default False
-s SKIP_REPORT_GENERATION, --SkipReportGeneration
Skip report generation step
default False
-j SKIP_PROJECT_DELETION, --SkipProjectDeletion
Skip project deletion step
default False
-x EXCLUDED_PROJECT_TOKENS, --excludedProjectTokens
List of excluded projects
-n EXCLUDED_PROJECT_NAME_PATTERNS, --excludedProjectNamePatterns
List of excluded project name patterns (comma seperated list). Case sensitive.
The following Mend project reports are available through the clean-up tool. These values can be specified with the -t flag to generate specific reports.
- alerts
- alerts_rejected_by_policy
- attribution
- bugs
- due_diligence
- ignored_alerts
- in_house_libraries
- inventory
- license_compatibility
- resolved_alerts
- request_history
- source_files
- source_file_inventory
- vulnerability
If you need to run a clean up script for your SAST environment, please refer to the Mend SAST clean up kit in the Mend Toolkit
note: The optimal cleanup scope is derived from the size of the environment, Mend scope size (memory and CPU) allocated for the server, and runtime time constraints.