Anonymizer tool for datasets such CSV files.
To generate fake data, you can choose between two excelent generators:
Using pip:
pip install datanonymizerUsing mimesis instead of the default Faker:
pip install datanonymizer[mimesis]Or from source:
git clone https://github.com/fgmacedo/datanonymizer
cd datanonymizer
python setup.py installPass your data through stdin and get it back anonymized on stdout.
Note
In this case, the output will be equal to the input as no conversions were applied.
cat input_file.csv | datanonymizer >output_file.csvUsing a config file to declare conversions and generators for the required fields:
cat input_file.csv | datanonymizer --config ./dataset_anon_config.yml >output_file.csvPlease see examples folder for a small demo:
cat examples/small.csv | python -m datanonymizer -i --config examples/small_faker.yml --seed my_seed >examples/small_anonymized_using_faker.csvOptional arguments:
-h, --help show this help message and exit
-l LANGUAGE, --language LANGUAGE
Language used by the Generator
-di DELIMITER_INPUT, --delimiter_input DELIMITER_INPUT
CSV delimiter
-do DELIMITER_OUTPUT, --delimiter_output DELIMITER_OUTPUT
CSV delimiter
-i, --ignore_errors Continue on errors
--head HEAD Outputs only the first <HEAD> lines
-g {faker,mimesis}, --generator {faker,mimesis}
Generator library to be used for fake data
--seed SEED Seed for the pseudo random generator providers
--config CONFIG Configuration file
You'l need a configuration file to setup transformations for each dataset.
This file is a simple yaml where you can configure fields.
Field names should match the column name declared into the CSV input file.
---
fields:
Task ID:
omit: true
Location:
conversions:
- fn: coords_to_h3
kwargs:
resolution: 8
Client Address:
conversions:
- fn: has_value
rename: has_address
Company Name:
generator:
provider: business.company
rename: company
Invoice ID:
generator:
provider: person.identifier
kwargs:
mask: "#######"
rename: invoiceThe generatos clause depends of the library you choose to provide fake data.
You can use any generator available at the generic API from Faker or mimesis.
For example, if you wanna mimic data with company names:
-
Faker
--- fields: Company Name: generator: provider: company
-
Mimesis
--- fields: Company Name: generator: provider: business.company
But you can replace the real names by names of fruits (using Mimesis) or any other provider:
---
fields:
Company Name:
generator:
provider: food.fruitOr generate random integers to replace real IDs:
-
Faker
--- fields: ID: generator: provider: pyint kwargs: min_value: 1 max_value: 15_000_000
-
Mimesis
--- fields: ID: generator: provider: person.identifier kwargs: mask: "#######"
You can apply any pre-configured conversion functions available.
- coords_to_h3
- has_value
