thule [t-OOL-ee]
A dumb, yet useful, file walker.
Usage
- Import
from thule import Actions, Walker - Subclass Actions for your use case (e.g.
class JSONtoYAML(Actions):) - Define in your class any or all of the following:
- one or more functions suffixed with
_directory, will run on any directory (or subdirectory) under the path passed to the walker's accept method - one or more functions suffixed with
_file, will run on every file contained in any directory (or any subdirectory) under the path passed to the walker's accept method - one or more functions suffixed with
_final, will run after all directories, subdirectories, and files have been visited under the path passed to the walker's accept method
- one or more functions suffixed with
- Instantiate the subclassed Actions (e.g.
converter = JSONtoYAML()) - Instantiate a Walker (e.g.
walker = Walker()) - Call accept on the walker (e.g.
walker.accept(<path>, converter))
Example
JSON to YAML Converter
Starting data
├── dir1
│  ├── file2.json
│  ├── sub1
│  │  ├── file3.json
│  │  └── file4.json
│  └── sub2
│  └── file5.json
├── dir2
│  └── sub3
│  └── file6.json
├── dir3
│  ├── sub4
│  │  └── file7.json
│  └── sub5
│  ├── file8.json
│  └── file9.json
└── file1.jsonThule Script to convert JSON to YAML
"""
json_to_yaml.py
converts all json files
in nested directories to yaml files
removes old json files
"""
import os
import sys
import json
import yaml
from thule import Actions, Walker
class JSONtoYAML(Actions):
def __init__(self):
self.num_processed = 0
def print_directory(self, dir_path):
print(f"processing files in {dir_path}")
def convert_file(self, dir_path, file_name):
if file_name.endswith(".json"):
try:
print(f" converting {file_name}....", end="", flush=True)
file_path = os.path.join(dir_path, file_name)
with open(file_path, 'r') as jf:
data = json.load(jf)
with open(file_path.replace(".json", ".yaml"), 'w') as yf:
yaml.dump(data, yf)
os.remove(file_path)
self.num_processed += 1
except Exception:
print("[failed]")
else:
print("[done]")
def total_converted_final(self):
print(f"{self.num_processed} files in total were converted from JSON to YAML")
if __name__ == "__main__":
cwd = os.getcwd()
if len(sys.argv) > 2:
print("usage: python example.py <relative path to root dir>")
exit(1)
rel_path = sys.argv[1]
abs_path = os.path.join(cwd, rel_path)
converter = JSONtoYAML()
walker = Walker()
walker.accept(abs_path, converter)Run the script as follows: python json_to_yaml.py relative/path/test_json
Script Output
processing files in cwd/relative/path/test_json
converting file1.json....[done]
processing files in cwd/relative/path/test_json/dir2
processing files in cwd/relative/path/test_json/dir2/sub3
converting file6.json....[done]
processing files in cwd/relative/path/test_json/dir3
processing files in cwd/relative/path/test_json/dir3/sub5
converting file9.json....[done]
converting file8.json....[done]
processing files in cwd/relative/path/test_json/dir3/sub4
converting file7.json....[done]
processing files in cwd/relative/path/test_json/dir1
converting file2.json....[done]
processing files in cwd/relative/path/test_json/dir1/sub1
converting file4.json....[done]
converting file3.json....[done]
processing files in cwd/relative/path/test_json/dir1/sub2
converting file5.json....[done]
9 files in total were converted from JSON to YAML
Resulting Data
├── dir1
│  ├── file2.yaml
│  ├── sub1
│  │  ├── file3.yaml
│  │  └── file4.yaml
│  └── sub2
│  └── file5.yaml
├── dir2
│  └── sub3
│  └── file6.yaml
├── dir3
│  ├── sub4
│  │  └── file7.yaml
│  └── sub5
│  ├── file8.yaml
│  └── file9.yaml
└── file1.yaml
