recurtx

Release 0.0.20

CLI to transform text files recursively

Homepage PyPI Python

Keywords
CLI, recursive, text, find, xargs, sed
License
Apache-2.0
Install
pip install recurtx==0.0.20

Documentation

recurtx

Python version PyPI version License

CLI to recursively search and transform text files in xargs style

Key features

  • search and substitute (replace) text in file contents or file paths in place
  • installable without sudo (using pip)

Background

This tool was developed as an opinionated (partial) alternative to the following CLI tools.

  • find to recursively find file paths
  • fd to recursively find file paths respecting .gitignore (fd is internally used by xunder and xbatch in default if installed)
  • grep to search text
  • rg to search text
  • ag to search text
  • sed to modify text
  • xargs to repeat similar to for-loop
  • tree to recursively check file paths
  • du to check disk usage
  • csvkit to search (and modify) text in csv files
  • spyql to search (and modify) text in csv files
  • clickhouse-local
  • polars-cli to search text in csv files
  • csvlens to view csv files
  • rich-cli to view csv files

This tool is quicker to write although execution might be slower depending on the amount of your text data.

Install

Prerequisite: Python 3 (3.8 or later recommended)

[Option 1] Install from PyPI

pip install recurtx

[Option 2] Install from source code

This is recommended only if you want to modify the source code.

git clone https://github.com/Minyus/recurtx.git
cd recurtx
python setup.py develop

Wrapper Commands

xunder

Run any scripts for each file under a directory recursively.

Examples

Run wc -l {FILEPATH} for each file under directory_foo recursively:

xunder directory_foo "wc -l"

Quoting for the script can be omitted for most cases.

xunder directory_foo wc -l

Caveat: int, float, tuple, list, dict could be formatted unexpectedly (by fire package), for example:

  • 00 (recognized as int by Python) will be converted to 0 while "00" (recognized as str by Python) will be kept as is

Description

NAME
    xunder

SYNOPSIS
    xunder PATH <flags> [SCRIPTS]...

POSITIONAL ARGUMENTS
    PATH
        Type: str
    SCRIPTS
        Type: str

FLAGS
    --glob=GLOB
        Type: str
        Default: '**/*'
    --replace_str=REPLACE_STR
        Type: str
        Default: '@@'
    --show_paths=SHOW_PATHS
        Type: bool
        Default: False
    --show_scripts=SHOW_SCRIPTS
        Type: bool
        Default: False

NOTES
    You can also use flags syntax for POSITIONAL ARGUMENTS

xbatch

Run any scripts for a batch of files in a directory recursively.

Examples

Concatenate all the contents in directory_foo.

xbatch directory_foo cat

Description

NAME
    xbatch

SYNOPSIS
    xbatch PATH <flags> [SCRIPTS]...

POSITIONAL ARGUMENTS
    PATH
        Type: str
    SCRIPTS
        Type: str

FLAGS
    --glob=GLOB
        Type: str
        Default: '**/*'
    --replace_str=REPLACE_STR
        Type: str
        Default: '@@'
    --show_paths=SHOW_PATHS
        Type: bool
        Default: False
    --show_scripts=SHOW_SCRIPTS
        Type: bool
        Default: False

NOTES
    You can also use flags syntax for POSITIONAL ARGUMENTS

Commands to transform text files

xsearch

Search a keyword, which may include wildcards, in the text file content, and optionally substitute (replace).

Examples

Search keyword_bar in each file under directory_foo recursively:

xunder directory_foo xsearch keyword_bar

Search keyword_bar and substitute (replace) with keyword_baz in each file under directory_foo recursively:

xunder directory_foo xsearch keyword_bar --sub keyword_baz

Description

NOTES
    You can also use flags syntax for POSITIONAL ARGUMENTS

NAME
    xsearch - Search a keyword, which may include wildcards, in the text file content, and optionally substitute (replace).

SYNOPSIS
    xsearch TARGET PATH <flags>

DESCRIPTION
    Search a keyword, which may include wildcards, in the text file content, and optionally substitute (replace).

POSITIONAL ARGUMENTS
    TARGET
        Type: str
    PATH
        Type: str

FLAGS
    --sub=SUB
        Type: Optional[typing.Unio...
        Default: None
    -w, --wildcard=WILDCARD
        Type: str
        Default: '*'
    --separator=SEPARATOR
        Type: str
        Default: '/'
    -v, --verbose=VERBOSE
        Type: int
        Default: 1
    -c, --context=CONTEXT
        Type: typing.Union[int, NoneType]
        Default: 1
    -p, --plain=PLAIN
        Type: bool
        Default: False

NOTES
    You can also use flags syntax for POSITIONAL ARGUMENTS

xfind

Find a keyword, which may include wildcards, in the file path, and optionally substitute (replace).

Examples

Search keyword_bar in each file path under directory_foo recursively:

xunder directory_foo xfind keyword_bar

Search keyword_bar and substitute (replace) with keyword_baz in each file path under directory_foo recursively:

xunder directory_foo xfind keyword_bar --sub keyword_baz

Description

NAME
    xfind

SYNOPSIS
    xfind TARGET PATH <flags>

POSITIONAL ARGUMENTS
    TARGET
        Type: str
    PATH
        Type: str

FLAGS
    -s, --sub=SUB
        Type: Optional[str]
        Default: None
    -w, --wildcard=WILDCARD
        Type: str
        Default: '*'
    -v, --verbose=VERBOSE
        Type: int
        Default: 1

NOTES
    You can also use flags syntax for POSITIONAL ARGUMENTS

xll

Alternative to ls -lah and du. Show approximate total size and max size in Bytes to respond quickly in default while it takes time to run du in a big directory including many files. Show number of files and the most common file extention(s) as well.

Examples

Run under directory_foo non-recursively (up to depth 1):

xll directory_foo

Run under directory_foo recursively up to depth 2:

xll -d 2 directory_foo

Description

NAME
    xll - Compute statistics for the directory recursively.

SYNOPSIS
    xll <flags> [PATHS]...

DESCRIPTION
    Compute statistics for the directory recursively.

POSITIONAL ARGUMENTS
    PATHS
        Type: str

FLAGS
    -d, --depth=DEPTH
        Type: int
        Default: 1
    -u, --unit_glob=UNIT_GLOB
        Type: str
        Default: '**/*'
    -t, --type=TYPE
        Type: Optional[typing.Unio...
        Default: None
    -g, --glob=GLOB
        Type: str
        Default: '**/*'
    -r, --regex=REGEX
        Type: str
        Default: '^(?!.*(\\.git\\/|__pycache__\\/|\\.ipynb_checkpoints\\/|\\....
    -n, --number_limit=NUMBER_LIMIT
        Type: int
        Default: 100
    -s, --sort_paths=SORT_PATHS
        Type: str
        Default: 'asc'
    -i, --info=INFO
        Type: bool
        Default: True
    -e, --extension_most_common=EXTENSION_MOST_COMMON
        Type: int
        Default: 1

xpandas

Read and transform tabular data using pandas.

Regarding options, see the documents for pandas.read_xxx such as:

Data types supported by pandas (not all were tested):

  • "pickle"
  • "table"
  • "csv"
  • "fwf"
  • "clipboard"
  • "excel"
  • "json"
  • "html"
  • "xml"
  • "hdf"
  • "feather"
  • "parquet"
  • "orc"
  • "sas"
  • "spss"
  • "sql_table"
  • "sql_query"
  • "sql"
  • "gbq"
  • "stata"

Install dependency

pip install pandas

Examples

Read files supported by pandas (such as csv and json) under directory_foo and concatenate:

xbatch directory_foo xpandas

xpolars

Read and transform tabular data using polars.

Regarding options, see the documents for polars.scan_xxx (or polars.read_xxx if scan function is not available), such as:

Data types supported by polars (not all were tested):

  • "csv"
  • "ipc"
  • "parquet"
  • "database"
  • "json"
  • "ndjson"
  • "avro"
  • "excel"
  • "delta"

Install dependency

pip install polars

Examples

Read files supported by polars (such as csv and json) under directory_foo and concatenate:

xbatch directory_foo xpolars

Dependency to enable CLI

Stats

Dependencies
1
Dependent packages
0
Dependent repositories
0
Total releases
20
Latest release
First release
Stars
0
Forks
0
Watchers
1
Contributors
1
Repository size
104 KB
SourceRank
8

Development practices

Source repo 2FA enabled
TEXT!
Package manager 2FA enabled
TEXT!
Is security responsive
TEXT!
Dependencies are managed
TEXT!
Issue-free release available
TEXT!
Succession plan available
TEXT!
Package manager 2FA enabled
TEXT!

The Tidelift Subscription provides access to a continuously curated stream of human-researched and maintainer-verified data on open source packages and their licenses, releases, vulnerabilities, and development practices.

Learn more →

Releases

0.0.1
0.0.10
0.0.2
0.0.3
0.0.4
0.0.5
0.0.6
0.0.7
0.0.8
0.0.9
See all 20 releases

Contributors

Yusuke Minami


See all contributors

Something wrong with this page? Make a suggestion

Export .ABOUT file for this package

Last synced: 2024-02-25 12:54:00 UTC

Login to resync this project