FTPwalker

Optimally traversing extremely large FTP directory trees.


Keywords
FTP, traverse, directory, tree, optimized, algorithms-datastructures, bioinformatics-databases, concurrency, dfs-algorithm, ftp-directory-trees, ftp-server, high-performance-computing, network-programming, parallel-computing, python, traversal
License
Other
Install
pip install FTPwalker==0.3

Documentation

Build Status Open Source Love PyPI version

FTPflow

FTPflow is a Python package for optimally traversing extremely large FTP directory trees. It constitutes the algorithmic heart of the PubData search engine. FTPflow creates a dictionary formatted as a JSON file in the user’s home directory containing all the full paths as keys and the respective filenames as values. FTPflow is designed with speed in mind by utilizing state-of-the-art high performance parallelism and concurrency algorithms to traverse FTP directory trees. The resultant hash table (i.e., dictionary) supports fast lookup for any file in any biological database.

Funding

FTPflow is financially supported by the United States Department of Defense (DoD) through the National Defense Science and Engineering Graduate Fellowship (NDSEG) Program. This research was conducted with Government support under and awarded by DoD, Army Research Office (ARO), National Defense Science and Engineering Graduate (NDSEG) Fellowship, 32 CFR 168a.

Screenshots

morpheus_walk

screen shot 2016-09-05 at 2 24 14 pm