I don't want to read the README
Just show me how it works as fast as possible!
npm i -g file-fetcher
mkdir /tmp/file-fetcher-test
cd !$
echo '[
{
"url": "https://dummyimage.com/200x100/ddd/777.jpg&text=file-fetcher+test",
"path": "img",
"name": "#.jpg",
"delay": 1
}
]' > config.json
file-fetcher config.jsonThe images are being downloaded to /tmp/file-fetcher-test/img every minute.
File fetcher
file-fetcher is a simple tool that lets you download files recurrently.
Installation
$ npm install -g file-fetcherUsage
- Create a config file with your configuration (See documentation and example below)
- Quick use:
$ file-fetcher <config file>- Pro use:
$ file-fetcher
[ --config | -c ] <config file>
[ --destination | -d <destination folder> ]Default destination folder is cwd.
Config file properties
The config file must contain an array of objects. Each object corresponds to a file and can/must have the following properties:
| Property | Type | Required? | Default | Description |
|---|---|---|---|---|
| url | String | ✓ | The complete URL from which to download the file | |
| path | String | ✓ | The relative local path where to save the file | |
| name | String | ✓ | The name pattern of the destination file (e.g.: my-file-number-#.jpg) | |
| description | String | name |
The string used to represent the file in log messages | |
| firstIndex | Number | 1 | The first number used as index (#) in name pattern | |
| digits | Number | 3 | The minimum number of digits for the index (#) in name pattern. Smaller numbers will be padded with 0s on the left. | |
| cron | String | ✓ or delay
|
The cron pattern that determines the download frequence | |
| delay | Number | ✓ or cron
|
The delay in minutes that determines the download frequence |
File name pattern
The file name needs to be a pattern in order to not overwrite the same file after each download.
Pattern substitutions:
| String | Replaced by |
|---|---|
| # | Index of the download iteration |
| {date} | A string representing the current time |
delay vs cron
To determine the download frequence of each file, you can use either the delay property or the cron property.
-
delayis simply a number of minutes between each download
N.B.: due to the cron implementation, using a number like 29 will trigger a download at 00:00, 00:29, 00:58, 01:00, 01:29, ... -
cronis a cron expression
Config example
[
{
"url": "http://www.my-site.com/path/to/file.jpg",
"name": "file_nb_#({date}).jpg",
"path": "relative/path/to/file",
"cron": "0 0/2 7-18 * * *",
"description": "my file"
},
{
"url": "http://www.my-site.com/path/to/other/file.jpg",
"name": "file_nb_#.jpg",
"firstIndex": 42,
"digits": 4,
"path": "relative/path/to/other/file",
"delay": 3,
"description": "my other file"
}
]A functional .sample.config.json file is provided in the sources.
