endl is a Node.js module for extracting links from web pages and downloading them.
Changelog | API | Examples | CLI | To-Do List |
---|
endl has a very simple also an advanced API for link extracting, file downloading, executing and unzipping.
Simple example
var endl = require('endl');
endl
.page('http://lame.buanzo.org/')
.find('a[href^="http://lame.buanzo.org/Lame_"]') // returns Promise
.then(function(container) {
container.download({
pageUrlAsReferrer: true,
filenameMode: { urlBasename: true }
});
});
Explanation
- We require our endl module.
-
endl.page()
loads the page we want. (It takes two arguments, second argument is an options object and optional.) -
find()
finds the elements we want. (Works just like jQuery and querySelectorAll) but it is a Promise that returns a containerInstance - Download our file to the current directory, using basename of our download link for file name and using our page URL as Referer header.
Things to note:
- We actually get 4 elements when we do
find()
butdownload()
automatically selects the first element (0-index). Useindex()
to change index of element array. -
download()
afterfind()
is a shortcut. The long way is: find(...) → href() → download(...)
Install
npm install endl
Prerequisites: Tools for building NodeJS native modules (Visual Studio or Visual Studio Express)
How do you pronounce endl?
Like Handel the composer, but without the h → andel :)
Current issues
- findXpath doesn't work. Blame web pages (for incorrect structure), xmldom and xpath modules.
To-Do
To-do list is at Trello (You can vote on notes)
If you don't have a Trello account and want to vote, you can use my referral link to register (They just give a month of free Gold membership for each referral so I can add custom background images to the board