krabber

Release 1.0.4

Easy to use webscraper

Homepage Repository npm Download

Keywords
HTML, scraping, webscraper, webscraping, axios, JSDOM, pagination, typescript
License
MIT
Install
npm install krabber@1.0.4

Documentation

krabber

Easy to use webscraper using Axios and JSDOM.

Codecov branch Gitlab pipeline status (branch) Libraries.io dependency status for latest release npm

Example

import { Scrape } from 'krabber';

const scrapeConfig = {
    url: 'https://github.com/axios/axios',
    options: {
        headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0' }
    },
    select: {
        numberOfCommits: ({ dom, res, prev, url }) => {
            return dom.querySelector('.commits span.text-emphasized.num').innerHTML.trim();
        }
    }
}

Scrape(scrapeConfig)
    .then((res) => {
        console.log(res.numberOfCommits);
    })

More examples, including pagination handling

Please see the test suite in src/lib/scrape.spec.ts

Stats

Dependencies
0
Dependent packages
0
Dependent repositories
0
Total releases
15
Latest release
First release
Stars
1
Forks
0
Contributors
0
SourceRank
8

Development practices

Source repo 2FA enabled
TEXT!
Package manager 2FA enabled
TEXT!
Is security responsive
TEXT!
Dependencies are managed
TEXT!
Issue-free release available
TEXT!
Succession plan available
TEXT!

The Tidelift Subscription provides access to a continuously curated stream of human-researched and maintainer-verified data on open source packages and their licenses, releases, vulnerabilities, and development practices.

Learn more →

Releases

1.1.7
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.6
1.0.5
1.0.4
See all 15 releases

Login to resync this project