@zigtech/scrapper

Headless scrapper developed to run in serverfull and serverless infra.


Keywords
scrapper, crawler, puppeteer, serverless, lambda, chrome-aws-lambda, headless
License
ISC
Install
npm install @zigtech/scrapper@0.0.5

Documentation

@zigtech/scrapper

Fast, easy and full typed scrapper.

NPM Version NPM Downloads Build Status

import { JobStep, Page } from '@zigtech/scrapper';

const run = async (page: Page) => {
  await page.open();

  const { step, options }: JobStep<'navigate'> = {
    step: 'navigate',
    options: {
      url: 'https://github.com/zigante/scrapper',
      waitTime: 1000,
    },
  };

  await page.processStep(step, options);
  await page.close();
};

run(new Page({ resultPath: './tmp' }));

Instalation

This is a Node.js module available through the npm registry.

Before installing, download and install Node.js. Node.js 0.10 or higher is required.

If this is a brand new project, make sure to create a package.json first with the npm init command.

Installation is done using the npm install command:

$$ npm install --save @zigtech/scrapper