A Web Crawler.

Is a computer program that browses the World Wide Web Start from the url and crawl the web pages with a specified depth.

Introduction :

This program is part of the assigment for the course “Development Tools”, of the Master on libre software of URJC 2013/2014 .

Requirements

• Installing BeatifulSoup

Installation:

Currently, there is no installer, just grab the source code by ether way then Run.

• Download for GIT: https://github.com/Roumia/mswl-dt.git
• Download as zip : https://github.com/Roumia/mswl-dt/archive/master.zip

How to run:

On command line type this command where : "url" is the web page. "level" is the depth of crawling .

$python Amal_crawler.py -n "level" "url"

Code Documentation:

├── Amal_crawler.py
├── AUTHORS
├── ChangeLog
├── LICENSE
├── pymycraawler
│   ├── init.py
│   └── init.pyc
├── README.md
├── setup.py
└── TODO.md

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License. see more in LICENSE part