reposcraping

Scraping GitHub repository


Keywords
github, scraping, repository, clone, github-repository, pip, pip3, pypi, python, python-library, python-module, python3, reposcraping
License
MIT
Install
pip install reposcraping==1.1.5

Documentation

reposcraping

Scraping GitHub repository

This library allows you to access the names of files and folders of any GitHub repository. Cloner class allows you to clone the file types you want to the path you want.

downloads

Downloads Downloads Downloads

setup

pip install reposcraping

usage

from reposcraping import RepoScraping
from reposcraping.cloner import Cloner

scraping = RepoScraping(
    "https://github.com/emresvd/random-video",
    p=True,
)

print(scraping.tree_urls)
print(scraping.file_urls)

cloner = Cloner(scraping)
cloner.clone(
    paths={
        ".py": "files/python_files",
        ".txt": "files/text_files",
        ".md": "files/markdown_files",
        ".html": "files/html_files",
        "": "files/other_files",
    },
    only_file_name=True,
    p=True,
)