pygora
BC Agora Portal.
A web crawler library that fetches and parses data fromGetting started (Python 3):
pip install pygora-phchcc
Examples
log in agora, download and print links to all subject pages
from pygora import *
session, gen_time = get_session("myAgoraUsername", "myAgoraPassword", check_valid=True)
# if gen_time == 0, we know something goes wrong (maybe you did not input the correct credential)
print(gen_time)
subjects = download_subjects(session, simple=True) # simple: each subject is a string
for i, line in enumerate(subjects):
print(i, line)
# subjects = download_subjects(session) #eacg subject is a dict, with more information
cache the username and password so that you don't have to write them explicitly in a script
from pygora import *
# to set credential, run it once so that username & password are stored locally
set_credential("myAgoraUsername", "myAgoraPassword")
# to clear out credential
set_credential("", "")
parse_subject_page
: print out all biology courses (school and subject codes can be found in subject.txt
), provided that if you have run set_credential
example of from pygora import *
session, gen_time = get_session(*get_credential(), check_valid=True)
# if you are confident that your username & password are correct, do
# session, gen_time = get_session(*get_credential())
url = SUBJECT_URL.format('2MCAS', '2BIOL') # get you a url string
resp = session.get(url) # use your session to HTTP get the url
courses = parse_subject_page(resp) # parse the subject page
for course in courses:
print(course)
parse_course_page
: print all information on a course page (the course code can be found in the output of parse_subject_page
)
example of from pygora import *
session, gen_time = get_session(*get_credential())
url = COURSE_URL.format('ACCT102101')
# a dummy dict in this example, could be your data fetched from database
info_dict = dict()
resp = session.get(url)
parse_course_page(resp, info_dict) # update the dict
for key, value in info_dict.items():
print(key, value)
Related Projects
EagleVision
the backend ofNew PEPS (planning)
the backend ofJoin Dev Team / Contact Us:
open an issue on Github to announce the feature/bug that you want to work on
or through email: (Haochen) phchcc_at_gmail_dot_com
or search our names in BC directory
Special Thanks
Special thanks to people who made EagleVision (this project's prototype) and pygora alive (names are listed in alphabetical order):
Baichuan (Patrick) Guo -- the original "Honest Team"
David Shen -- the EagleVision Dev Team
Estevan Feliz -- the original "Honest Team" & the EagleVision Dev Team
Roger Wang -- the EagleVision Dev Team
Yuning (Tommy) Yang -- the original "Honest Team"
Yuxuan (Jacky) Jin -- the EagleVision Dev Team