paginateit
Used to paginate REST API Calls / Mostly on MultiThreaded API Calls via Python
Description
Takes 2 inputs:
- count -> total number of items in all pages
- max_workers -> number of threads need to be created (Based upon the number provided, skip and limits are calculated dynamically)
Getting Started
Dependencies
Python 3.6 and further releases.
Installing
pip install paginateit
Executing program
from paginateit import paginateit as pg
pg.page(23052, 4)
----------------|---------------|---------------|------------|
|SKIP | LIMIT | FLIMIT | COUNT |
|---------------|---------------|---------------|------------|
|skip1: 0 |limit1: 5763 |flimit1: 763 |count1: 5763|
|---------------|---------------|---------------|------------|
|skip2: 5763 |limit2: 11526 |flimit2: 1526 |count2: 5763|
|---------------|---------------|---------------|------------|
|skip3: 11526 |limit3: 17289 |flimit3: 7289 |count3: 5763|
|---------------|---------------|---------------|------------|
|skip4: 17289 |limit4: 23052 |flimit4: 3052 |count4: 5763|
|---------------|---------------|---------------|------------|
|skip5: 23052 |limit5: 28815 |flimit5: 8815 |count5: 5763|
|---------------|---------------|---------------|------------|
We can now use the variables like skip(x) or limit(x) in the API where filter.
SKIP: A skip filter omits the specified number of returned records
LIMIT: A limit filter limits the number of records returned to the specified number
FLIMIT (Custome field): A flimit filter can be used to fetch all the remaining records at the last page of every thread
COUNT: A count filter tell the maximum number of records available in that page.
Other examples:
pg.page(1700, 8) --> Max records availabe in all pages is 1700 and are creating 8 threads / loops (We see skip[1-9] values)
pg.page(90005, 6) --> Max records availabe in all pages is 90005 and are creating 6 threads / loops (We see skip[1-7] values)
pg.page(5025) --> Max records availabe in all pages is 1700 and are creating 8 threads /loops (We see skip[1-3] values)
pg.page(count, max_workers) # max_workers defaults to 2 unless specified
Usage
Access the dynamic calculated variables as shown below:
pg.skip1, pg.skip3, pg.limit2, pg.flimit4, pg.count2
Help
Any advise for common problems or issues, please feel free to create a issue in Github
Authors
Manjesh N manjesh_n@hotmail.com
License
This project is licensed under the Apache License 2.0 License - see the LICENSE file for details
