TEST REST API
Create fast modern asynchronous tests for REST API testing
#FAST #EASY #ASYNC #RESTAPI #TESTING #AUTOMATION
- Features
- Installation
- Usage
-
Examples
- My first test
- Configure my test
- Replace inbuilt assert
- My first log
- Set global variables value
- Set global variables value as constant
- Get global variables value
- My first bug
- Configure my bug
- My first rest api
- Configure my rest api
- Reuse my rest api
- Send my rest api
- Rest api response
- Demo with all the above features
- Reports
- Author
- License
Features
- Asyncronus programming (Powered by asyncio)
- Auto discovery of test modules and functions
- Supports both asyncronus & syncronuous tests
- High speed test executions using async functions
- Html test reporting with custom logs & summary dashboards
- Create complex flows using parameterization & correlation
- Group similar tests using hashtags eg: #smoke
- Supports CI/CD test automation integrations
- Designed to be easy to use & learn
Installation
If you already have Python with pip installed, you can simply run:
pip install test_rest_api
Usage
1. Basic usage
Syntax
python -m test_rest_api -t "<Test folder/file path>"
- Tests are executed from the command line using the test_rest_api module directly
- The basic usage is by giving a file path or directory path as an argument
- -t stands for Test suite path
- We can organise folders, sub folders and python files in any custom structure
- test_rest_api will autodetect python files and folders as Test suites
2. Set report path
Syntax
python -m test_rest_api -t "<Test folder/file path>" -r "Result folder path"
- In the above example, html test report is saved under the same test folder path
- We can save the final report to our custom path by providing -r followed by path
- -r stands for Report path
- test_rest_api creates beautiful rich test report with summary dashboards
- We can also add our custom logs to each individual tests in the test report
3. Set .env path
Syntax
python -m test_rest_api -t "<Test folder/file path>" -r "Result folder path" -e ".env file path"
- We can set variables with values, example Domain, Username, Password etc. in .env file
- test_rest_api will auto fetch all these values and save under Global Variables as constants
- Global Variables can be accessed in tests for parametrization
- One example can be dynamic url creation. Domain can be parameterised in url creation
- This will also help developers to run the same tests in different environments by creating separate .env files
- We can set the environment variables by providing -e followed by path
- -e stands for Environment path
4. Set hashtags
Syntax
python -m test_rest_api -t "<Test folder/file path>" -r "Result folder path" -e ".env file path" -h #SMOKE#SANITY
- Group your test by providing tags in test creation
- This helps to execute a group of testcases, example smoke testing, regression testing etc.
- test_rest_api will fetch all tests with the provided tags in the command line and executes them
- We can set the hashtags for execution by providing -h followed by tags, example #SMOKE#SANITY
- -h stands for Hashtags
- When no tags are provided, all the tests are executed
- When we provide a single tag, only those testcases with that tag are executed
- Some tests like login should run always for all tags irrespective of the value
- In this case it's not practical to add all tags to the login test function
- Adding all tags will be hard to maintain when we introduce new custom tags
- To tackle this issue, we can use inbuilt #ALL tag
- #ALL tagged tests will always be executed. It will not be skipped
- Tests like login, logout etc. are perfect candidates for #ALL
5. Project structure
.
├── api # Store all your rest api files
│ ├── auth # Custom structure for subfolders & files
│ │ ├── login.py # Python file to store rest_api code
│ │ │ ├── def user_login() # Python function to create rest_api
│ │ │ ├── def admin_login()
│ │ │ └── ...
│ │ └── ...
│ └── ...
├── testsuite # Root testsuite folder
│ ├── auth # Custom structure for subfolders & files
│ │ ├── login.py # Python file as testsuite to store tests
│ │ │ ├── async def t001() # Python async function as testcases
│ │ │ ├── async def t002()
│ │ │ └── ...
│ │ └── ...
│ └── ...
└── ...
- 2 sub folders api & testsuite
- Any custom names can be used instead of api & testsuite
- api should store all the api creation python files
- testsuite should store all the test creation python files
- Any custom structure with multiple sub folders can be used to organise the files
- Separating api from tests will avoid code duplication for rest_api creation
- Root testsuite folder path will be used in command line execution value for -t
Examples
1. My first test
from test_rest_api import test
@test()
async def my_first_test():
assert 4 == 5- Create an async python function with any custom name
- Decorate the function using @test()
- Async python functions decoratd with @test() will be auto-detected as testcase
- Normal python functions decoratd with @test() will not be considered as testcase
Congrats !
You have successfully created your first test (my_first_test)
Now let's execute it from command line . . .
- Create a python virtual environment
- Install test_rest_api using command
pip install test_rest_api - Create new python file (eg: my_first_testsuite.py) and paste the above code
- Execute the test from the command line using the test_rest_api module directly
python -m test_rest_api -t "<Test folder/file path>" -r "Result folder path"
2. Configure my test
from test_rest_api import test
@test(name='Custom Name', desc='Description', enabled=True, tags=['SMOKE', 'ABC'], is_async=True, execution_order='1')
async def my_second_test():
assert 4 == 5-
In our first example, we used @test() decorator with empty parameters
-
In this example, we are using parameters to configure our test function
-
This can be considered as adding settings to your test function
-
name
- Mandatory: False
- Data Type: str
- Expected: Custom name for your test
- Default: Function name
-
desc
- Mandatory: False
- Data Type: str
- Expected: Test description
- Default: Empty string
-
enabled
- Mandatory: False
- Data Type: bool
- Expected: True or False. Provide False, to disable the test
- Default: True
-
tags
- Mandatory: False
- Data Type: list
- Expected: List of tags. Group your tests with custom tags
- Default: Empty list
-
is_async
- Mandatory: False
- Data Type: bool
- Expected: True or False. Provide False, to run tests sequentially
- Default: True
-
execution_order
- Mandatory: False
- Data Type: str
- Expected: Custom text for ordering. This will work only when is_async = False
- Default: 'zzzzz'
3. Replace inbuilt assert
from test_rest_api import test, Assert
@test()
async def my_first_logger():
# assert 4 == 5
Assert.equal(4, 5)- Assert is same as inbuilt python assert statement
- Using test rest api Assert class improves logging
- Assertions done using Assert will be automatically logged in final html report
- It is recommended to use Assert instead of inbuilt assert statement for all your tests
4. My first log
from test_rest_api import test
@test()
async def my_first_logger():
print('My 1st log')
print('My 2nd log')
assert 4 == 5- Python inbuilt
print()function is used to add custom messages to the final html test report - Add print() functions inside your test function
- Add any number of log messages without any limit
- It is recommended to add logs for all your tests
- Note:
print()statements will not be printed to console
5. Set global variables value
from test_rest_api import test, GlobalVariables
@test()
async def set_global_variables_value():
GlobalVariables.set(name='token', value='token from response')- Global variables are used to save & retrieve values in runtime
- This will help in parameterization and correlation of tests
- Save Token from login api response in your first test
- Retrieve this Token in all your upcoming tests for authentication
- Here we will learn how to set the value in GlobalVariables
GlobalVariables.set(name='token', value='token from response')
-
name
- Mandatory: True
- Data Type: str
- Expected: Custom name for your global variable
-
value
- Mandatory: True
- Data Type: any
- Expected: Any python data type can be stored as global variables value
6. Set global variables value as constant
from test_rest_api import test, GlobalVariables
@test()
async def set_global_variables_value_as_constant():
GlobalVariables.set(name='pi', value=3.14, is_constant=True)- In the above example we learned to set value to global variables
- Multiple set calls are possible for the same variable
- Token variable can be set again in a different test function
- To avoid this, we can set global variables as a constant value
- Use is_consant optional parameter & make it True
GlobalVariables.set(name='pi', value=3.14, is_constant=True)
-
name
- Mandatory: True
- Data Type: str
- Expected: Custom name for your global variable
-
value
- Mandatory: True
- Data Type: any
- Expected: Any python data type can be stored as global variables value
-
is_constant
- Mandatory: False
- Data Type: bool
- Expected: True or False. Provide True, to create constants
- Default: False
7. Get global variables value
from test_rest_api import test, GlobalVariables
@test()
async def get_global_variables_value():
token: str = GlobalVariables.get(name='token')
pi: float = GlobalVariables.get(name='pi')- In the above examples we learned to set varying and constant value to global variables
- Now it's time to retrieve them in our tests
- Please make sure to use valid variable names while retrieving the values
- If the variable name is not present in global variables, test will terminate with error
GlobalVariables.get(name='token')
-
name
- Mandatory: True
- Data Type: str
- Expected: Valid name of any saved global variable
8. My first bug
from test_rest_api import test, Bug
@test()
async def my_first_bug():
Bug()- Bug is used to raise issues in tests
- Add custom checks in your tests to validate rest api response
- If actual result is not the expected result, just call
Bug() - This will terminate the current test function execution
- Bug details can be viewed in final html test report
9. Configure my bug
from test_rest_api import test, Bug, Logger
@test()
async def my_second_bug():
logger = Logger()
logger.log('step 1')
logger.log('step 2')
logger.log('Consider all steps are logged')
Bug(message="msg", priority=Bug.PRIORITY.BLOCKER, actual_result="", expected_result="", steps_to_reproduce=logger)- In our first bug example, we used Bug() with empty attributes
- In this example, we are using attributes to configure our Bug
- This can be considered as adding more info to your bug in the final html test report
- We are adding custom logging in this example to show how logger instance is useful in Bug creation
- Logger instance variable can be passed to steps_to_reproduce attribute during Bug creation
Bug(message='', priority='', actual_result='', expected_result='', steps_to_reproduce='')
-
message
- Mandatory: False
- Data Type: str
- Expected: Custom message for your bug
- Default: Empty string
-
priority
- Mandatory: False
- Data Type: str
- Expected: Priority for this bug. Supported list: Bug.PRIORITY.[ items ]
- Default: Empty string
-
actual_result
- Mandatory: False
- Data Type: str
- Expected: Provide the actual result
- Default: Empty string
-
expected_result
- Mandatory: False
- Data Type: str
- Expected: Provide the expected result
- Default: Empty list
-
steps_to_reproduce
- Mandatory: False
- Data Type: str
- Expected: Logger instance can be used to auto-populate this field
- Default: Empty string
10. My first rest api
from test_rest_api import test, RestApi
@test()
async def my_first_rest_api():
rest_api = RestApi(url='https://dummyjson.com/products/1')- RestApi is used create rest api instance in tests
- Here we have created a basic rest api with just the url information
- This example is only about creating rest api, no send action is performed here
- We will use this instance variable for sending the request in upcoming examples
11. Configure my rest api
from test_rest_api import test, RestApi
@test()
async def configure_my_rest_api():
rest_api = RestApi(url='my url', parameters={'p1': 'v1', 'p2': 'val2'}, headers={'h1': 'v1', 'h2': 'val1'}, body={})-
In the above example, we have only used url attribute for rest api creation
-
Other attributes for creation are parameters, headers and body
-
This example shows the syntax for adding these optional attributes
-
url
- Mandatory: True
- Data Type: str
- Expected: Rest api url
-
parameters
- Mandatory: False
- Data Type: dict
- Expected: Key value pairs of parameter name & value
- Default: {}
-
headers
- Mandatory: False
- Data Type: dict
- Expected: Key value pairs of header name & value
- Default: {}
-
body
- Mandatory: False
- Data Type: dict
- Expected: Provide the json request payload
- Default: {}
12. Reuse my rest api
from test_rest_api import RestApi, GlobalVariables
def login_api(username: str, password: str):
domain = GlobalVariables.get('DOMAIN')
url = f'https://{domain}/login'
body = {'username': username, 'password': password}
return RestApi(url=url, body=body)- A rest api will be used for multiple tests
- Creation of rest api inside test async functions will result in code duplication
- Duplicate code makes your program lengthy and decreases your code quality
- Updating & maintaining this rest api creations in multiple tests will be difficult
- New changes to rest api, will result in changing the same code multiple times
- To avoid duplication, we can use a separate folder for rest api files
- Use python functions to create a rest api which will avoid code duplication
- You can call a function 100 times instead of writing it 100 times
- In this example we have created a simple login api
- All the dynamic values for rest api creation can be passed as function parameters
- This helps in calling the same api with different inputs
- Return the RestApi instance which can be used in test functions for sending
13. Send my rest api
from test_rest_api import test, RestApi
@test()
async def send_my_rest_api():
rest_api = RestApi(url='https://dummyjson.com/products/1')
response1 = await rest_api.get()
response2 = await rest_api.send(method='get')
response3 = await rest_api.send(method=rest_api.METHODS.GET)- In the above examples, we learned to create a rest api
- Now it's time to send them using http methods
- Supported http methods are GET, POST, PUT, PATCH, DELETE, OPTIONS & HEAD
- Here we are sending the rest_api using GET http method
- All the responses (1, 2 & 3) will have the same result
- Because they perform the same functionality with different syntax
- Similarly, other http methods can be used, with your desired syntax
14. Rest api response
from test_rest_api import test, RestApi
@test()
async def send_my_rest_api():
rest_api = RestApi(url='https://dummyjson.com/products/1')
response = await rest_api.get()
status_code = response.status_code
body = response.body
headers = response.headers
content_type = response.content_type
obj = response.obj-
We have covered rest api creation & sending part
-
Now lets see more about the response object
-
Mostly all checks will be performed using response object data
-
Response object contains the below details
-
response.status_code
- Data Type: int
- Value: Response status code
-
response.body
- Data Type: dict
- Value: Response body
-
response.headers
- Data Type: dict
- Value: Response headers
-
response.content_type
- Data Type: str
- Value: Response content type
-
response.obj
- Data Type: aiohttp.ClientResponse
- Value: Python aiohttp ClientResponse object
15. Demo with all the above features
.env file
---------
DOMAIN=dummyjson.com
from test_rest_api import test, RestApi, Bug, Logger, GlobalVariables
@test(name='Demo', desc='All features')
async def demo():
logger = Logger()
logger.log('Starting the test')
logger.log('Creating rest api')
domain = GlobalVariables.get('DOMAIN')
rest_api = RestApi(url=f"https://{domain}/products/1")
logger.log('Sending rest api')
response = await rest_api.get()
logger.log(f'Response code = {response.status_code}')
logger.log(f'Response body = {response.body}')
logger.log('Validate response code')
if response.status_code != 200:
Bug(message='Invalid status code', priority=Bug.PRIORITY.MINOR, actual_result=response.status_code,
expected_result='200', steps_to_reproduce=logger)
logger.log('Save price details to global variables')
title = response.body.get('title', '')
price = response.body.get('price', '')
GlobalVariables.set(name=title, value=price)
logger.log('Successfully completed the test')Reports
1. My first report
- Single basic test with PASS status
Click here to view the html report
Console output
2023-04-08 15:40:22,712:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 15:40:22,712: Starting test setup
2023-04-08 15:40:22,712: Auto detecting test suites
2023-04-08 15:40:22,712: Total test suites: 1
2023-04-08 15:40:22,712: Auto detecting tests
2023-04-08 15:40:22,713: Total synchronous tests: 0
2023-04-08 15:40:22,714: Total asynchronous tests: 1
2023-04-08 15:40:22,714: Total tests: 1
2023-04-08 15:40:22,714: Created aiohttp client session
2023-04-08 15:40:22,714: Completed test setup
2023-04-08 15:40:22,714:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 15:40:22,714: PASS my_first_test (root file) [1]
2023-04-08 15:40:22,715:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: PASS
Tests: 1
Start: 2023-04-08 15-40-22
End: 2023-04-08 15-40-22
Duration: 0.000956233 seconds
Tags: []
PASS: 1
FAIL: 0
ERROR: 0
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 02. Async tests report
- 5 async tests
- Each tests takes 1 second to complete
Click here to view the html report
Console output
2023-04-08 15:52:05,062:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 15:52:05,062: Starting test setup
2023-04-08 15:52:05,062: Auto detecting test suites
2023-04-08 15:52:05,062: Total test suites: 1
2023-04-08 15:52:05,062: Auto detecting tests
2023-04-08 15:52:05,063: Total synchronous tests: 0
2023-04-08 15:52:05,063: Total asynchronous tests: 5
2023-04-08 15:52:05,063: Total tests: 5
2023-04-08 15:52:05,063: Created aiohttp client session
2023-04-08 15:52:05,063: Completed test setup
2023-04-08 15:52:05,063:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 15:52:06,064: PASS testcase_1 (root file) [1]
2023-04-08 15:52:06,064: PASS testcase_2 (root file) [2]
2023-04-08 15:52:06,065: PASS testcase_3 (root file) [3]
2023-04-08 15:52:06,065: PASS testcase_4 (root file) [4]
2023-04-08 15:52:06,065: PASS testcase_5 (root file) [5]
2023-04-08 15:52:06,066:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: PASS
Tests: 5
Start: 2023-04-08 15-52-05
End: 2023-04-08 15-52-06
Duration: 1.002069748 seconds
Tags: []
PASS: 5
FAIL: 0
ERROR: 0
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 03. Sync tests report
- 5 sync tests
- Each tests takes 1 second to complete
Click here to view the html report
Console output
2023-04-08 15:56:19,128:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 15:56:19,129: Starting test setup
2023-04-08 15:56:19,129: Auto detecting test suites
2023-04-08 15:56:19,129: Total test suites: 1
2023-04-08 15:56:19,129: Auto detecting tests
2023-04-08 15:56:19,129: Total synchronous tests: 5
2023-04-08 15:56:19,129: Total asynchronous tests: 0
2023-04-08 15:56:19,129: Total tests: 5
2023-04-08 15:56:19,129: Created aiohttp client session
2023-04-08 15:56:19,129: Completed test setup
2023-04-08 15:56:19,130:
=======================================================
|| ................ S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 15:56:20,131: PASS testcase_1 (root file) [1]
2023-04-08 15:56:21,132: PASS testcase_2 (root file) [2]
2023-04-08 15:56:22,132: PASS testcase_3 (root file) [3]
2023-04-08 15:56:23,134: PASS testcase_4 (root file) [4]
2023-04-08 15:56:24,135: PASS testcase_5 (root file) [5]
2023-04-08 15:56:24,137:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: PASS
Tests: 5
Start: 2023-04-08 15-56-19
End: 2023-04-08 15-56-24
Duration: 5.006235552 seconds
Tags: []
PASS: 5
FAIL: 0
ERROR: 0
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 04. Sync & Async report
- 5 sync & async tests each
- Total 10 tests
- Each tests takes 1 second to complete
Click here to view the html report
Console output
2023-04-08 15:59:38,170:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 15:59:38,170: Starting test setup
2023-04-08 15:59:38,170: Auto detecting test suites
2023-04-08 15:59:38,170: Total test suites: 1
2023-04-08 15:59:38,170: Auto detecting tests
2023-04-08 15:59:38,172: Total synchronous tests: 5
2023-04-08 15:59:38,172: Total asynchronous tests: 5
2023-04-08 15:59:38,172: Total tests: 10
2023-04-08 15:59:38,173: Created aiohttp client session
2023-04-08 15:59:38,173: Completed test setup
2023-04-08 15:59:38,173:
=======================================================
|| ................ S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 15:59:39,174: PASS testcase_1 (root file) [1]
2023-04-08 15:59:40,176: PASS testcase_2 (root file) [2]
2023-04-08 15:59:41,177: PASS testcase_3 (root file) [3]
2023-04-08 15:59:42,178: PASS testcase_4 (root file) [4]
2023-04-08 15:59:43,180: PASS testcase_5 (root file) [5]
2023-04-08 15:59:43,180:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 15:59:44,182: PASS testcase_async_1 (root file) [6]
2023-04-08 15:59:44,182: PASS testcase_async_2 (root file) [7]
2023-04-08 15:59:44,183: PASS testcase_async_3 (root file) [8]
2023-04-08 15:59:44,183: PASS testcase_async_4 (root file) [9]
2023-04-08 15:59:44,183: PASS testcase_async_5 (root file) [10]
2023-04-08 15:59:44,185:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: PASS
Tests: 10
Start: 2023-04-08 15-59-38
End: 2023-04-08 15-59-44
Duration: 6.01064299 seconds
Tags: []
PASS: 10
FAIL: 0
ERROR: 0
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 05. Multi status report
- 5 async tests with different status values
- Status list: PASS, FAIL, ERROR, DISABLE & SKIP
Click here to view the html report
Console output
2023-04-08 16:17:44,328:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 16:17:44,329: Starting test setup
2023-04-08 16:17:44,329: Auto detecting test suites
2023-04-08 16:17:44,329: Total test suites: 1
2023-04-08 16:17:44,329: Auto detecting tests
2023-04-08 16:17:44,330: Total synchronous tests: 0
2023-04-08 16:17:44,330: Total asynchronous tests: 5
2023-04-08 16:17:44,330: Total tests: 5
2023-04-08 16:17:44,330: Created aiohttp client session
2023-04-08 16:17:44,330: Completed test setup
2023-04-08 16:17:44,330:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 16:17:44,330: ERROR t_error (root file) [2]
2023-04-08 16:17:44,331: FAIL t_fail (root file) [3]
2023-04-08 16:17:44,331: PASS t_pass (root file) [4]
2023-04-08 16:17:44,332:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 5
Start: 2023-04-08 16-17-44
End: 2023-04-08 16-17-44
Duration: 0.001355524 seconds
Tags: ['SMOKE']
PASS: 1
FAIL: 1
ERROR: 1
DISABLE: 1
SKIP: 1
LOW: 0
MINOR: 0
MAJOR: 1
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 16. Multi bug report
- 5 async tests with different bug priority values
- Priority list: LOW, MINOR, MAJOR, CRITICAL, BLOCKER
Click here to view the html report
Console output
2023-04-08 16:30:21,400:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 16:30:21,400: Starting test setup
2023-04-08 16:30:21,400: Auto detecting test suites
2023-04-08 16:30:21,400: Total test suites: 1
2023-04-08 16:30:21,400: Auto detecting tests
2023-04-08 16:30:21,401: Total synchronous tests: 0
2023-04-08 16:30:21,401: Total asynchronous tests: 5
2023-04-08 16:30:21,401: Total tests: 5
2023-04-08 16:30:21,402: Created aiohttp client session
2023-04-08 16:30:21,402: Completed test setup
2023-04-08 16:30:21,402:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 16:30:21,402: FAIL t_blocker (root file) [1]
2023-04-08 16:30:21,404: FAIL t_critical (root file) [2]
2023-04-08 16:30:21,404: FAIL t_low (root file) [3]
2023-04-08 16:30:21,404: FAIL t_major (root file) [4]
2023-04-08 16:30:21,404: FAIL t_minor (root file) [5]
2023-04-08 16:30:21,406:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 5
Start: 2023-04-08 16-30-21
End: 2023-04-08 16-30-21
Duration: 0.003122595 seconds
Tags: []
PASS: 0
FAIL: 5
ERROR: 0
DISABLE: 0
SKIP: 0
LOW: 1
MINOR: 1
MAJOR: 1
CRITICAL: 1
BLOCKER: 1
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 07. Rest api errors
- Developers can make errors while coding
- General error exception messages will increase the time to find and fix these issues
- test_rest_api provides short & precise error's with exact traceback info
- Here we are purposefully making errors in RestApi functions
Click here to view the html report
Console output
2023-04-08 19:41:19,099:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 19:41:19,099: Starting test setup
2023-04-08 19:41:19,099: Auto detecting test suites
2023-04-08 19:41:19,099: Total test suites: 1
2023-04-08 19:41:19,099: Auto detecting tests
2023-04-08 19:41:19,099: Total synchronous tests: 0
2023-04-08 19:41:19,099: Total asynchronous tests: 12
2023-04-08 19:41:19,099: Total tests: 12
2023-04-08 19:41:19,099: Created aiohttp client session
2023-04-08 19:41:19,099: Completed test setup
2023-04-08 19:41:19,099:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 19:41:19,100: ERROR t1 (root file) [1]
2023-04-08 19:41:19,101: ERROR t10 (root file) [2]
2023-04-08 19:41:19,110: ERROR t2 (root file) [5]
2023-04-08 19:41:19,110: ERROR t3 (root file) [6]
2023-04-08 19:41:19,110: ERROR t4 (root file) [7]
2023-04-08 19:41:19,110: ERROR t5 (root file) [8]
2023-04-08 19:41:19,110: ERROR t6 (root file) [9]
2023-04-08 19:41:19,111: ERROR t7 (root file) [10]
2023-04-08 19:41:19,111: ERROR t8 (root file) [11]
2023-04-08 19:41:19,111: ERROR t9 (root file) [12]
2023-04-08 19:41:19,115: ERROR t11 (root file) [3]
2023-04-08 19:41:19,285: ERROR t12 (root file) [4]
2023-04-08 19:41:19,288:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 12
Start: 2023-04-08 19-41-19
End: 2023-04-08 19-41-19
Duration: 0.18714125 seconds
Tags: []
PASS: 0
FAIL: 0
ERROR: 12
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 12
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 08. Global variables errors
- Developers can make errors while coding
- General error exception messages will increase the time to find and fix these issues
- test_rest_api provides short & precise error's with exact traceback info
- Here we are purposefully making errors in GlobalVariables functions
Click here to view the html report
Console output
2023-04-08 18:38:05,825:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 18:38:05,825: Starting test setup
2023-04-08 18:38:05,825: Auto detecting test suites
2023-04-08 18:38:05,826: Total test suites: 1
2023-04-08 18:38:05,826: Auto detecting tests
2023-04-08 18:38:05,827: Total synchronous tests: 0
2023-04-08 18:38:05,827: Total asynchronous tests: 13
2023-04-08 18:38:05,827: Total tests: 13
2023-04-08 18:38:05,827: Created aiohttp client session
2023-04-08 18:38:05,827: Completed test setup
2023-04-08 18:38:05,827:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 18:38:05,827: ERROR t_1 (root file) [1]
2023-04-08 18:38:05,828: ERROR t_10 (root file) [2]
2023-04-08 18:38:05,828: ERROR t_11 (root file) [3]
2023-04-08 18:38:05,828: ERROR t_12 (root file) [4]
2023-04-08 18:38:05,829: ERROR t_13 (root file) [5]
2023-04-08 18:38:05,829: ERROR t_2 (root file) [6]
2023-04-08 18:38:05,829: ERROR t_3 (root file) [7]
2023-04-08 18:38:05,829: ERROR t_4 (root file) [8]
2023-04-08 18:38:05,829: ERROR t_5 (root file) [9]
2023-04-08 18:38:05,830: ERROR t_6 (root file) [10]
2023-04-08 18:38:05,830: ERROR t_7 (root file) [11]
2023-04-08 18:38:05,830: ERROR t_8 (root file) [12]
2023-04-08 18:38:05,830: ERROR t_9 (root file) [13]
2023-04-08 18:38:05,831:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 13
Start: 2023-04-08 18-38-05
End: 2023-04-08 18-38-05
Duration: 0.003679225 seconds
Tags: []
PASS: 0
FAIL: 0
ERROR: 13
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 13
BUG: 0
LOGGER: 0
UNEXPECTED: 09. Bug errors
- Developers can make errors while coding
- General error exception messages will increase the time to find and fix these issues
- test_rest_api provides short & precise error's with exact traceback info
- Here we are purposefully making errors in Bug functions
Click here to view the html report
Console output
2023-04-08 19:55:51,870:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 19:55:51,871: Starting test setup
2023-04-08 19:55:51,871: Auto detecting test suites
2023-04-08 19:55:51,871: Total test suites: 1
2023-04-08 19:55:51,871: Auto detecting tests
2023-04-08 19:55:51,872: Total synchronous tests: 0
2023-04-08 19:55:51,872: Total asynchronous tests: 7
2023-04-08 19:55:51,872: Total tests: 7
2023-04-08 19:55:51,872: Created aiohttp client session
2023-04-08 19:55:51,872: Completed test setup
2023-04-08 19:55:51,872:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 19:55:51,872: ERROR t1 (root file) [1]
2023-04-08 19:55:51,874: ERROR t2 (root file) [2]
2023-04-08 19:55:51,875: ERROR t3 (root file) [3]
2023-04-08 19:55:51,875: ERROR t4 (root file) [4]
2023-04-08 19:55:51,875: ERROR t5 (root file) [5]
2023-04-08 19:55:51,875: ERROR t6 (root file) [6]
2023-04-08 19:55:51,876: ERROR t7 (root file) [7]
2023-04-08 19:55:51,877:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 7
Start: 2023-04-08 19-55-51
End: 2023-04-08 19-55-51
Duration: 0.003555115 seconds
Tags: []
PASS: 0
FAIL: 0
ERROR: 7
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 7
LOGGER: 0
UNEXPECTED: 010. Logger errors
- Developers can make errors while coding
- General error exception messages will increase the time to find and fix these issues
- test_rest_api provides short & precise error's with exact traceback info
- Here we are purposefully making errors in Logger functions
Click here to view the html report
Console output
2023-04-08 21:54:56,023:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 21:54:56,023: Starting test setup
2023-04-08 21:54:56,023: Auto detecting test suites
2023-04-08 21:54:56,023: Total test suites: 1
2023-04-08 21:54:56,023: Auto detecting tests
2023-04-08 21:54:56,023: Total synchronous tests: 0
2023-04-08 21:54:56,023: Total asynchronous tests: 7
2023-04-08 21:54:56,023: Total tests: 7
2023-04-08 21:54:56,023: Created aiohttp client session
2023-04-08 21:54:56,023: Completed test setup
2023-04-08 21:54:56,023:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 21:54:56,023: ERROR t1 (root file) [1]
2023-04-08 21:54:56,025: ERROR t2 (root file) [2]
2023-04-08 21:54:56,025: ERROR t3 (root file) [3]
2023-04-08 21:54:56,026: ERROR t4 (root file) [4]
2023-04-08 21:54:56,026: ERROR t5 (root file) [5]
2023-04-08 21:54:56,026: ERROR t6 (root file) [6]
2023-04-08 21:54:56,026: ERROR t7 (root file) [7]
2023-04-08 21:54:56,027:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 7
Start: 2023-04-08 21-54-56
End: 2023-04-08 21-54-56
Duration: 0.003578533 seconds
Tags: []
PASS: 0
FAIL: 0
ERROR: 7
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 7
UNEXPECTED: 011. Unexpected errors
Click here to view the html report
- Developers can make errors while coding
- General error exception messages will increase the time to find and fix these issues
- test_rest_api provides short & precise error's with exact traceback info
- Here we are purposefully making unexpected errors in tests
Console output
2023-04-08 21:58:04,529:
=======================================================
|| ..................................................... ||
|| ..................................................... ||
|| ........... T E S T - R E S T - A P I ............ ||
|| ..................................................... ||
|| ..................................................... ||
=======================================================
2023-04-08 21:58:04,529: Starting test setup
2023-04-08 21:58:04,530: Auto detecting test suites
2023-04-08 21:58:04,530: Total test suites: 1
2023-04-08 21:58:04,530: Auto detecting tests
2023-04-08 21:58:04,530: Total synchronous tests: 0
2023-04-08 21:58:04,530: Total asynchronous tests: 6
2023-04-08 21:58:04,530: Total tests: 6
2023-04-08 21:58:04,531: Created aiohttp client session
2023-04-08 21:58:04,531: Completed test setup
2023-04-08 21:58:04,531:
=======================================================
|| ............... A S Y N C - T E S T S ............... ||
=======================================================
2023-04-08 21:58:04,531: ERROR t1 (root file) [1]
2023-04-08 21:58:04,532: ERROR t2 (root file) [2]
2023-04-08 21:58:04,533: ERROR t3 (root file) [3]
2023-04-08 21:58:04,534: ERROR t4 (root file) [4]
2023-04-08 21:58:04,534: ERROR t5 (root file) [5]
2023-04-08 21:58:04,534: ERROR t6 (root file) [6]
2023-04-08 21:58:04,535:
=======================================================
|| ..................................................... ||
|| ............ T E S T - S U M M A R Y ............. ||
|| ..................................................... ||
=======================================================
Status: FAIL
Tests: 6
Start: 2023-04-08 21-58-04
End: 2023-04-08 21-58-04
Duration: 0.003570664 seconds
Tags: []
PASS: 0
FAIL: 0
ERROR: 6
DISABLE: 0
SKIP: 0
LOW: 0
MINOR: 0
MAJOR: 0
CRITICAL: 0
BLOCKER: 0
REST API: 0
GLOBAL VARIABLES: 0
BUG: 0
LOGGER: 0
UNEXPECTED: 6Author
License
Copyright Troy M Jose, 2023.
Distributed under the terms of the MIT license, test_rest_api is free and open source software.
