Project description

eztest is a Python package and execution used for performance/load testing.

Home page: https://github.com/lgt1001/eztest

Features

Test:

Normal: Run selected cases only once.
Continuous: Run cases [repeat] times with [interval] seconds' sleeping.
Simultaneous: Start [stress] threads and run cases in each thread, sleep [interval] seconds after all cases are finished, and then start testing again with [repeat] times.
Concurrency: Start [stress] threads and each thread will continuously run cases with [interval] seconds' sleeping.
Frequent: Start [stress] threads per [interval] seconds. And only can have [limit] available threads running.

Report:

Start|Stop report server, and eztest can send and save report on remote server.
Dump failure rate and average of time taken from remote report server.
Calculate failure rate and average of time taken for report files generated by eztest.

Control:

Stop testing and report server.

Usage

eztest command:

$ eztest -h
usage: eztest [-h] [--version] {test,stop,calc,server,dump} ...

eztest

positional arguments:
  {test,stop,calc,server,dump}
    test                Start eztest for target cases, classes, modules.
    stop                Stop eztest and its report server.
    calc                Calculate report files generated by eztest.
    server              Start|Stop|Restart report server.
    dump                Dump data from report server.

optional arguments:
  -h, --help            show this help message and exit
  --version, -v         show program's version number and exit

usage: eztest [-h] [--target TARGET] [--classes CLASSES [CLASSES ...]]
              [--not-classes NOT_CLASSES [NOT_CLASSES ...]]
              [--cases CASES [CASES ...]]
              [--not-cases NOT_CASES [NOT_CASES ...]]
              [--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}]
              [--stress STRESS] [--repeat REPEAT] [--interval INTERVAL]
              [--limit LIMIT] [--starts STARTS] [--duration DURATION]
              [--ends ENDS] [--mail-config MAIL_CONFIG]
              [--report-folder REPORT_FOLDER] [--noreport] [--nolog]
              [--calc CALC [CALC ...]] [--group-minutes GROUP_MINUTES]
              [--version]

eztest test command:

$ eztest test -h
usage: eztest test [-h] --target TARGET [--classes CLASSES [CLASSES ...]]
               [--not-classes NOT_CLASSES [NOT_CLASSES ...]]
               [--cases CASES [CASES ...]]
               [--not-cases NOT_CASES [NOT_CASES ...]]
               [--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}]
               [--stress STRESS] [--repeat REPEAT] [--interval INTERVAL]
               [--limit LIMIT] [--starts STARTS] [--duration DURATION]
               [--ends ENDS] [--report-folder REPORT_FOLDER]
               [--report-server REPORT_SERVER] [--noreport] [--nolog]
               [--mail-config MAIL_CONFIG]

optional arguments:
  -h, --help            show this help message and exit

Case Group:
  Define arguments of case related.

  --target TARGET, -t TARGET
                        Folder or file path, or a module, a __init__.py file is required under that folder/module.
  --classes CLASSES [CLASSES ...], -cl CLASSES [CLASSES ...]
                        Class names to be tested. It will be considered if target is file.
  --not-classes NOT_CLASSES [NOT_CLASSES ...], -ncl NOT_CLASSES [NOT_CLASSES ...]
                        Class names to be ignored. It will be considered if target is file.
  --cases CASES [CASES ...], -c CASES [CASES ...]
                        Case names to be tested.
                        It can be whole case name or part of them(e.g.: "*a", "a*", "*a*").
  --not-cases NOT_CASES [NOT_CASES ...], -nc NOT_CASES [NOT_CASES ...]
                        Case names to be ignored.
                        It can be whole case name or part of them(e.g.: "*a", "a*", "*a*").

Test Mode Group:
  Define arguments of test mode related.

  --mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent},
  -m {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}
                        (a)0 or normal: Run selected cases only once.
                        (b)1 or continuous: Run cases [repeat] times with [interval] seconds' sleeping.
                        (c)2 or simultaneous: Start [stress] threads and run cases in each thread,
                        sleep [interval] seconds after all cases are finished,
                        and then start testing again with [repeat] times.
                        (d)3 or concurrency: Start [stress] threads and each thread will continuously run
                        cases with [interval] seconds' sleeping.
                        (e)4 or frequent: Start [stress] threads per [interval] seconds.
                        And only can have [limit] available threads running.
  --stress STRESS, -s STRESS
                        Start [stress] threads in each round of testing. Default value is 1.
  --repeat REPEAT, -r REPEAT
                        Repeat [repeat] times of testing. Default value is 1
  --interval INTERVAL, -i INTERVAL
                        Sleep [interval] seconds after one round of testing. Default value is 0.
  --limit LIMIT, -l LIMIT
                        Only can have [limit] count of running threads.
                        No limitation if this is less than or equals to [stress].
  --starts STARTS, -st STARTS
                        Testing will be started at [starts]. It is datetime string(e.g.: "2014-01-02 03:04:05").
  --duration DURATION, -d DURATION
                        Testing will continue with [duration] minutes. Will be ignored if 'ends' is provided.
  --ends ENDS, -et ENDS
                        Testing will be stopped at [ends]. It is datetime string(e.g.: "2014-01-02 03:04:05").

Report/Log Group:
  Define arguments of report or log related.

  --report-folder REPORT_FOLDER, -rf REPORT_FOLDER
                        Report and log files will be saved under [report-folder].
  --report-server REPORT_SERVER, -rs REPORT_SERVER
                        Report server.
                        The format is "host_name:port_number" or "host_name" with default port number 8765.
  --noreport, -nr       No report file will be generated if [noreport] is clarified.
  --nolog, -nl          No log file will be generated if [nolog] is clarified.
  --mail-config MAIL_CONFIG, -mc MAIL_CONFIG
                        Mail configuration file which contains mail server information etc.
                        It should be INI format file(http://en.wikipedia.org/wiki/INI_file).
                        Will send report by mail only if mail-config is provided and report file is generated.
                        Section is "SMTP" and properties can be "server", "from_mail", "to_mails",
                        "cc_mails", "bcc_mails", "username", "password", "need_authentication" and "subject".
                        "server", "from_mail" and "to_mails" are mandatory.
                        "to_mails", "cc_mails" and "bcc_mails" can be multiple values separated by comma.
                        "need_authentication" is boolean, "username" and "password" are required if
                        "need_authentication" is True.

eztest server command:

$eztest server -h
usage: eztest server [-h] {start,stop} ...

positional arguments:
  {start,stop}

optional arguments:
  -h, --help    show this help message and exit


$eztest server start -h
usage: eztest server start [-h] [--port PORT] [--handler HANDLER]

optional arguments:
  -h, --help            show this help message and exit
  --port PORT, -p PORT  Port number.
  --handler HANDLER, -hl HANDLER
                        Custom handler.
                        The format is: "file_path:handler_class_name", or "module_name:handler_class_name".

eztest dump command:

$eztest dump -h
usage: eztest dump [-h] [--report-server REPORT_SERVER]

optional arguments:
  -h, --help            show this help message and exit
  --report-server REPORT_SERVER, -rs REPORT_SERVER
                        Report server.
                        The format is "host_name:port_number" or "host_name" with default port number 8765.

eztest calc command:

$eztest calc -h
usage: eztest calc [-h] --path PATH [PATH ...] [--group-minutes GROUP_MINUTES]

optional arguments:
  -h, --help            show this help message and exit
  --path PATH [PATH ...], -p PATH [PATH ...]
                        Report folders or files to be calculated.
  --group-minutes GROUP_MINUTES, -gm GROUP_MINUTES
                        Calculate by grouping case results with [group-minutes] minutes. Default is 60 minutes.

Examples

Test examples:

# Normal testing
$ eztest test --target examples/target_is_unittest/test_case.py

# Continuous testing and repeat 100 times
$ eztest test --mode continuous --target examples/target_is_unittest/test_case.py --repeat 100 --nolog

# Simultaneous testing, start 50 threads and repeat 100 times
$ eztest test --mode simultaneous --target examples/target_is_unittest/test_case.py --stress 50 --repeat 100 --nolog

# Concurrency testing, start 50 threads and run 1 hour
$ eztest test --mode simultaneous --target examples/target_is_unittest/test_case.py --stress 50 --duration 60 --nolog

# Frequent testing, start 50 threads and run 1 hour
$ eztest test --mode frequent --target examples/target_is_unittest/test_case.py --stress 50 --duration 60 --nolog

# Ignore cases
$ eztest test --target examples/target_is_unittest/test_case.py --not-cases test_hello

# Target is a module with CASES defined.
$ eztest test --target examples.target_is_module

# Send and save case report to remote server.
$ eztest test --target examples.target_is_module --report-server localhost:8765

# Stop testing or report server
$ eztest stop

Report related examples:

# Start report server.
$ eztest server start --port 8765

# Stop report server.
$ eztest server stop

# Dump testing summary from remote report server
$ eztest dump --report-server localhost:8765

# Calculate failure rate and average of time taken for report files.
$ eztest calc --path "/tmp/a.csv" "/tmp/b.csv" --group-minutes 30

# Calculate failure rate and average of time taken for files under report folder.
$ eztest calc --path "/tmp/reports" --group-minutes 30

Prerequisites

C Python 2.7, 3.2 and higher.
psutil https://pypi.org/project/psutil/

Authors

lgt

License

GNU GPL v2, see http://www.gnu.org/licenses/gpl-2.0.html

eztest
Release 2.0.1

Release 2.0.1

2.0.1

2.0.0

1.0.10

1.0.9

1.0.8

1.0.7

1.0.6

1.0.5

1.0.4

1.0.3

Documentation

Project description

Features

Usage

Examples

Prerequisites

Authors

License

Stats

Development practices

Releases

Contributors

eztest Release 2.0.1

Release 2.0.1 Toggle Dropdown 2.0.1 2.0.0 1.0.10 1.0.9 1.0.8 1.0.7 1.0.6 1.0.5 1.0.4 1.0.3

Documentation

Project description

Features

Usage

Examples

Prerequisites

Authors

License

Stats

Development practices

Releases

Contributors

eztest
Release 2.0.1

Release 2.0.1

2.0.1

2.0.0

1.0.10

1.0.9

1.0.8

1.0.7

1.0.6

1.0.5

1.0.4

1.0.3