discogui

GUI discovery


Keywords
GUI
License
BSD-3-Clause
Install
pip install discogui==0.4

Documentation

Experimental Python library for discovering GUI elements.

Links:

workflow

Features:

  • python module
  • works on Linux
  • does not depend on Accessibility technologies
  • toolkit independent
  • only basic tests on very simple GUI
  • GUI should be displayed on Xvfb or Xephyr

Possible applications:

  • automatic GUI testing
  • automatic GUI control

Installation::

$ sudo apt install xvfb
$ python3 -m pip install discogui

Usage

button discovery on zenity using tab order

# discogui/examples/taborder.py

"""
1. start zenity Yes/No dialog on Xvfb
2. discover buttons using tab order: 
    send TAB keyboard events and check how the screen changes
3. print button rectangle positions
4. draw red rectangles for buttons on screenshot
"""
from easyprocess import EasyProcess
from pyvirtualdisplay.smartdisplay import SmartDisplay

from discogui.buttons import discover_buttons
from discogui.draw import draw_indexed_rect_list
from discogui.imgutil import autocrop

with SmartDisplay(visible=False) as disp:
    with EasyProcess(["zenity", "--question"]):
        img = disp.waitgrab(timeout=60, autocrop=False)
        rectangles = discover_buttons()
        print(rectangles)

img = draw_indexed_rect_list(img, rectangles)
img = autocrop(img)

# save results
img.save("taborder.png")

Run it:

$ python3 -m discogui.examples.taborder
[ScreenRect((487,405,569,435)), ScreenRect((576,405,658,435))]

Image:

button discovery on zenity using mouse hovering

# discogui/examples/hover.py

"""
1. start zenity Yes/No dialog on Xvfb
2. discover buttons using mouse hovering: 
    move the mouse in grid positions and check how the screen changes
3. print button rectangle positions
4. draw red rectangles for buttons on screenshot
"""
from easyprocess import EasyProcess
from pyvirtualdisplay.smartdisplay import SmartDisplay

from discogui.draw import draw_indexed_rect_list
from discogui.hover import active_rectangles
from discogui.imgutil import autocrop

with SmartDisplay(size=(640, 480), visible=False) as disp:
    with EasyProcess(["zenity", "--question"]):
        img = disp.waitgrab(timeout=60, autocrop=False)
        rectangles = active_rectangles(grid=30)
        print(rectangles)
img = draw_indexed_rect_list(img, rectangles)
img = autocrop(img)

# save results
img.save("hover.png")

Run it:

$ python3 -m discogui.examples.hover
[ScreenRect((295,260,378,292)), ScreenRect((384,260,467,292))]

Image:

button test

# discogui/examples/clickbutton.py

"""
1. start zenity Yes/No dialog on Xvfb
2. discover buttons using `discogui.buttons` module
3. click first button, print return code
4. click second button, print return code
"""

from easyprocess import EasyProcess
from pyvirtualdisplay.smartdisplay import SmartDisplay

from discogui.buttons import discover_buttons
from discogui.mouse import PyMouse


def click_button_get_return_code(disp, which_button):
    with EasyProcess(["zenity", "--question"]) as p:
        disp.waitgrab(timeout=60)
        rectangles = discover_buttons()
        PyMouse().click(*rectangles[which_button].center)
        return p.wait().return_code


with SmartDisplay() as disp:
    print(click_button_get_return_code(disp, 0))
with SmartDisplay() as disp:
    print(click_button_get_return_code(disp, 1))

Run it:

$ python3 -m discogui.examples.clickbutton
1
0