Screen Actions
Screen Actions are actions to interact with the computer screen.
Actions
screen.crop_screen
Takes a screenshot from desired part of the screen.
Parameters:
x - Left most position of the screen to be croped
y - Top most position of the screen to be croped
width - Width of the screen to crop
height - Height of the screen to crop
save_path - Path to save the croped screenshot image
Returns:
screenshot - Image of entire screen
crop - Image of croped area
Exceptions:
This action gives no exception
screen.find_image
Find the best match of template image, from assets, inside the screen and return the position on screen.
Parameters:
template_image - name of the asset image used as template.
threshold (optional) - minimum match to consider that the template was found (default=0.9 - 90%).
screenshot_img (optional) - if the template should be found on a given screen beside on current screen (default=None - take a new screenshot).
Returns:
x - left position of the found image on screen/screenshot_img
y - top position of the found image on screen/screenshot_img
width - width of the found image on screen/screenshot_img
height - height of the found image on screen/screenshot_img
Exceptions:
ImageNotFoundException - if no matching image were found until the timeout is reached
screen.find_image_center
Find the best match of template image, from assets, inside the screen and return the center position of the found image.
Parameters:
template_image - name of the asset image used as template.
threshold (optional) - minimum match to consider that the template was found (default=0.9 - 90%).
screenshot_img (optional) - if the template should be found on a given screen beside on current screen (default=None - take a new screenshot).
Returns:
x - center position from left of the found image on screen/screenshot_img
y - center position from top of the found image on screen/screenshot_img
screen.find_image_multiple
Find all matches of template image, inside the screen and return its position.
Parameters:
template_image - name of the asset image used as template.
threshold (optional) - minimum match to consider that the template was found (default=0.9 - 90%).
screenshot_img (optional) - if the template should be found on a given screen beside on current screen (default=None - take a new screenshot).
overlap_factor (optional) - percentage of overlap to consider the same image. If the overlap between two images is higher than the overlap_factor, the two found images are considered to be the same (default=0.5 - 50%).
Returns:
List of image posisions, with values:
x[num] - left position of the found image on screen/screenshot_img
y[num] - top position of the found image on screen/screenshot_img
width[num] - width of the found image on screen/screenshot_img
height[num] - height of the found image on screen/screenshot_img
length - number of images found
screen.find_image_multiple_center
Find all matches of template image, inside the screen and return its center position.
Parameters:
template_image - name of the asset image used as template.
threshold (optional) - minimum match to consider that the template was found (default=0.9 - 90%).
screenshot_img (optional) - if the template should be found on a given screen beside on current screen (default=None - take a new screenshot).
overlap_factor (optional) - percentage of overlap to consider the same image. If the overlap between two images is higher than the overlap_factor, the two found images are considered to be the same (default=0.5 - 50%).
Returns:
List of image posisions, with values:
x[num] - center position from left of the found image on screen/screenshot_img
y[num] - center position from top of the found image on screen/screenshot_img
length - number of images found
screen.screenshot
Takes a screenshot from entire screen.
Parameters:
This action has no parameters
Returns:
screenshot - Image of entire screen
Screen actions
The actions below reflect exactly the functions exposed by screen.py.
Note: action return values become robot state variables — use the exact return key name prefixed with $.
Only functions decorated with @decorators.robotaction are documented.
screen.crop_screen
Capture the screen (or use the provided screen) and save/return the cropped area.
Signature / Parameters (use these exact names):
- x
- y
- width
- height
- save_path
- screen (optional)
Returns (robot state variables):
- screenshot — full captured image (numpy array).
- crop — cropped image (numpy array).
Exceptions:
- FileNotFoundException
- InvalidImageExtension
screen.screenshot
Takes a screenshot and returns the image.
Parameters: none
Returns:
- screenshot — full captured image (numpy array).
screen.find_image
Finds the best match for template_image and returns position and size.
Parameters:
- template_image
- threshold (optional, default=0.9)
- screenshot_img (optional)
Returns (robot state variables):
- x
- y
- width
- height
Exceptions:
- ImageNotFoundException
screen.find_image_center
Same output as find_image, but x/y are adjusted to the center of the found area.
Parameters: same as find_image.
Returns:
- x, y, width, height (with centered x/y).
screen.find_image_multiple
Finds multiple occurrences of template_image and returns indexed positions.
Parameters:
- template_image
- threshold (optional, default=0.9)
- screenshot_img (optional)
- overlap_factor (optional, default=0.5)
Returns (robot state variables):
- x[0], y[0], width[0], height[0], x[1], y[1], ...
- length — number of occurrences found.
Exceptions:
- ImageNotFoundException
screen.find_image_multiple_center
Same as find_image_multiple, but x[n]/y[n] are adjusted to the center of each occurrence.
Parameters: same as find_image_multiple.
Returns:
- x[n], y[n], width[n], height[n], length.
screen.wait_image
Waits until template_image appears on screen or until timeout.
Parameters:
- template_image
- threshold (optional, default=0.9)
- screenshot_img (optional)
- timeout (optional, default=60)
Returns (robot state variables):
- x, y, width, height — position/size of the found image.
Exceptions:
- ImageNotFoundException
screen.wait_images
Waits until any image in template_image_list appears on screen.
Parameters:
- template_image_list
- threshold (optional, default=0.9)
- screenshot_img (optional)
- timeout (optional, default=60)
Returns (robot state variables):
- image_found — name of the template that was located.
- x, y, width, height — position/size of the found image.
Exceptions:
- ImageNotFoundException
Example:
screen.wait_images(["error-1.png", "error-2.png"])
if $image_found == "error-1.png":
prompt.alert("Error 1 found on screen")
elif $image_found == "error-2.png":
prompt.alert("Error 2 found on screen")
screen.wait_image_disapear
Waits until template_image disappears from the screen (note function name: wait_image_disapear).
Parameters:
- template_image
- threshold (optional, default=0.9)
- screenshot_img (optional)
- timeout (optional, default=60)
Returns: - none
Exceptions:
- TimeoutException