Region and SikuliEvent classes to fulfill the roles of those described at http://doc.sikuli.org/region.html
Manages interaction with Sikuli’s Region, reflecting http://doc.sikuli.org/region.html#Region
Returns a new Region that is defined above the current region’s top border with a height of range number of pixels. So it does not include the current region. If range is omitted, it reaches to the top of the screen. The new region has the same width and x-position as the current region.
| Parameters: | range_px – a positive integer defining the new height |
|---|---|
| Return type: | a Region object |
Returns a new Region that is defined below the current region’s bottom border with a height of range number of pixels. So it does not include the current region. If range is omitted, it reaches to the bottom of the screen. The new region has the same width and x-position as the current region.
| Parameters: | range_px – a positive integer defining the new height |
|---|---|
| Return type: | a Region object |
Perform a mouse click on the click point using the left button.
| Parameters: |
|
|---|---|
| Return type: | the number of performed clicks (actually 1). A 0 (integer null) means that because of some reason, no click could be performed (in case of PS may be PatternNotFound). |
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Example:
# Windows XP
click("xpstart.png")
# Windows Vista
click("vistastart.png")
# Windows 7
click("w7start.png")
# Mac
click("apple.png")
Perform a mouse double-click on the click point using the left button.
| Parameters: |
|
|---|---|
| Return type: | the number of performed double-clicks (actually 1). A 0 ( integer null) means that because of some reason, no click could be performed (in case of PS may be PatternNotFound). |
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Start a drag-and-drop operation by dragging at the given click point.
| Parameters: | PSMRL – a pattern, a string, a match, a region or a location that evaluates to a click point. |
|---|---|
| Return type: | the number 1 if the operation could be performed. A 0 (integer null) returned means that because of some reason, no move could be performed (in case of PS may be PatternNotFound). |
The mousepointer is moved to the click point and the left mouse button is pressed and held, until another mouse action is performed (e.g. a Region.dropAt() afterwards). This is nomally used to start a drag-and-drop operation.
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Perform a drag-and-drop operation from a starting click point to the target click point indicated by the two PSMRLs respectively.
| Parameters: |
|
|---|
If one of the parameters is PS, the operation might fail due to PatternNotFound.
Sideeffect: when using PS, the match of the target can be accessed using Region.getLastMatch() afterwards. If only the first parameter is given as PS, this match is returned by Region.getLastMatch().
When the operation does not perform as expected (usually caused by timing problems due to delayed reactions of applications), you may adjust the internal timing parameters Settings.DelayAfterDrag and Settings.DelayBeforeDrop eventually combined with the internal timing parameter Settings.MoveMouseDelay.
Another solution might be, to use a combination of Region.drag() and Region.dropAt() combined with your own wait() usages. If the mouse movement from source to target is the problem, you might break up the move path into short steps using Region.mouseMove().
Complete a drag-and-drop operation by dropping a previously dragged item at the given target click point.
| Parameters: |
|
|---|---|
| Return type: | the number 1 if the operation could be performed. A 0 (integer null) returned means that because of some reason, no move could be performed (in case of PS may be PatternNotFound). |
The mousepointer is moved to the click point. After waiting for delay seconds the left mouse button is released. This is normally used to finalize a drag-and-drop operation. If it is necessary to visit one ore more click points after dragging and before dropping, you can use Region.mouseMove() inbetween.
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Check whether the give pattern is visible on the screen.
| Parameters: |
|
|---|---|
| Return type: | a Match object that contains the best match or fails if PatternNotFound. None is returned, if nothing is found within the specified waiting time. |
Does exactly the same as Region.wait(), but no exception is raised in case of FindFailed. So it can be used to symplify scripting in case that you only want to know wether something is there or not to decide how to proceed in your workflow. So it is typically used with an if statement. At least one find operation is performed, even if 0 seconds is specified. So specifying 0 seconds saves some time, in case there is no need to wait, since its your intention to get the information “not found” directly.
Side Effect lastMatch: the best match can be accessed using Region.getLastMatch() afterwards.
Note: You may adjust the scan rate (how often a search during the wait takes place) by setting Settings.WaitScanRate appropriately.
| Parameters: | PS – a Pattern object or a string (path to an image file or just plain text) |
|---|---|
| Return type: | a Match object that contains the best match or fails if PatternNotFound |
Find a particular GUI element, which is seen as the given image or just plain text. The given file name of an image specifies the element’s appearance. It searches within the region and returns the best match, which shows a similarity greater than the minimum similarity given by the pattern. If no similarity was set for the pattern by Pattern.similar() before, a default minimum similarity of 0.7 is set automatically.
If autoWaitTimeout is set to a non-zero value, find() just acts as a wait().
Side Effect lastMatch: the best match can be accessed using Region.getLastMatch() afterwards.
| Parameters: | PS – a Pattern object or a string (path to an image file or just plain text) |
|---|---|
| Return type: | one or more Match objects as an iterator object or fails if PatternNotFound |
Repeatedly find ALL instances of a pattern, until no match can be found anymore, that meets the requirements for a single Region.find() with the specified pattern.
By default, the returned matches are sorted by the similiarty. If you need them ordered by their positions, say the Y coordinates, you have to use Python’s sorted function. Here is a example of sorting the matches from top to bottom.
def by_y(match):
return match.y
icons = findAll("png_icon.png")
# sort the icons by their y coordinates and put them into a new
sorted_icons = sorted(icons, key=by_y)
# another shorter version is using lambda.
sorted_icons = sorted(icons, key=lambda m:m.y)
for icon in sorted_icons:
pass # do whatever you want to do with the sorted icons
Side Effect lastMatches: a reference to the returned iterator object containing the found matches is stored with the region that was searched. It can be accessed using getLastMatches() afterwards.
| Parameters: | images – list of strings giving paths to images |
|---|---|
| Returns: | found images |
| Return type: | float |
|---|
Get the current value of the maximum waiting time for find operations
| Return type: | Match |
|---|
All successful find operations (explicit like find() or implicit like click()), store the best match in the lastMatch attribute of the region that was searched.
| Return type: | generator |
|---|
findAll() stores all found matches into lastMatches attribute of the region that was searched as a generator.
Returns a new Location object derived from the given parameter. In case of PS, internally a find() is done inside this region. If found, the match’s target offset position is returned. In case RM, just a copy of the given region’s respective location (center or target offset) is returned.
| Parameters: | PSMRL – a Pattern, String, Region, Match or Location object |
|---|---|
| Return type: | a new Location object |
Returns a new Region object derived from the given parameter. In case of PS, internally a find() is done inside this region. If found, the match is returned. In case RM, just a copy of the given region is returned.
| Parameters: | PSRM – a Pattern, String, Region or Match object |
|---|---|
| Return type: | a new Region object |
| Return type: | Screen |
|---|
Returns the screen object that contains this region.
This method only makes sense in Multi Monitor Environments, since it always returns the default screen in a single monitor environment.
Highlight the region for some period of time.
| Parameters: | seconds – a decimal number taken as duration in seconds |
|---|
The region is highlighted showing a red colored frame around it. If the parameter seconds is given, the script is suspended for the specified time. If no time is given, the highlighting is started and the script continues.
When later on the same highlight call without a parameter is made, the highlighting is stopped (behaves like a toggling switch).
Example:
m = find(some_image)
# the red frame will blink for about 7 - 8 seconds
for i in range(5):
m.highlight(1)
wait(0.5)
# a second red frame will blink as an overlay to the first one
m.highlight()
for i in range(5):
m.highlight(1)
wait(0.5)
m.highlight()
# the red frame will grow 5 times
for i in range(5):
m.highlight(1)
m = m.nearby(20)
Note: The red frame is just an overlay in front of all other screen content and stays in its place, independently from the behavior of this other content, which means it is not “connected” to the content of the defining region. But it will be adjusted automatically, if you change position and/or dimension of this region in your script, while it is highlighted.
Move the mouse cursor to hover above a click point.
| Parameters: |
|
|---|---|
| Return type: | the number 1 if the mousepointer could be moved to the click point. A 0 (integer null) returned means that because of some reason, no move could be performed (in case of PS may be PatternNotFound). |
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Returns the same object. Retained for upward compatibility.
This method can be used to make scripts more readable. region.inside().find() is totally equivalent to region.find().
| Return type: | Region |
|---|
Press and hold the specified key(s) until released by a later call to Region.keyUp().
| Parameters: | key – one or more keys (use the consts of class Key). A list of keys is a concatenation of several key constants using “+”. |
|---|---|
| Return type: | the number 1 if the operation could be performed and 0 if otherwise. |
Release given keys. If no key is given, all currently pressed keys are released.
| Parameters: | key – one or more keys (use the consts of class Key). A list of keys is a concatenation of several key constants using “+”. |
|---|---|
| Return type: | the number 1 if the operation could be performed and 0 if otherwise. |
Returns a new Region that is defined left of the current region’s left border with a width of range number of pixels. So it does not include the current region. If range is omitted, it reaches to the left border of the screen. The new region has the same height and y-position as the current region.
| Parameters: | range_px – a positive integer defining the new width |
|---|---|
| Return type: | a Region object |
| Parameters: | region – Region |
|---|
Set the position and dimension of this region to the corresponding values of the region given as parameter.
Press the mouse button down.
| Parameters: | button – one or a combination of the button constants Button.LEFT, Button.MIDDLE, Button.RIGHT. |
|---|---|
| Return type: | the number 1 if the operation is performed successfully, and zero if otherwise. |
The mouse button or buttons specified by button are pressed until another mouse action is performed.
Move the mouse pointer to a location indicated by PSRML.
| Parameters: | PSMRL – a pattern, a string, a match, a region or a location that evaluates to a click point. |
|---|---|
| Return type: | the number 1 if the operation could be performed. If using PS (which invokes an implicity find operation), find fails and you have switched off FindFailed exception, a 0 (integer null) is returned. Otherwise, the script is stopped with a FindFailed exception. |
Sideeffects: when using PS, the match can be accessed using Region.getLastMatch() afterwards
| Parameters: | location – Location - the new top left corner |
|---|---|
| Return type: | Region – the modified region object |
Set the position of this region regarding its top left corner to the given location (the x and y values are modified).
Returns a new Region that includes the nearby neighbourhood of the the current region. The new region is defined by extending the current region’s dimensions in all directions by range number of pixels. The center of the new region remains the same.
| Parameters: | range_px – int > 0, default = 50 |
|---|---|
| Return type: | Region |
Returns a new Region object, whose upper left corner is relocated adding the location’s x and y value to the respective values of the given region. Width and height are the same. So this clones a region at a different place.
| Parameters: | location – Location |
|---|---|
| Return type: | Region |
new_reg = reg.offset(Location(xoff, yoff)) # same as
new_reg = Region(reg.x + xoff, reg.y + yoff, reg.w, reg.h)
Paste the text at a click point.
| Parameters: |
|
|---|---|
| Return type: | the number 1 if the operation could be performed, otherwise 0 (integer null), which means, that because of some reason, it was not possible or the click could be performed (in case of PS may be PatternNotFound). |
Pastes text using the clipboard (OS-level shortcut (Ctrl-V or Cmd-V)). So afterwards your clipboard contains text. paste() is a temporary solution for typing international characters or typing on keyboard layouts other than US-QWERTY.
If PSMRL is given, a click on the clickpoint is performed before typing, to gain the focus. (Mac: it might be necessary, to use switchApp() to give focus to a target application before, to accept typed/pasted characters.)
If PSMRL is omitted, it performs the paste on the current focused component (normally an input field).
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Note: Special keys (ENTER, TAB, BACKSPACE, ...) cannot be used with paste(). If needed, you have to split your complete text into two or more paste() and use type() for typing the special keys inbetween. Characters like n (enter/new line) and t (tab) should work as expected with paste(), but be aware of timing problems, when using e.g. intervening t to jump to the next input field of a form.
Returns a new Region that is defined right of the current region’s right border with a width of range number of pixels. So it does not include the current region. If range is omitted, it reaches to the right border of the screen. The new region has the same height and y-position as the current region.
| Parameters: | range_px – a positive integer defining the new width |
|---|---|
| Return type: | a Region object |
Perform a mouse click on the click point using the right button.
| Parameters: |
|
|---|---|
| Return type: | the number of performed double-clicks (actually 1). A 0 ( integer null) means that because of some reason, no c |
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
| Parameters: | seconds – float - The internal granularity is milliseconds. |
|---|
Set the maximum waiting time for all subsequent find operations.
This method enables all find operations to wait for the given pattern to appear until the specified amount of time has elapsed. The default is 3.0 seconds. This method is intended for users to override this default setting. As such it lets Region.find() work like Region.wait(), without being able to set an individual timeout value for a specific find operation.
| Parameters: | num – number - the new value |
|---|
Set the respective attribute of the region to the new value. This effectively moves the region around and/or changes its dimension.
| Parameters: |
|
|---|
Set position and dimension to new values. The motivation for this alias is to make scripts more readable: setROI() is intended to restrict the search to a smaller area to speed up processing searches (region of interest).
| Parameters: |
|
|---|
Set position and dimension to new values. The motivation for this alias is to make scripts more readable: setRect() should be used to redefine a region (which could be enlarging it).
| Parameters: | num – number - the new value |
|---|
Set the respective attribute of the region to the new value. This effectively moves the region around and/or changes its dimension.
| Parameters: | num – number - the new value |
|---|
Set the respective attribute of the region to the new value. This effectively moves the region around and/or changes its dimension.
| Parameters: | num – number - the new value |
|---|
Set the respective attribute of the region to the new value. This effectively moves the region around and/or changes its dimension.
Extract the text contained in the region using OCR.
| Return type: | the text as a string. |
|---|
Note: Since this feature is still in an experimental state, be aware, that in some cases it might not work as expected. If you face any problems look at the Questions & Answers / FAQ’s and the Bugs.
Type the text at the current focused input field or at a click point specified by PSMRL.
| Parameters: |
|
|---|---|
| Return type: | the number 1 if the operation could be performed, otherwise 0 (integer null), which means, that because of some reason, it was not possible or the click could be performed (in case of PS may be PatternNotFound). |
This method simulates keyboard typing interpreting the characters of text based on the layout/keymap of the standard US keyboard (QWERTY). Special keys (ENTER, TAB, BACKSPACE, ...) can be incorporated into text by using the constants defined in Class Key using the standard string concatenation (+).
If PSMRL is given, a click on the clickpoint is performed before typing, to gain the focus. (Mac: it might be necessary, to use switchApp() to give focus to a target application before, to accept typed/pasted characters.)
If PSMRL is omitted, it performs the typing on the current focused visual component (normally an input field or an menu entry that can be selected by typing something).
Side Effect if PS was used, the match can be accessed using Region.getLastMatch() afterwards.
Note: If you need to type international characters or you are using layouts/keymaps other than US-QWERTY, you should use Region.paste() instead. Since type() is rather slow because it simulates each key press, for longer text it is preferrable to use Region.paste().
Best Practice: As a general guideline, the best choice is to use paste() for readable text and type() for action keys like TAB, ENTER, ESC, .... Use one type() for each key or key combination and be aware, that in some cases a short wait() after a type() might be necessary to give the target application some time to react and be prepared for the next Sikuli action.
| Parameters: |
|
|---|---|
| Return type: | a Match object that contains the best match or fails if PatternNotFound |
If PS is not specified, the script just pauses for the specified amount of time. It is still possible to use sleep(seconds) instead, but this is deprecated.
If PS is specified, it keeps searching the given pattern in the region until the image appears ( would have been found with Region.find()) or the specified amount of time has elapsed. At least one find operation is performed, even if 0 seconds is specified.)
Side Effect lastMatch: the best match can be accessed using Region.getLastMatch() afterwards.
Note: You may adjust the scan rate (how often a search during the wait takes place) by setting Settings.WaitScanRate appropriately.
| Parameters: |
|
|---|---|
| Return type: | a Match object that contains the best match or fails if PatternNotFound |
If PS is not specified, the script just pauses for the specified amount of time. It is still possible to use sleep(seconds) instead, but this is deprecated.
If PS is specified, it keeps searching the given pattern in the region until the image appears ( would have been found with Region.find()) or the specified amount of time has elapsed. At least one find operation is performed, even if 0 seconds is specified.)
Side Effect lastMatch: the best match can be accessed using Region.getLastMatch() afterwards.
Note: You may adjust the scan rate (how often a search during the wait takes place) by setting Settings.WaitScanRate appropriately.
Manages interaction with Sikuli’s SikuliEvent, reflecting http://doc.sikuli.org/region.html#SikuliEvent
Todo
Implement