如何识别图像并点击它们

Question

我想制作一个脚本，根据要求点击图像，它需要浏览图像列表。例如，如果程序要求用户点击绿色圆圈：

question_list = greencircle, redcircle, bluesquare, redtriangle
if(greencircle == greencircle.png){
pyautogui.click(greencircle.png)
}

有人能帮忙吗？

Answer 1

PyAutoGUI有一个名为locateOnScreen()的内置函数，如果它能在当前屏幕上找到图像，它会返回图像中心的x，y坐标(它会截图并分析它(。

图像必须与完全匹配才能工作；也就是说，如果你想点击button.png，那么按钮图片的大小/分辨率必须与窗口中的按钮完全相同，程序才能识别它。实现这一点的一种方法是截屏，用油漆打开它，只剪下你想要按下的按钮(或者你可以让PyAutoGUI为你做这件事，我将在后面的例子中展示(。

import pyautogui
question_list = ['greencircle', 'redcircle', 'bluesquare', 'redtriangle']
user_input = input('Where should I click? ')
while user_input not in question_list:
print('Incorrect input, available options: greencircle, redcircle, bluesquare, redtriangle')
user_input = input('Where should I click?')
location = pyautogui.locateOnScreen(user_input + '.png')
pyautogui.click(location)

上面的示例要求您在目录中已经有greencircle.png和所有其他.png

PyAutoGUI还可以进行屏幕截图，您可以指定要拍摄屏幕的哪个区域pyautogui.screenshot(region=(0, 0, 0, 0))。前两个值是要选择的区域左上角的x、y坐标，第三个值是向右(x(多远，第四个值是向下(y(多远。

以下示例截取Windows 10徽标的屏幕截图，将其保存到文件中，然后使用指定的.png文件单击徽标

import pyautogui
pyautogui.screenshot('win10_logo.png', region=(0, 1041, 50, 39))
location = pyautogui.locateOnScreen('win10_logo.png')
pyautogui.click(location)

您也不必将屏幕截图保存到文件中，只需将其保存为变量即可

import pyautogui
win10 = pyautogui.screenshot(region=(0, 1041, 50, 39))
location = pyautogui.locateOnScreen(win10)
pyautogui.click(location)

让一个程序检测用户是否点击了某个区域(比如说windows 10的标志(需要另一个像pynput这样的库。

from pynput.mouse import Listener    
def on_click(x, y, button, pressed):
if 0 < x < 50 and 1080 > y > 1041 and str(button) == 'Button.left' and pressed:
print('You clicked on Windows 10 Logo')
return False    # get rid of return statement if you want a continuous loop
with Listener(on_click=on_click) as listener:
listener.join()

把它放在一起

import pyautogui
from pynput.mouse import Listener
win10 = pyautogui.screenshot(region=(0, 1041, 50, 39))
location = pyautogui.locateOnScreen(win10)
# location[0] is the top left x coord
# location[1] is the top left y coord
# location[2] is the distance from left x coord to right x coord
# location[3] is the distance from top y coord to bottom y coord
x_boundary_left = location[0]
y_boundary_top = location[1]
x_boundary_right = location[0] + location[2]
y_boundary_bottom = location[1] + location[3]

def on_click(x, y, button, pressed):
if x_boundary_left < x < x_boundary_right and y_boundary_bottom > y > y_boundary_top and str(button) == 'Button.left' and pressed:
print('You clicked on Windows 10 Logo')
return False    # get rid of return statement if you want a continuous loop

with Listener(on_click=on_click) as listener:
listener.join()