如何在Python中记录xbox/gamepad控制器的状态? [英] How to record state of xbox/gamepad controller in Python?

问题描述

我需要在特定时间知道xbox控制器的所有按钮的值.原因是我正在为神经网络构建训练集，并且试图同时拍摄屏幕快照和拍摄控制器状态的快照".请注意，对于该项目的键盘版本，我能够成功完成此操作，但是xbox控制器给我带来了麻烦.

I need to know at a specific time the value of all buttons of an xbox controller. The reason being that I'm building a training set for a neural network, and I'm trying to simultaneously take a snapshot of the screen and take a "snapshot" of the controller state. Note that I was able to successfully do this for a keyboard version of this project, but the xbox controller is giving me difficulty.

我尝试过的工作是创建按钮和值的字典，并在每次接收到来自控制器的事件时更新字典.然后，我将图像和字典另存为训练数据的实例.但是，输入最终根本不与图像同步.我认为该问题可能与用于读取控制器的软件包之一中的线程或子进程有关，但是我不够熟练，无法知道如何解决它.

What I've tried is creating a dictionary of buttons and values, and updating the dictionary every time I receive an event from the controller. Then I would save the image and dictionary as an instance of training data. However, the inputs end up not at all synced with the images. I'm thinking that the issue might be related to threading or subprocesses in one of the packages used to read the controller, but I'm not skilled enough to know how to fix it.

下面是我的代码.

from inputs import get_gamepad
import time
import cv2
import numpy as np
from mss.windows import MSS as mss

#Only track relevant inputs
gp_state = {#'ABS_HAT0X' : 0, #-1 to 1
             #'ABS_HAT0Y' : 0, #-1 to 1
             #'ABS_RX' : 0, #-32768 to 32767
             #'ABS_RY' : 0, #-32768 to 32767
             'ABS_RZ' : 0, #0 to 255
             'ABS_X' : 0, #-32768 to 32767
             'ABS_Y' : 0, #-32768 to 32767
             #'ABS_Z' : 0, #0 to 255
             'BTN_EAST' : 0,
             'BTN_NORTH' : 0,
             #'BTN_SELECT' : 0,
             'BTN_SOUTH' : 0,
             #'BTN_START' : 0,
             #'BTN_THUMBL' : 0,
             #'BTN_THUMBR' : 0,
             'BTN_TL' : 0,
             'BTN_TR' : 0,
             'BTN_WEST' : 0,
             #'SYN_REPORT' : 0,
             }

dead_zone = 7500

def screen_record(): 
    last_time = time.time()
    while(True):
        # 800x600 windowed mode
        printscreen =  np.array(ImageGrab.grab(bbox=(0,40,800,640)))
        last_time = time.time()
        cv2.imshow('window',cv2.cvtColor(printscreen, cv2.COLOR_BGR2RGB))
        if cv2.waitKey(25) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            break

def process_img(image):
    original_image = image
    processed_img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    contrast = 1
    brightness = 0
    out = cv2.addWeighted(processed_img, contrast, processed_img, 0, brightness)
    return out

def main():

    #Give myself time to switch windows
    #Screen should be in top left
    for _ in range(4):
        time.sleep(1)

    controller_input = np.zeros(5)
    training_data = []
    training_files = 0

    with mss() as sct:
        while True:
            #Get screen and display
            bbox = (150,240,650,490)
            screen =  np.array(sct.grab(bbox))
            new_screen = process_img(screen)
            cv2.imshow('window', new_screen)
            new_screen = cv2.resize(new_screen, (100,50))

            #Map events to dictionary
            events = get_gamepad()
            for event in events:
                gp_state[event.code] = event.state

            #Set to zero if in dead zone
            if abs(gp_state['ABS_X']) < dead_zone:
                gp_state['ABS_X'] = 0 

            if abs(gp_state['ABS_Y']) < dead_zone:
                gp_state['ABS_Y'] = 0 

            #Set values to be between 0 and 1.
            controller_input[0] = (gp_state['ABS_X'] + 32768) / (32767 + 32768)
            controller_input[1] = gp_state['ABS_RZ'] / 255
            controller_input[2] = gp_state['BTN_SOUTH']
            controller_input[3] = gp_state['BTN_EAST']
            controller_input[4] = gp_state['BTN_TR']


            record = gp_state['BTN_NORTH'] #Record while holding y button
            if record:
                training_data.append(np.array([new_screen, controller_input]))
                print(controller_input)
                time.sleep(1)

            if len(training_data) % 500 == 0 and record:
                filename = f"training_data/rlb_XBOXtrain_{time.time()}.npy"
                np.save(filename, training_data)
                training_files += 1
                print(f"Trained {training_files} files!")
                training_data = []


            if cv2.waitKey(25) & 0xFF == ord('q'):
                cv2.destroyAllWindows()
                break

main()

我觉得我正在使这种方式变得比所需的更加困难.但是，有没有更简单的方法来仅在特定时间点获得控制器的状态?

I feel like I am making this way harder than it needs to be. But is there an easier way to just get the state of the controller at a certain point in time?

请注意，我找到了一些适用于Linux的解决方案，但是我正在Windows 10中运行.这是Linux解决方案的示例: https://github.com/FRC4564/Xbox

Note that I've found some solutions that work for Linux, but I am running in Windows 10. Here is an example of a Linux solution: https://github.com/FRC4564/Xbox

推荐答案

TensorKart项目已经解决了该问题:

the TensorKart Project has already solved that problem: https://github.com/kevinhughes27/TensorKart/blob/master/utils.py

这篇关于如何在Python中记录xbox/gamepad控制器的状态?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！