After not getting really good result with the synthetic data I decided to collect data directly from the Pi camera mounted on the jetson Nano . For this I setup the Jetson Nano as a server and the PC running the game as a client . Once the game is started it connect to the server, the server then start taking picture and ask the game to change frame after each picture taken .

The other task I had to tackle was to control the arm actuating the gamepad joystick . I assumed I could just control the servos directly from the GPIO's of the jetson Nano . Unfortunately it was not the case . I decided that reusing the arduino from the previous project would be the easiest way to deal with this issue . The jetson Nano send some data to the arduino that move the servos .