This project may seem abandoned but I'm actually still working on it. I just am not really making any progress.
I got real time screen capture and input sending working pretty quickly but then it turned out that is not the hard part. I was planning to do a "pure" AI that would watch the screen to infer the gamestate but it's turning out that this is a very hard thing to do. I went down a computer vision rabbit hole for a while but there isn't really any good way to do this and I'd much rather spend my time writing the bot, not becoming a machine learning expert.
I'm currently trying to find different approaches like inspecting the game memory to infer where things are.
At some point I think I'll do a bot for HMH (once there is more of a game to play). Doing this for HMH is going to be wonderful because of the clear separation between platform and game, It'll be easy to just inject some code there than can inspect the screen, make decisions and send the input for the next frame. Heck I can even write an AI only platform layer and run 1000's of games at once, use genetic algorithms to evolve AIs, log all the results to be analyzed later. It's going to be a lot of fun I think.
Thanks, I saw that in the forums. I ended up doing a small screen capture thing (on windows) a few months ago to clone part of the screen and magnify it in a window to be able to play an "old" point and click game that ran in a small window I couldn't move. I simply used GetDesktopWindow( ) to get a handle of the desktop window and GetDC to get it's device context, and used StretchBlt to copy it in my window magnified.
I actually restarted working on this and have ported it to windows. I'm using the dxgi 1.2 desktop duplication api and doing something similar to you where I only capture a small portion of the screen. It's quite nice, I'll probably add it to that screen capture lib at some point too.
I'll be starting a series of videos on the project where I go over progress every time I have something substantial. The first one will be about the screen capture stuff in a few weeks probably.