Natural mouse movements and defeating bot detection systems like captchas
 

Does anyone know how we could build a huge database of "natural" mouse movements in a given context.

For example robot versus human detection is done almost exclusively now based on how the movement of the mouse goes. But we all know using tons of Sleep and mouse arc shapes pixel gap by pixel gap and such could theoretically look exactly as per human behavior. But ideally one would have a very large database of natural mouse movement. Of course every context - e.g. click a button, drag and drop, scroll, etc has a different set of signature movements.

Even online games now are able to detect cheating 99.99% of the time using mouse movements or perhaps touch screen for a mobile device. But it seems like the big obvious research area is realistic mouse movement faking which will give a big headache in differentiating again. After that all that is left is checking if a person is too perfect or too excellent even with image identification, written questions, games, as neural network training algorithms are showing now as humans tend to err at certain tasks given enough repetitions but even that can be easily heuristically faked.

Any ideas or thoughts or experience on this?

In terms of games detecting things, it's a matter of emulating the movement properly and not injecting the data in a manner that allows the system to tell it was not done with the actual hardware. Windows has a flag to determine if a keypress/mouse movement was injected using one of the various input API's such as SendInput.

When it comes to games, you would generally hook the method of input (often times DirectInput or standard window messages ie. WM_MOUSEMOVE) and simulate the data yourself in the flow of the game polling for the data. It's easy to do on games. With applications it can be done in a similar manner though depending on how the input is read/handled.

What is this flag specifically? But this would be for desktop or DX apps I imagine not browser or store apps.

Also what about in the web browser window - I imagine that javascript and such cannot detect directly. Probably SendInput is okay here. I think in sophisticated situations, they are recording detailed timing and mouse position information, and then matching it against a classification algorithm which is trained to determine robots verse humans - totally empirically.

And modifying DOM in browser windows could definitely be detected by introspection or reflection techniques. Reading the DOM might even be detectable too though I have not much knowledge about that - if it could do something like constantly modifying the DOM and measure time too see if its too slow from monitoring, or if by some other tricks.

I was looking into chess websites which are detecting cheating now by monitoring and recording the mouse. I think with timing and mouse data, 99.9% can be detected. But of course, nobody even realizes this as they have kept it very secretive as its the last remains of their business to defend against this. But we are in a new computing era. All you have to do is visually grab the board, line detection (9x9 grid), piece detection using shape recognition, or shape areas, etc, single move detection heuristics, then send it off to an analysis engine. Further to not always make engine moves, make good moves or blunders or inaccuracies when its even or when winning from time to time deliberately. Use a time per move algorithm with randomization and logarithmic decay as it goes towards running out of time when in a complex position to avoid predictability but in trivial positions instant moves. At this point, its impossible to prove anything if you can move the mouse in human-like ways - or just draw an analysis window in a desktop app and hand make the moves. Unfortunately these sites are quick to ban an account and IP address - after running invasive monitor tricks with 30% CPU ;). It is a pretty big business. But given that grandmasters already have secret apps doing exactly the sophisticated set of techniques I describe and are certainly cheating, I think its time for some indistinguishable robot/human type games to have a strong solution open sourced.