Go Back   Exetools > General > General Discussion


Thread Tools Display Modes
Old 01-14-2019, 14:01
chants chants is offline
Join Date: Jul 2016
Posts: 439
Rept. Given: 1
Rept. Rcvd 30 Times in 18 Posts
Thanks Given: 352
Thanks Rcvd at 695 Times in 321 Posts
chants Reputation: 30
Natural mouse movements and defeating bot detection systems like captchas

Does anyone know how we could build a huge database of "natural" mouse movements in a given context.

For example robot versus human detection is done almost exclusively now based on how the movement of the mouse goes. But we all know using tons of Sleep and mouse arc shapes pixel gap by pixel gap and such could theoretically look exactly as per human behavior. But ideally one would have a very large database of natural mouse movement. Of course every context - e.g. click a button, drag and drop, scroll, etc has a different set of signature movements.

Even online games now are able to detect cheating 99.99% of the time using mouse movements or perhaps touch screen for a mobile device. But it seems like the big obvious research area is realistic mouse movement faking which will give a big headache in differentiating again. After that all that is left is checking if a person is too perfect or too excellent even with image identification, written questions, games, as neural network training algorithms are showing now as humans tend to err at certain tasks given enough repetitions but even that can be easily heuristically faked.

Any ideas or thoughts or experience on this?
Reply With Quote
The Following User Says Thank You to chants For This Useful Post:
niculaita (01-18-2019)
Old 01-17-2019, 16:26
atom0s's Avatar
atom0s atom0s is online now
Join Date: Jan 2015
Posts: 240
Rept. Given: 24
Rept. Rcvd 101 Times in 47 Posts
Thanks Given: 42
Thanks Rcvd at 415 Times in 163 Posts
atom0s Reputation: 100-199 atom0s Reputation: 100-199
In terms of games detecting things, it's a matter of emulating the movement properly and not injecting the data in a manner that allows the system to tell it was not done with the actual hardware. Windows has a flag to determine if a keypress/mouse movement was injected using one of the various input API's such as SendInput.

When it comes to games, you would generally hook the method of input (often times DirectInput or standard window messages ie. WM_MOUSEMOVE) and simulate the data yourself in the flow of the game polling for the data. It's easy to do on games. With applications it can be done in a similar manner though depending on how the input is read/handled.
Personal Projects Site: https://atom0s.com
Reply With Quote
The Following 2 Users Say Thank You to atom0s For This Useful Post:
chants (01-18-2019), niculaita (01-18-2019)
Old 01-18-2019, 00:27
chants chants is offline
Join Date: Jul 2016
Posts: 439
Rept. Given: 1
Rept. Rcvd 30 Times in 18 Posts
Thanks Given: 352
Thanks Rcvd at 695 Times in 321 Posts
chants Reputation: 30
What is this flag specifically? But this would be for desktop or DX apps I imagine not browser or store apps.

Also what about in the web browser window - I imagine that javascript and such cannot detect directly. Probably SendInput is okay here. I think in sophisticated situations, they are recording detailed timing and mouse position information, and then matching it against a classification algorithm which is trained to determine robots verse humans - totally empirically.

And modifying DOM in browser windows could definitely be detected by introspection or reflection techniques. Reading the DOM might even be detectable too though I have not much knowledge about that - if it could do something like constantly modifying the DOM and measure time too see if its too slow from monitoring, or if by some other tricks.

I was looking into chess websites which are detecting cheating now by monitoring and recording the mouse. I think with timing and mouse data, 99.9% can be detected. But of course, nobody even realizes this as they have kept it very secretive as its the last remains of their business to defend against this. But we are in a new computing era. All you have to do is visually grab the board, line detection (9x9 grid), piece detection using shape recognition, or shape areas, etc, single move detection heuristics, then send it off to an analysis engine. Further to not always make engine moves, make good moves or blunders or inaccuracies when its even or when winning from time to time deliberately. Use a time per move algorithm with randomization and logarithmic decay as it goes towards running out of time when in a complex position to avoid predictability but in trivial positions instant moves. At this point, its impossible to prove anything if you can move the mouse in human-like ways - or just draw an analysis window in a desktop app and hand make the moves. Unfortunately these sites are quick to ban an account and IP address - after running invasive monitor tricks with 30% CPU . It is a pretty big business. But given that grandmasters already have secret apps doing exactly the sophisticated set of techniques I describe and are certainly cheating, I think its time for some indistinguishable robot/human type games to have a strong solution open sourced.
Reply With Quote

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Arrest the mouse into the Form(VB6) wilson bibe Source Code 0 02-10-2015 19:17
Problems with mouse mlb2gm5x General Discussion 3 03-13-2004 07:19
SoftIce kills Mouse aldente General Discussion 11 09-03-2003 16:13
No mouse in TRW 2000 BoostMan General Discussion 1 07-31-2002 16:55

All times are GMT +8. The time now is 14:11.

Always Your Best Friend: Aaron, JMI, ahmadmansoor, ZeNiX