x64 and anti-debugging

[lena151]

In reversing, anti-debugging tricks have always been a highly interesting matter. Since the migration towards x64 hardware and OS'es, some things have changed though.

The other day, I came across a x64 software which was always fake detecting debugging on a certain test system. Diving into the matter and circumventing all anti-debugging tricks under debugger, it worked fine. The reason of faillure outside debugger proved to be the well-known rep stos/movs trick.

Example code

	t1 equ goodboy-badboy-2
	new: db 0EBh,01,81h,0EBh,t1

   ; cut //

	lea rsi,[new]
	lea rdi,[here]
	mov rdx,[rdi]
	mov rcx,3
  here:
	rep movsw
  badboy:
	mov r9,30h
	lea r8,[DebugStatus]
	lea rdx,[DbgFoundText]
	xor rcx,rcx
	invoke MessageBox			; badboy!
	jmp Exit
  goodboy:
	mov r9,40h
	lea r8,[DebugStatus]
	lea rdx,[DbgNotFoundText]
	xor rcx,rcx
	invoke MessageBox			; goodboy!

   ; cut //

The rep stos/movs trick does not need further explaining since everybody knows this one since 16 bit. However, be warned not to use it anymore on x64. For testing, I attached an exe. Single step it F7 (F8 on the messagebox call) and it will always detect you, however I'm sure that a small percentage -having the newest x64 CPU technology- will get fake detected outside debugger!

Carpe Diem,

lena151.