A weird debugging question

I'm 'enhancing a program without the authors consent'. I'm not cracking it, I don't even like it. But there are people who do, and they really want it to talk to a piece of hardware that the Author doesn't want it to talk to (because they have competing hardware).

I got it working in less than a day, but there's a bug. I need to dismiss a connection to the actual hardware driver, and the only way I know I'm done with it is in DLL_PROCESS_DETATCH. I can't dismiss it there, because the unload code needs to do things that can't be done at that time.

Without the bug being fixed, the user has to end-task the executable (which, btw, has a .MOD extension). Very few of the 90+ DLLs have sane extensions. Some have almost pseudo-random extensions. No match for PEiD.

None of the modules are packed, none are convoluted other than what you'd see with a severely over-bloated piece-of-dung software package.

Anyway, I used VTune to find the module I was interested in. It happily pointed to the right spot, bouncing up and down like a loyal puppy-dog. I wrote my replacement DLL in a matter of hours, despite some of the arguments being pointers to structures 100's of bytes long. IDA is wicked stuff.

But I have no way to know before hand that the application is going to terminate. If this was for MY use, I'd just patch the MOD exe to call a new routine in my DLL saying "Hey, were done now". But patching is (nearly) out of the question. (there are a number of builds, I don't want to write a number of patches).

My problem: The 'MOD' program is called by an 'EXE', which goes away. When I try to attach Olly, it kills a thread and becomes catonic. I really need to debug to determine if there is some circumstance I can use to detect the program is about to unload BEFORE I get the DLL_PROCESS_DETATCH.

Any ideas?

Is this for a dongle?

LOL!

I guess, in a weird sort of way.

Company 'P' makes a $5000 video editing board, as well as some bloated software that only wants to talk to their board. (and, believe it or not, they charge extra for the software!).

Company 'N' is really cool and makes a better card for $3000. There are some people that want to run 'P's bloated software, but on Company 'N's board.

Company 'P' is stingy and doesn't want 3rd parties developing stuff for them, let alone producing things that could allow users to use their bloated software (that they PAID FOR!!!) on boards that Company 'P' didn't make. So, they don't provide SDKs. IDA's enough for me, thank you!!

Company 'N' likes it when additional software suddenly supports their board, and happily provide SDKs to 3rd parties, even people like ME.

So whenever I get a chance to jab Company 'P' in the ribs, I do so, and I always release the results for free. It's a hobby, you see. Death to Compnay 'P'!!!

I think I figured out how to do it. By creating a process and using a memory mapped file and a pair of named events, I should be able bypass DLL_PROCESS_DETATCH's limitations. Have all communications with the hardware take place out-of-process. It involves an extra memcpy, but I think I'll be OK.

Thank you for reading!

I read your post but i didn't get the problem, may be the post lacks the details, or may be i lack the organized mind to get it , But what is VTune?? is it Intel VTune?? and How could you use VTune to find that module??

Why can't u use OllyDbg or even SoftICE??

Thanks

Olly seems to be freezing Threads on my PC too...I guess that's why it become "catatonic".

Best regards,
Alex Ionescu
Relsoft Technologies
http://www.relsoft.net