Exetools - FlexNet / FlexLM with Java

Exetools (https://forum.exetools.com/index.php)

- General Discussion (https://forum.exetools.com/forumdisplay.php?f=2)

- - FlexNet / FlexLM with Java (https://forum.exetools.com/showthread.php?t=18864)

FlexNet / FlexLM with Java

Hi,

I'm trying to understand how FlexNet works and of course find out the vendor keys.
Note: Application is based on short sign (-> no ECC!) and accepts HostID=ANY

Well, I've searched for the FlexNet routines inside the native executables but didn't found any.
The FlexNet routines seem to be implemented in Java.
I've found a class file (c.class) in one of the JARs, which contains the following lines:

Code:

  

private static final String a = "LMCOMSOL.opt";

private static final String b = "VENDOR LMCOMSOL";

private static final String c = "USER_BASED";

This looks pretty much like FlexNet stuff.

In this class file, there are two methods JD-GUI cannot decompile (it shows the byte code).
Do you think I should try to reverse engineer these two methods? I guess it's likely, that it contains the logic.

Also, has anyone ever dealt with Java based FlexNet? Are there any tricks to find out the vendor keys?

Cheers!

//EDIT:
Tried different decompiler for Java (CFR)
Error for method:

Code:

This method has failed to decompile.  [...]

org.benf.cfr.reader.util.ConfusedCFRException: Tried to end blocks [1[TRYBLOCK]], but top level block is 13[DOLOOP]

But it seems like there is only one method for which CFR fails.

//EDIT 2:
Oops.
CFR is able to decompile the methods JD cannot ("b"), however there is one method ("c") that JD can decompile but CFR not.

There is no ideal java decompiler
usually I"m using these:

procyon
krakatau
JDGUI
CFR
jadx
DJ Java
androchef
....

Regards
>>Also, has anyone ever dealt with Java based FlexNet?
java is a wrapper around FN (my assumption)

Yes you are right, the application uses JNI to access native functions...

Like

Code:

private native String getLicenseNumber(final CPointer p0, final String p1) throws FlNativeException;

However I cannot find the actual DLL this is referring to - How can I find out?

//EDIT:
Attached "DLL Export Viewer" and searched for the Java stubs, managed to find the x64 dll. I will now search in the setups files for a 32bit version, this would make the RE a little easier, doesn't it?

//EDIT 2:
Well, IDA doesn't seem to like the lib.

Code:

.text:0000000180006370     ; __unwind { // __GSHandlerCheck_EH

.text:0000000180006370 000                 sub     rsp, 108h

.text:0000000180006377 108                 mov     [rsp+108h+var_90], 0FFFFFFFFFFFFFFFEh

.text:0000000180006380 108                 mov     rax, cs:qword_180172E78

.text:0000000180006387 108                 xor     rax, rsp

.text:000000018000638A 108                 mov     [rsp+108h+var_10], rax

.text:0000000180006392 108                 mov     [rsp+108h+var_D8], rcx

.text:0000000180006397

.text:0000000180006397     loc_180006397:                          ; DATA XREF: .rdata:0000000180112E1C↓o

.text:0000000180006397     ;   try {                               ; struct JNIEnv_ *

.text:0000000180006397 108                 mov     rdx, r8

.text:000000018000639A 108                 call    ?getWsPtr@flbase@@YAPEAXPEAUJNIEnv_@@PEAV_jobject@@@Z ; flbase::getWsPtr(JNIEnv_ *,_jobject *)

.text:000000018000639F 108                 mov     rcx, rax        ; this

.text:00000001800063A2 108                 call    ?blockInternal@License@@QEAAXXZ ; License::blockInternal(void)

.text:00000001800063A7 108                 nop

.text:00000001800063A8

.text:00000001800063A8     loc_1800063A8:                          ; CODE XREF: sub_1800C4F8D+26↓j

.text:00000001800063A8                                             ; sub_1800C4FCD+122↓j ...

.text:00000001800063A8 108                 mov     rcx, [rsp+108h+var_10]

.text:00000001800063B0 108                 xor     rcx, rsp        ; StackCookie

.text:00000001800063B3 108                 call    __security_check_cookie

.text:00000001800063B8 108                 add     rsp, 108h

.text:00000001800063BF 000                 retn

.text:00000001800063BF     ; ---------------------------------------------------------------------------

.text:00000001800063C0 000                 db 0CCh

.text:00000001800063C0     ;   } // starts at 180006397

.text:00000001800063C0     ; } // starts at 180006370

.text:00000001800063C0     Java_com_comsol_nativejni_util_FlLicense_blockInternal endp ; sp-analysis failed

I don't unterstand, why the sp-analysis fails.
When the retn instruction is executed, the Stack Pointer is 0 again, so no offset...
But why does IDA state that the sp-analysis fails?

usually the name of native dll is in the same java module (.class)
AFAIK java does not use Import table records for this purpose

Yeah, I found the entry in the class file.

I now need help with the reverse engineering stuff.

I still don't understand why IDA complains about the stack pointer...

IDA is not a 100% predictor in terms of API type analysis and it mihgt be wrong/fooled/etc
you have to help it to correct the sp pointer
as far as I know a "K" letter is for sp corection

2) give it a try to use demangled names: menu - Options - Demangled names

I recommend also to use a good plugin for type reconstruction named ClassInformer (not sure if it exists for v7.x, but definitely good stuff used in v6.x)

good luck

That's already turned on.

The problem is:
IDA shows the current Stackptr on the left side.
It starts at 0 (see my code) and ends at 0 (see my code, at the return statement)...

So where is IDAs problem with the Stackptr?

>> don't unterstand, why the sp-analysis fails.
Usually IDA says on what address it fails with SP register
do you have the full message?

Java3ever - is there maybe also a vendor daemon laying around? I tend to recover the seeds from this one, and dig more into the java later one (if its needed, such as ECC).