How to link a dynamic .so library to create a static executable?

[CZC]

I'm trying to reverse the license mechanism of a Rigol oscilloscope (DHO800 series) based on a cortex-a72 CPU.
It is Android based and has its main application an .apk file.
Inside this there is a huge ARM64 .so library that contains all the relevant code.
I've first disassembled it in IDA Pro and produced a 1.4GB asm file.
All symbols are present in the file so that is very helpful.

My first approach was to write a Perl script and extract only the functions of interest while also replacing some asm instructions so it compiles with GAS.
I've made some test programs calling asm functions (which to make life harder were written in C++ and I had to de/mangle their names).
I've cross-compiled it for aarch64 and then run the static executable with qemu-aarch64. This worked fine and I was able to extract some knowledge.
Unfortunately the calling tree is huge and I couldn't keep up with all the unresolved references.

My second approach was to create a ARM64 disk image with a full debian system which can run in emulation with qemu-system-aarch64.
I wanted to use the original .so file and just create a dynamically linked executable which I can then run in the emulator.
This didn't work either because a lot of the dynamic symbols were versioned and I had to use all the original scope libraries.
The main .so requires half a dozen of other .so which then require almost 50 other .so libraries. Linking failed with some cryptic "index" mismatches.

I've then tried to use patchelf which lets you add/remove needed libraries, clear-symbol-version and rename-dynamic-symbols of my library.
I've then created my own .so lib with all the unresolved functions (as hooks which then call relevant libc functions). This almost worked:
I was able to compile my test program without any errors. But when I try to run it on qemu or when I ldd the library I get:

$ ldd libscope-auklet.so
Inconsistency detected by ld.so: dl-version.c: 204: _dl_check_map_versions: Assertion `needed != NULL' failed!

I think using the dynamic linker is a lost cause so what I want to do is to
get rid of all the dynamic symbols in my library and convert them to static
(function calls). Using objdump/objcopy I could place all the missing functions as an extension in the .text section.
Can someone suggest a way to do this with as little manual work as possible?

[chants]

So for ELF shared objects (SO), these are now just a simple object, while static libraries are a collection of objects. So information has been lost when objects are combined.

Going from a static library (.a) -> shared object (.so) is of course trivial with gcc or ar (in absence of the -fPIC option).

I think objdump -d to disassemble and then recompile is really the only reasonable route.

[CZC]

I've made some progress and I am now able to link and run my test program under qemu.
I did the following to the original .so lib:

• patchelf --remove-needed all.so
• patchelf --add-needed libx.so
• Then with my own patcher I've removed all versions in the .gnu.version_r section and inserted my own "HOOK" version
• I've set all external functions to "HOOK" version in section .gnu.version
• I've compiled my own shared library libx.so will all external functions
  and I've versioned these functions using the linker option --version-script
• I've dumped libx.so with readelf and got the hash value which I then reinserted in the .gnu.version_r of the original lib

Functions in my libx.so then call the equivalent ones from glibc

int
puts( const char *s )
{
  static int    ( *fn )( const char *s ) = NULL;
  if( fn == NULL )
    fn = dlsym( RTLD_NEXT, "puts" );
  return fn( s );
}

I've also knocked out the .init_array and .fini_array sections just to make sure no code is executed at library loading time.

Now my program links and runs but as soon as I reference a function from the binary shared library (or if I manually use dlopen on it) the execution hangs:

Reading symbols from ./test...
(No debugging symbols found in ./test)
(gdb) run
Starting program: /home/zzz/test
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
Start main
^C
Program received signal SIGINT, Interrupt.
0x0000fffff4bfc434 in std::set_unexpected(void (*)()) ()
   from ./libscope-auklet.so
(gdb) quit

I presume it is the dynamic linker which fails to resolve the .plt/.got relocations and ends up in one of the library's function (std::set_unexpected)

I don't see a solution to this last problem so now I will investigate a way to convert that .so to a (big) .o.
This means getting rid of the .dynsym section and replacing all stubs in the .plt section - not an easy task.

[CZC]

Originally Posted by CZC

This means getting rid of the .dynsym section and replacing all stubs in the .plt section - not an easy task.

This turned out to be way more complicated that I've anticipated so I've dropped it.

In the end I've found an easier alternative:

I've installed the Android NDK (not SDK) on my linux computer and configured it for arm64.
That enabled me to compile my own ELF executable linked to the above library and the Android libs.
Since I had root access on the scope I've used the scope as my testing platform: I've transferred to the scope a statically linked gdb (arm64) and
then I've traced my testing programs until I found all the info I needed to create the scope options.
So now the scope is enhanced with all the options.

[dion]

no intention to hijack the thread, but i am on the same goal as CZC, to understand codes in android .so native library (arm64).

i have tried to debug using jeb (the apk itself) on rooted phone, turned out one has to set debuggable flag. I did (after repack and resign the apk), but then the app crashed. so i try another way.

the native has somekind of java wrapper, it load them using system.loadlibrary. so, i look into android studio sample, which incorporate external native library and calls them. then i add the target native library and modify gradle, cmakelists.txt, the source code, and i don't remember whatelse, to accomodate the new native library.

so, i press debug in android studio, and the app crashed, i know by looking at the logcat window. the cause was something like this :

JNI NewGlobalRef called with pending exception java.lang.ClassNotFoundException: Didn't find class "com.example.hello" on path: DexPathList [[zip file "/data/app/com.example.hello-JXyLr8y_WKw9Tt8GbtoaIw==/base.apk"],nativeLibraryDirectories=[/data/app/com.example.hello-JXyLr8y_WKw9Tt8GbtoaIw==/lib/arm64, /data/app/com.example.hello-JXyLr8y_WKw9Tt8GbtoaIw==/base.apk!/lib/arm64-v8a, /system/lib64]]'

i looked, it turned out it is on JNI_OnLoad() function inside one of the native library. ida 'interpret' the function like this :

jint JNI_OnLoad(JavaVM *vm, void *reserved)
{
  jint result; // w0
  __int64 v3; // x19
  __int64 v4; // x8
  __int64 v5[2]; // [xsp+0h] [xbp-30h] BYREF

  v5[1] = *(_QWORD *)(_ReadStatusReg(ARM64_SYSREG(3, 3, 13, 0, 2)) + 40);
  v5[0] = 0LL;
  if ( (*vm)->GetEnv(vm, (void **)v5, 65540LL)
    || (v3 = v5[0],
        (v4 = (*(__int64 (__fastcall **)(__int64, const char *))(*(_QWORD *)v5[0] + 48LL))(
                v5[0],
                "com/original/class/name")) == 0) )
  {
    result = -1;
  }
  else
  {
    result = ((*(int (__fastcall **)(__int64, __int64, char **, __int64))(*(_QWORD *)v3 + 1720LL))(
                v3,
                v4,
                off_70010,
                53LL) >> 31) | 0x10004;
  }
  return result;
}

since the expected class name was different (i masked the original class name here), i hex edited the class name inside the so file, and got the logcat like above when debug.

later, i know the function that return the error was findclass(). i was thinking to patch the function, but i am not sure about the function after else syntax, that looks like do something with the results.

have search google couple days, and applied what suggested, like using -keepclass at proguard.pro rule. but nothing works so far.

does anyone can help with the problem?
thanks in advance.

[chants]

Fortunately Java is very simple. There is a class path and that represents the root. Packages paths are just folder names. Finally the class name should be a .class file in thr appropriate folder. So if you are getting class not found exceptions, either adjust your class path or move the right things into it. For example if your files are in a .jar or Java archive which is common and just a glorified zip file with a metadata folder, you could unzip the class file(s) or merely add the jar to the class path.

But fixing this is far easier than .so/DLL hell described above. Its something new Java programming students are expected to learn in fact.

[Moe]

Originally Posted by dion

later, i know the function that return the error was findclass(). i was thinking to patch the function, but i am not sure about the function after else syntax, that looks like do something with the results.

have search google couple days, and applied what suggested, like using -keepclass at proguard.pro rule. but nothing works so far.

does anyone can help with the problem?
thanks in advance.

Is it possible to share the apk? Difficult to comment without seeing it, except wild speculations. I assume you did the proper zip align after editing...

[chants]

Originally Posted by Abdul Moeed

Is it possible to share the apk? Difficult to comment without seeing it, except wild speculations. I assume you did the proper zip align after editing...

TechLord, I don't think anyone wants to share private files with you. You are the least trustworthy person to ever use this forum. Not to mention, many problems are resolved by describing them without rushing to send a package file. A but fishy to so quickly request it, but we know your cheesy tactics.

[Moe]

Originally Posted by chants

TechLord, I don't think anyone wants to share private files with you. A but fishy to so quickly request it, but we know your cheesy tactics.

This chants guy seems to have gone mad. Or maybe obsessed.
Seems to be always dreaming about Techlord and addressing everyone as Techlord, lol. Seems that he does not have anything better to do.

Techlord is banned here? But in every other post I see his name invoked.
And @chants keeps chanting his name in every post. Almost as if Techlord is immortal.

[dion]

Originally Posted by chants

Fortunately Java is very simple. There is a class path and that represents the root. Packages paths are just folder names. Finally the class name should be a .class file in thr appropriate folder. So if you are getting class not found exceptions, either adjust your class path or move the right things into it. For example if your files are in a .jar or Java archive which is common and just a glorified zip file with a metadata folder, you could unzip the class file(s) or merely add the jar to the class path.

But fixing this is far easier than .so/DLL hell described above. Its something new Java programming students are expected to learn in fact.

i looked into apk file folders name, but nothing describe the class name(com.example.hello). i guess it is on classes.dex file.

i uploaded the android studio file here. i hope this is still ok, since nothings on it from original apk, but native library files.

btw, you would need arm64 root phone or emulator to debug this.