vdexExtractor

# Vdex Extractor Command line tool to decompile and extract Android Dex bytecode from Vdex files that are generated along with Oat files when optimizing bytecode from dex2oat ART runtime compiler. Vdex file format has been introduced in the Oreo (API-26) build. More information is available [here][vdex-cr]. It should be noted that Oat files are no longer storing the matching Dex files inside their `.rodata` section. Instead they're always paired with a matching Vdex file. ## Compile * Clone this repository * Install Android NDK if you want to cross-compile for Android devices * Invoke `make.sh` bash script with desired build target * `$ ./make.sh` - if CC not defined from env use gcc by default * `$ ./make.sh gcc` - compile with gcc * `$ ./make.sh clang` - compile with clang * `$ ./make.sh cross-android` - cross-compile (armeabi-v7a, arm64-v8a, x86 & x86_64) for Android with NDK * Executables are copied under the `bin` directory * For debug builds use `$ DEBUG=true ./make.sh` ## Usage ``` $ bin/vdexExtractor -h vdexExtractor ver. 0.4.1 Anestis Bechtsoudis <anestis@census-labs.com> Copyright 2017 - 2018 by CENSUS S.A. All Rights Reserved. -i, --input=<path> : input dir (search recursively) or single file -o, --output=<path> : output path (default is same as input) -f, --file-override : allow output file override if already exists (default: false) --no-unquicken : disable unquicken bytecode decompiler (don't de-odex) --deps : dump verified dependencies information --dis : enable bytecode disassembler --ignore-crc-error : decompiled Dex CRC errors are ignored (see issue #3) --new-crc=<path> : text file with extracted Apk or Dex file location checksum(s) -v, --debug=LEVEL : log level (0 - FATAL ... 4 - DEBUG), default: '3' (INFO) -l, --log-file=<path>: save disassembler and/or verified dependencies output to log file (default is STDOUT) -h, --help : this help ``` ## Bytecode Unquickening Decompiler The Vdex file includes all quick_info data (old vtable) required to revert the dex-to-dex transformations applied during bytecode optimization. The idea here is to create a quick standalone tool capable to revert optimized bytecode, that does not require building the entire libart from AOSP. The Vdex fully unquicken functionality has been also implemented as part of the AOSP oatdump libart tool. The upstream contribution is available [here][aosp-master]. If you want to use oatdump with Oreo release you can use the corresponding patch [here][oatdump-oreo] or fork and build (inside and AOSP_SRC_ROOT workspace) the oreo-release branch of the [oatdump++][oatdump-plus] tool. ## Verified Dependencies Iterator When the Dex bytecode files are compiled (optimized) for the first time, dex2oat executes the verification dependencies collector as part of the MethodVerifier The verification dependencies collector class is used to record resolution outcomes and type assignability tests of classes/methods/fields defined in the classpath. The compilation driver initializes the class and registers all Dex files which are being compiled. Classes defined in Dex files outside of this set (or synthesized classes without associated Dex files) are considered being in the classpath. All recorded dependencies are stored in the generated Vdex file along with the corresponding Oat file from the OatWriter class. vdexExtractor tool integrates a Vdex dependencies walker function that is capable to iterate all dependencies information and dump them in a human readable format. The following snippet demonstrates a dependencies dump example of a sample Vdex file. ``` $ bin/vdexExtractor -i /tmp/BasicDreams.vdex -o /tmp --deps -f [INFO] Processing 1 file(s) from /tmp/BasicDreams.vdex ------- Vdex Deps Info ------- dex file #0 extra strings: number_of_strings=2 0000: 'Ljava/lang/Thread;' 0001: 'Ljava/lang/Throwable;' assignable type sets: number_of_sets=8 0000: 'Landroid/service/dreams/DreamService;' must be assignable to 'Landroid/content/Context;' 0001: 'Landroid/view/TextureView;' must be assignable to 'Landroid/view/View;' 0002: 'Ljava/nio/FloatBuffer;' must be assignable to 'Ljava/nio/Buffer;' ... unassignable type sets: number_of_sets=0 class dependencies: number_of_classes=34 0000: 'Landroid/graphics/Color;' 'must' be resolved with access flags '1' 0001: 'Landroid/graphics/SurfaceTexture;' 'must' be resolved with access flags '1' ... 0024: 'Ljavax/microedition/khronos/egl/EGL10;' 'must' be resolved with access flags '513' ... field dependencies: number_of_fields=4 0000: 'Ljavax/microedition/khronos/egl/EGL10;'->'EGL_DEFAULT_DISPLAY':'Ljava/lang/Object;' is expected to be in class 'Ljavax/microedition/khronos/egl/EGL10;' and have the access flags '9' 0001: 'Ljavax/microedition/khronos/egl/EGL10;'->'EGL_NO_CONTEXT':'Ljavax/microedition/khronos/egl/EGLContext;' is expected to be in class 'Ljavax/microedition/khronos/egl/EGL10;' and have the access flags '9' 0002: 'Ljavax/microedition/khronos/egl/EGL10;'->'EGL_NO_DISPLAY':'Ljavax/microedition/khronos/egl/EGLDisplay;' is expected to be in class 'Ljavax/microedition/khronos/egl/EGL10;' and have the access flags '9' 0003: 'Ljavax/microedition/khronos/egl/EGL10;'->'EGL_NO_SURFACE':'Ljavax/microedition/khronos/egl/EGLSurface;' is expected to be in class 'Ljavax/microedition/khronos/egl/EGL10;' and have the access flags '9' method dependencies: number_of_methods=84 0000: 'Landroid/graphics/Color;'->'HSVToColor':'([F)I' is expected to be in class 'Landroid/graphics/Color;', have the access flags '9 0001: 'Landroid/opengl/GLES20;'->'glAttachShader':'(II)V' is expected to be in class 'Landroid/opengl/GLES20;', have the access flags '9 0002: 'Landroid/opengl/GLES20;'->'glClear':'(I)V' is expected to be in class 'Landroid/opengl/GLES20;', have the access flags '9 ... unverified classes: number_of_classes=0 ----- EOF Vdex Deps Info ----- [INFO] 1 out of 1 Vdex files have been processed [INFO] 1 Dex files have been extracted in total [INFO] Extracted Dex files are available in '/tmp' ``` ## Integrated Disassembler To debug the decompiler and assist the Dex bytecode investigation tasks, a light disassembler has been implemented. The disassembler output is very similar to the one provided by the AOSP dexdump2 utility of the `platform/art` project. The disassembler can be used independently of the unquickening decompiler. A sample output is illustrated in the following snippet. Lines prefixed with `[new]` illustrate the output of the decompiled instruction (previous line) located in that offset. Notice that all the quickened offsets and vtable references have been reverted back to original signatures and prototypes. ``` $ bin/vdexExtractor -i /tmp/Videos.vdex -o /tmp -f --dis -l /tmp/dis.log [INFO] Processing 1 file(s) from /tmp/Videos.vdex [INFO] 1 out of 1 Vdex files have been processed [INFO] 2 Dex files have been extracted in total [INFO] Extracted Dex files are available in '/tmp' $ head -90 /tmp/dis.log ------ Dex Header Info ------ magic : dex-035 checksum : e14de163 (3779977571) signature : 9a91f8e5f2afe2c6b5c2b4853832d3c5ed01aef8 fileSize : 8ca638 (9217592) headerSize : 70 (112) endianTag : 12345678 (305419896) linkSize : 0 (0) linkOff : 0 (0) mapOff : 8ca568 (9217384) stringIdsSize: ef06 (61190) stringIdsOff : 70 (112) typeIdsSize : 29f4 (10740) typeIdsOff : 3bc88 (244872) protoIdsSize : 3df9 (15865) protoIdsOff : 46458 (287832) fieldIdsSize : a79d (42909) fieldIdsOff : 74c04 (478212) methodIdsSize: fed7 (65239) methodIdsOff : c88ec (821484) classDefsSize: 2288 (8840) classDefsOff : 147fa4 (1343396) dataSize : 73d594 (7591316) dataOff : 18d0a4 (1626276) ----------------------------- file #0: classDefsSize=8840 class #0: a.a ('La$a;') access=0601 (PUBLIC INTERFACE ABSTRACT) source_file=SourceFile, class_data_off=851907 (8722695) sta