M. Saquib Sarfraz

89 papers A* 13A 8B 5C 2Misc 3Journal 44Unranked 14
YearRankTypeTitle / Venue / Authors
2026 J jnl
CoRR
Mingzhe Tao, Ruiping Liu, Junwei Zheng, Yufan Chen, Kedi Ying, M. Saquib Sarfraz, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen
2026 J jnl
Int. J. Comput. Vis.
Kunyu Peng, Di Wen, M. Saquib Sarfraz, Yufan Chen, Junwei Zheng, David Schneider, Kailun Yang, Jiamin Wu, Alina Roitberg, Rainer Stiefelhagen
2026 J jnl
Trans. Mach. Learn. Res.
Mei-Yen Chen, Thi Thu Uyen Hoang, Michael Hahn, M. Saquib Sarfraz
2025 J jnl
CoRR
Kunyu Peng, Di Wen, Kailun Yang, Jia Fu, Yufan Chen, Ruiping Liu, Jiamin Wu, Junwei Zheng, M. Saquib Sarfraz, Luc Van Gool, Danda Pani Paudel, Rainer Stiefelhagen
2025 J jnl
CoRR
Di Wen, Lei Qi, Kunyu Peng, Kailun Yang, Fei Teng, Ao Luo, Jia Fu, Yufan Chen, Ruiping Liu, Yitian Shi, M. Saquib Sarfraz, Rainer Stiefelhagen
2025 J jnl
CoRR
Simon Reiß, Zdravko Marinov, Alexander Jaus, Constantin Seibold, M. Saquib Sarfraz, Erik Rodner, Rainer Stiefelhagen
2025 J jnl
CoRR
Yahya Sowti Khiabani, Farris Atif, Chieh Hsu, Sven Stahlmann, Tobias Michels, Sebastian Kramer, Benedikt Heidrich, M. Saquib Sarfraz, Julian Merten, Faezeh Tafazzoli
2025 J jnl
CoRR
Mei-Yen Chen, Thi Thu Uyen Hoang, Michael Hahn, M. Saquib Sarfraz
2024 A* conf
NeurIPS
Kunyu Peng, Di Wen, Kailun Yang, Ao Luo, Yufan Chen, Jia Fu, M. Saquib Sarfraz, Alina Roitberg, Rainer Stiefelhagen
2024 J jnl
CoRR
Kunyu Peng, Di Wen, Kailun Yang, Ao Luo, Yufan Chen, Jia Fu, M. Saquib Sarfraz, Alina Roitberg, Rainer Stiefelhagen
2024 conf
ICDAR (1)
Omar Moured, Jiaming Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen
2024 C conf
IV
Ruiping Liu, Jiaming Zhang, Kunyu Peng, Yufan Chen, Ke Cao, Junwei Zheng, M. Saquib Sarfraz, Kailun Yang, Rainer Stiefelhagen
2024 J jnl
CoRR
Ruiping Liu, Jiaming Zhang, Kunyu Peng, Yufan Chen, Ke Cao, Junwei Zheng, M. Saquib Sarfraz, Kailun Yang, Rainer Stiefelhagen
2024 A* conf
CVPR
Muhammad Sohail Danish, Muhammad Haris Khan, Muhammad Akhtar Munir, M. Saquib Sarfraz, Mohsen Ali
2024 J jnl
CoRR
Muhammad Sohail Danish, Muhammad Haris Khan, Muhammad Akhtar Munir, M. Saquib Sarfraz, Mohsen Ali
2024 J jnl
CoRR
David Schneider, Sina Sajadmanesh, Vikash Sehwag, M. Saquib Sarfraz, Rainer Stiefelhagen, Lingjuan Lyu, Vivek Sharma
2024 A* conf
NeurIPS
David Schneider, Simon Reiß, Marco Kugler, Alexander Jaus, Kunyu Peng, Susanne Sutschet, M. Saquib Sarfraz, Sven Matthiesen, Rainer Stiefelhagen
2024 J jnl
CoRR
David Schneider, Simon Reiß, Marco Kugler, Alexander Jaus, Kunyu Peng, Susanne Sutschet, M. Saquib Sarfraz, Sven Matthiesen, Rainer Stiefelhagen
2024 A* conf
AAAI
Kunyu Peng, Cheng Yin, Junwei Zheng, Ruiping Liu, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg
2024 Misc conf
DICTA
Muhammad Sohail Danish, Javed Iqbal, Mohsen Ali, M. Saquib Sarfraz, Salman H. Khan, Muhammad Haris Khan
2024 A* conf
ICML
M. Saquib Sarfraz, Mei-Yen Chen, Lukas Layer, Kunyu Peng, Marios Koulakis
2024 J jnl
CoRR
M. Saquib Sarfraz, Mei-Yen Chen, Lukas Layer, Kunyu Peng, Marios Koulakis
2024 J jnl
Knowl. Based Syst.
Darko Jekauc, Diana Burkart, Julian Fritsch, Marc Hesenius, Ole Meyer, M. Saquib Sarfraz, Rainer Stiefelhagen
2024 conf
ECCV (19)
Kunyu Peng, Jia Fu, Kailun Yang, Di Wen, Yufan Chen, Ruiping Liu, Junwei Zheng, Jiaming Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg
2024 J jnl
CoRR
Kunyu Peng, Jia Fu, Kailun Yang, Di Wen, Yufan Chen, Ruiping Liu, Junwei Zheng, Jiaming Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg
2024 J jnl
CoRR
Lukas Heine, Fabian Hörst, Jana Fragemann, Gijs Luijten, Miriam Balzer, Jan Egger, Fin Hendrik Bahnsen, M. Saquib Sarfraz, Jens Kleesiek, Constantin Seibold
2024 A* conf
ACM Multimedia
Kunyu Peng, David Schneider, Alina Roitberg, Kailun Yang, Jiaming Zhang, Chen Deng, Kaiyu Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen
2023 J jnl
CoRR
Muhammad Akhtar Munir, Muhammad Haris Khan, M. Saquib Sarfraz, Mohsen Ali
2023 J jnl
IEEE Trans. Pattern Anal. Mach. Intell.
Muhammad Akhtar Munir, Muhammad Haris Khan, M. Saquib Sarfraz, Mohsen Ali
2023 J jnl
CoRR
Kunyu Peng, Di Wen, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg
2023 J jnl
CoRR
Kunyu Peng, David Schneider, Alina Roitberg, Kailun Yang, Jiaming Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen
2023 J jnl
CoRR
Kunyu Peng, Cheng Yin, Junwei Zheng, Ruiping Liu, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg
2022 conf
MICCAI (5)
Constantin Seibold, Simon Reiß, M. Saquib Sarfraz, Rainer Stiefelhagen, Jens Kleesiek
2022 J jnl
CoRR
Constantin Seibold, Simon Reiß, M. Saquib Sarfraz, Rainer Stiefelhagen, Jens Kleesiek
2022 A conf
BMVC
Constantin Marc Seibold, Simon Reiß, M. Saquib Sarfraz, Matthias A. Fink, Victoria Mayer, Jan Sellner, Moon-Sung Kim, Klaus H. Maier-Hein, Jens Kleesiek, Rainer Stiefelhagen
2022 J jnl
CoRR
Constantin Seibold, Simon Reiß, M. Saquib Sarfraz, Matthias A. Fink, Victoria Mayer, Jan Sellner, Moon-Sung Kim, Klaus H. Maier-Hein, Jens Kleesiek, Rainer Stiefelhagen
2022 A* conf
CVPR
M. Saquib Sarfraz, Marios Koulakis, Constantin Seibold, Rainer Stiefelhagen
2022 J jnl
CoRR
M. Saquib Sarfraz, Marios Koulakis, Constantin Seibold, Rainer Stiefelhagen
2022 conf
CVPR Workshops
David Schneider, M. Saquib Sarfraz, Alina Roitberg, Rainer Stiefelhagen
2022 A* conf
NeurIPS
Muhammad Akhtar Munir, Muhammad Haris Khan, M. Saquib Sarfraz, Mohsen Ali
2022 J jnl
CoRR
Muhammad Akhtar Munir, Muhammad Haris Khan, M. Saquib Sarfraz, Mohsen Ali
2021 A* conf
NeurIPS
Muhammad Akhtar Munir, Muhammad Haris Khan, M. Saquib Sarfraz, Mohsen Ali
2021 J jnl
CoRR
Muhammad Akhtar Munir, Muhammad Haris Khan, M. Saquib Sarfraz, Mohsen Ali
2021 A* conf
CVPR
M. Saquib Sarfraz, Naila Murray, Vivek Sharma, Ali Diba, Luc Van Gool, Rainer Stiefelhagen
2021 J jnl
CoRR
M. Saquib Sarfraz, Naila Murray, Vivek Sharma, Ali Diba, Luc Van Gool, Rainer Stiefelhagen
2021 A conf
WACV
Vivek Sharma, Naila Murray, Diane Larlus, M. Saquib Sarfraz, Rainer Stiefelhagen, Gabriela Csurka
2021 A* conf
ICCV
Ali Diba, Vivek Sharma, Reza Safdari, Dariush Lotfi, M. Saquib Sarfraz, Rainer Stiefelhagen, Luc Van Gool
2020 A conf
BMVC
Alexander Wolpert, Michael Teutsch, M. Saquib Sarfraz, Rainer Stiefelhagen
2020 J jnl
CoRR
Alexander Wolpert, Michael Teutsch, M. Saquib Sarfraz, Rainer Stiefelhagen
2020 B conf
FG
Vivek Sharma, Makarand Tapaswi, M. Saquib Sarfraz, Rainer Stiefelhagen
2020 J jnl
CoRR
Vivek Sharma, Makarand Tapaswi, M. Saquib Sarfraz, Rainer Stiefelhagen
2020 J jnl
IEEE Trans. Biom. Behav. Identity Sci.
Vivek Sharma, Makarand Tapaswi, M. Saquib Sarfraz, Rainer Stiefelhagen
2019 A conf
BMVC
M. Saquib Sarfraz, Constantin Seibold, Haroon Khalid, Rainer Stiefelhagen
2019 J jnl
CoRR
M. Saquib Sarfraz, Constantin Seibold, Haroon Khalid, Rainer Stiefelhagen
2019 A* conf
CVPR
M. Saquib Sarfraz, Vivek Sharma, Rainer Stiefelhagen
2019 J jnl
CoRR
M. Saquib Sarfraz, Vivek Sharma, Rainer Stiefelhagen
2019 conf
EMBC
David Ahmedt-Aristizabal, M. Saquib Sarfraz, Simon Denman, Kien Nguyen, Clinton Fookes, Sasha Dionisio, Rainer Stiefelhagen
2019 B conf
FG
Vivek Sharma, Makarand Tapaswi, M. Saquib Sarfraz, Rainer Stiefelhagen
2019 J jnl
CoRR
Vivek Sharma, Makarand Tapaswi, M. Saquib Sarfraz, Rainer Stiefelhagen
2019 conf
EMBC
David Ahmedt-Aristizabal, Kien Nguyen, Simon Denman, M. Saquib Sarfraz, Sridha Sridharan, Sasha Dionisio, Clinton Fookes
2018 A* conf
CVPR
M. Saquib Sarfraz, Arne Schumann, Andreas Eberle, Rainer Stiefelhagen
2018 conf
ICME Workshops
Benjamin S. Riggan, Nathaniel J. Short, M. Saquib Sarfraz, Shuowen Hu, He Zhang, Vishal M. Patel, Sanka Rasnayaka, Jing Li, Terence Sim, Seyed Mehdi Iranmanesh, Nasser M. Nasrabadi
2018 conf
TRECVID
Hervé Bredin, Benjamin Maurice, Ruiqing Yin, Camille Guinaudeau, Aman Behre, Claude Barras, Vivek Sharma, M. Saquib Sarfraz
2017 J jnl
CoRR
M. Saquib Sarfraz, Angela Constantinescu, Melanie Zuzej, Rainer Stiefelhagen
2017 J jnl
Inform. Spektrum
M. Saquib Sarfraz, Angela Constantinescu, Melanie Zuzej, Rainer Stiefelhagen
2017 J jnl
CoRR
M. Saquib Sarfraz, Arne Schumann, Andreas Eberle, Rainer Stiefelhagen
2017 J jnl
Int. J. Comput. Vis.
M. Saquib Sarfraz, Rainer Stiefelhagen
2017 A conf
BMVC
M. Saquib Sarfraz, Arne Schumann, Yan Wang, Rainer Stiefelhagen
2017 J jnl
CoRR
M. Saquib Sarfraz, Arne Schumann, Yan Wang, Rainer Stiefelhagen
2017 B conf
FG
Shuowen Hu, Nathaniel J. Short, Benjamin S. Riggan, Matthew Chasse, M. Saquib Sarfraz
2016 J jnl
CoRR
M. Saquib Sarfraz, Rainer Stiefelhagen
2015 conf
GCPR
Anke Schwarz, Esther-Sabrina Wacker, Manuel Martin, M. Saquib Sarfraz, Rainer Stiefelhagen
2015 J jnl
CoRR
M. Saquib Sarfraz, Rainer Stiefelhagen
2015 A conf
BMVC
M. Saquib Sarfraz, Rainer Stiefelhagen
2015 J jnl
Int. J. Document Anal. Recognit.
Muhammad Fraz, M. Saquib Sarfraz, Eran A. Edirisinghe
2014 A conf
BMVC
Muhammad Fraz, M. Saquib Sarfraz, Eran A. Edirisinghe
2014 B conf
ICPR
Muhammad Fraz, Eran A. Edirisinghe, M. Saquib Sarfraz
2013 A conf
BMVC
M. Saquib Sarfraz, Muhammad Adnan Siddique, Rainer Stiefelhagen
2013 J jnl
J. Real Time Image Process.
M. Saquib Sarfraz, Azeem Shahzad, Muhammad Adnan Elahi, Muhammad Fraz, Iffat Zafar, Eran A. Edirisinghe
2012 C conf
IGARSS
Muhammad Adnan Siddique, M. Saquib Sarfraz, David Bornemann, Olaf Hellwich
2011 conf
VISAPP
M. Saquib Sarfraz, Muhammad Haris Khan
2011 conf
VISAPP
Azeem Shahzad, Muhammad Fraz, Muhammad Adnan Elahi, M. Saquib Sarfraz
2011 conf
VISAPP
Muhammad Fraz, M. Saquib Sarfraz
2010 J jnl
Image Vis. Comput.
M. Saquib Sarfraz, Olaf Hellwich
2009 Misc conf
FIT
M. Saquib Sarfraz, Ahmed Saeed, Muhammad Haris Khan, Zahid Riaz
2009 Misc conf
FIT
Zahid Riaz, Christoph Mayer, Michael Beetz, Bernd Radig, M. Saquib Sarfraz
2008 conf
VISAPP (1)
M. Saquib Sarfraz, Olaf Hellwich
2008 conf
VISIGRAPP (Selected Papers)
M. Saquib Sarfraz, Olaf Hellwich
2008 B conf
FG
M. Saquib Sarfraz, Olaf Hellwich
APK_CODE_ANALYSIS_PDD.md
← Index APK_CODE_ANALYSIS_PDD.md markdown
# APK Code Analysis — Product Design Document

**Author:** Engineering Team
**Date:** 2026-03-07
**Status:** Draft
**Target:** redb ingestor pipeline
**Depends on:** APK_FEATURES_PDD.md (APK static analysis extractors — implemented)

---

## 1. Overview

This document describes the design for adding **DEX code analysis** (decompilation, disassembly, call graphs, cross-references, and function similarity) to the redb ingestor pipeline. This is the Android equivalent of the Binary Ninja code analysis pipeline that exists for PE and ELF binaries.

### 1.1 Goals

- Decompile and disassemble APK DEX bytecode at the **method level**, producing per-method content and reference records analogous to the Binary Ninja `code_binja_*` tables
- Extract **call graphs and cross-references** (caller/callee relationships) for each method
- Compute **function similarity hashes** (SHA-256, ssdeep, TLSH, MinHash) for method-level clustering and hunting
- **Filter out library/framework code** to focus on user-written application logic — same philosophy as the `is_lib_or_thunk()` filter in Binary Ninja analysis
- Produce **decompiled Java source** (via JADX) and **smali disassembly** (via apktool) for each method, following the content/reference split pattern used by Binary Ninja tables
- Integrate with the existing APK extractor pipeline (runs after the Phase 1 APK extractors from `APK_FEATURES_PDD.md`)

### 1.2 Non-Goals

- **Native .so library analysis** — These are equivalent to external DLLs/shared libraries in PE/ELF. They are catalogued by `APKNativeLibExtractor` but not decompiled. If deep native analysis is needed, the existing ELF pipeline can be used on extracted `.so` files in a future phase.
- **Dynamic analysis / emulation** — Out of scope
- **Full APK repackaging / patching** — We use apktool for disassembly only, not rebuild
- **Inter-procedural data-flow analysis** (e.g., FlowDroid taint tracking) — Future consideration

### 1.3 Relationship to Existing Work

| Existing | New (this PDD) |
|----------|----------------|
| `APK_FEATURES_PDD.md` — APK metadata, manifest, permissions, certificates, DEX summary, resources, native libs | DEX **code-level** analysis: per-method decompilation, disassembly, call graphs, similarity hashes |
| `DecompileBinja` — PE/ELF code analysis via Binary Ninja | `DecompileAPK` — APK/DEX code analysis via androguard + JADX + apktool |
| `code_binja_*` ClickHouse tables | `code_apk_*` ClickHouse tables (same content/reference split pattern) |

---

## 2. Background

### 2.1 DEX Bytecode vs Native Code

| Aspect | PE/ELF (Binary Ninja) | APK/DEX (This PDD) |
|--------|----------------------|---------------------|
| Code format | Machine code (x86, ARM) | Dalvik bytecode (register-based VM) |
| Basic unit | Function (by address) | Method (by class + signature) |
| Disassembly | x86/ARM mnemonics | Smali (Dalvik assembly) |
| Decompilation | Pseudo-C (HLIL) | Java source code |
| Library filtering | `is_lib_or_thunk()` — symbol type | Package prefix filtering (e.g., `android.*`, `androidx.*`, `com.google.*`) |
| Similarity hashing | SHA-256 of normalized disassembly | SHA-256 of normalized smali |

### 2.2 Tool Selection

Three tools are combined to replicate the Binary Ninja analysis depth:

| Tool | Role | Integration | License |
|------|------|-------------|---------|
| **Androguard** (Python library) | Method enumeration, call graphs, cross-references, bytecode access, permissions analysis | Direct Python import — `from androguard.misc import AnalyzeAPK` | Apache 2.0 |
| **JADX** (Java CLI) | High-quality Java decompilation (equivalent to Binary Ninja HLIL) | subprocess (following CAPA/DIE pattern) | Apache 2.0 |
| **apktool** (Java CLI) | Smali disassembly with resource decoding (equivalent to Binary Ninja disassembly) | subprocess (following CAPA/DIE pattern) | Apache 2.0 |

**Why all three:**
- **Androguard** is the analysis engine — it provides call graphs, xrefs, and method enumeration natively in Python. However, its decompiler (DAD) produces lower-quality Java than JADX.
- **JADX** produces the best Java decompilation available. It is the industry standard for Android reverse engineering (47k+ GitHub stars).
- **apktool** produces canonical smali output with decoded resources. While androguard can access bytecode, apktool's smali output is the standard interchange format for Android RE.

### 2.3 Library Filtering Strategy

Native `.so` libraries are **not reverse-engineered** — they are equivalent to external DLLs in PE or shared libraries in ELF, and are already inventoried by `APKNativeLibExtractor`.

For DEX code, we filter out **framework/library packages** to focus on user-written code. This is the Android equivalent of `is_lib_or_thunk()` in the Binary Ninja pipeline.

**Default filter list** (configurable via environment variable `APK_LIBRARY_PREFIXES`):

```
android.*              # Android SDK
androidx.*             # AndroidX support libraries
com.google.android.*   # Google Play Services, Firebase
com.google.firebase.*  # Firebase
com.google.gson.*      # Gson JSON library
com.google.protobuf.*  # Protocol Buffers
kotlin.*               # Kotlin stdlib
kotlinx.*              # Kotlin extensions
org.apache.*           # Apache Commons
com.squareup.*         # OkHttp, Retrofit, Moshi
io.reactivex.*         # RxJava
org.reactivestreams.*  # Reactive Streams
com.facebook.*         # Facebook SDK
com.crashlytics.*      # Crashlytics
io.fabric.*            # Fabric
org.junit.*            # Test frameworks
org.mockito.*          # Test frameworks
```

Methods in filtered packages are still counted in call graph edges (caller/callee arrays) but their content is not stored in content tables. This mirrors how Binary Ninja records calls to library functions in `functions_caller`/`functions_call` arrays without decompiling the library functions themselves.

### 2.4 Packer Detection

Packer/protector detection for APKs uses **DetectItEasy (DIE)**, consistent with how packer detection works for PE/ELF/Mach-O in the existing pipeline. DIE already has signatures for common Android packers (Qihoo 360, Bangcle, Ijiami, Tencent Legu, Baidu, etc.).

The existing `DIEExtractor` runs as a format-agnostic extractor before format-specific analysis and requires no changes.

---

## 3. Architecture

### 3.1 Extractor Class Hierarchy

```
Extractor (redb/extractors/extractor.py)
└── DecompileAPK (NEW — redb/extractors/decompiler/DecompileAPK.py)
    ├── Uses: APKCodeAnalyzer (NEW — redb/extractors/decompiler/apk/analyzer.py)
    │   ├── AndroguardAnalysis — call graphs, xrefs, method enumeration
    │   ├── JADXDecompiler — Java decompilation (subprocess)
    │   └── ApktoolDisassembler — smali extraction (subprocess)
    └── Produces: multi_table ClickHouse export (same pattern as DecompileBinja)
```

**Design rationale:** `DecompileAPK` extends `Extractor` directly (not `APKExtractor`) because it follows the `DecompileBinja` pattern — a standalone extractor with its own analysis engine, rather than an APK metadata extractor that shares a parsed `APK` object. The APK parsing object from androguard is used internally but not shared with other extractors.

### 3.2 Analysis Pipeline Flow

```
APK file
  │
  ├─[1]─► apktool d <apk> ─► smali files on disk (temp dir)
  │
  ├─[2]─► jadx <apk> --no-res ─► Java source files on disk (temp dir)
  │
  └─[3]─► androguard AnalyzeAPK() ─► Analysis object (in-memory)
              │
              ├── Method enumeration ──► filter library packages
              │
              ├── For each user method:
              │     ├── Read smali from apktool output [1]
              │     ├── Read Java source from JADX output [2]
              │     ├── Get xrefs from Analysis object [3]
              │     ├── Compute content hashes (SHA-256 of smali, SHA-256 of Java)
              │     ├── Compute similarity hashes (ssdeep, TLSH of smali)
              │     └── Emit content + reference records
              │
              └── Call graph export ──► caller/callee arrays per method
```

Steps [1], [2], and [3] run in parallel (apktool and JADX as subprocess, androguard in-process). All three must complete before per-method analysis begins.

### 3.3 Content/Reference Split Pattern

Following the Binary Ninja schema pattern exactly:

- **Content tables** — Keyed by `function_hash` (SHA-256 of the method content). Deduplicated: if two APKs share identical method code, only one content record exists.
- **Reference tables** — Keyed by `(sha256, method_hash)`. Links a specific binary to its methods. Contains per-binary metadata (method name, class, address, callers, callees, fuzzy hashes).

This is the same pattern as `code_binja_decompiled_functions_content` / `code_binja_decompiled_functions_references`.

### 3.4 Method-Level Hashing

Hashing is computed at the **method level** for consistency with the Binary Ninja pipeline:

| Hash | Input | Purpose |
|------|-------|---------|
| `decompiled_method_hash` | SHA-256 of decompiled Java source (whitespace-normalized) | Content deduplication, exact match |
| `smali_method_hash` | SHA-256 of smali body (instructions only, no `.method`/`.end method` directives) | Content deduplication, exact match |
| `ssdeep_smali` | ssdeep of smali body | Fuzzy similarity search |
| `tlsh_smali` | TLSH of smali body | Fuzzy similarity search |
| `minhash_smali` | MinHash signature of smali instruction n-grams | LSH-based similarity clustering |

### 3.5 Obfuscation Indicators (per method)

Computed from the smali representation:

- `short_method_name` — Method name is <= 2 characters (a, b, c — typical R8/ProGuard output)
- `short_class_name` — Enclosing class has a single-letter name
- `has_string_encryption` — Method contains `const-string` followed by decryption-pattern calls
- `has_reflection_calls` — Method uses `java.lang.reflect.*` APIs
- `excessive_goto_count` — Number of `goto` instructions exceeds threshold (control flow flattening indicator)

---

## 4. Data Model

### 4.1 New Dataclasses

```python
@dataclass
class APKDecompiledMethodContent:
    """Decompiled Java source for a single method (content table — deduplicated by hash)."""
    decompiled_method_hash: str          # SHA-256 of normalized Java source
    decompiled_method: str               # Full Java method source
    method_type: str                     # "USER" or "LIBRARY"
    has_string_encryption: bool = False
    has_reflection_calls: bool = False
    excessive_goto_count: bool = False


@dataclass
class APKDecompiledMethodReference:
    """Links a specific APK to one of its decompiled methods (reference table)."""
    sha256: str                          # APK hash
    sha1: str
    md5: str
    decompiled_method_hash: str          # FK to content table
    smali_method_hash: Optional[str]     # FK to smali content table
    class_name: str                      # e.g., "com.example.MainActivity"
    method_name: str                     # e.g., "onCreate"
    method_signature: str                # e.g., "(Landroid/os/Bundle;)V"
    method_prototype: str                # e.g., "void onCreate(Bundle)"
    functions_caller: List[str]          # Methods that call this method
    functions_call: List[str]            # Methods called by this method


@dataclass
class APKSmaliMethodContent:
    """Smali disassembly for a single method (content table — deduplicated by hash)."""
    smali_method_hash: str               # SHA-256 of normalized smali body
    smali_method: str                    # Full smali method body
    method_type: str                     # "USER" or "LIBRARY"
    instructions_count: int = 0
    register_count: int = 0
    has_string_encryption: bool = False
    has_reflection_calls: bool = False
    excessive_goto_count: bool = False


@dataclass
class APKSmaliMethodReference:
    """Links a specific APK to one of its smali methods (reference table)."""
    sha256: str
    sha1: str
    md5: str
    smali_method_hash: str               # FK to content table
    decompiled_method_hash: Optional[str] # FK to decompiled content table
    class_name: str
    method_name: str
    method_signature: str
    ssdeep_smali: Optional[str] = None
    tlsh_smali: Optional[str] = None


@dataclass
class APKMethodSimilarityMetrics:
    """Similarity hashes for method-level clustering (keyed by smali hash)."""
    smali_method_hash: str
    cyclomatic_complexity: Optional[int] = None
    ssdeep_smali: Optional[str] = None
    tlsh_smali: Optional[str] = None
    minhash: Optional[List[int]] = None


@dataclass
class APKCodeAnalysisError:
    """Error encountered during method analysis."""
    sha256: str
    class_name: Optional[str] = None
    method_name: Optional[str] = None
    error_location: str = ""             # "jadx", "apktool", "androguard", "analysis"
    error_message: Optional[str] = None
    error_type: Optional[str] = None
```

### 4.2 Tag Enum Addition

```python
# In redb/extractors/enum.py
APK_DECOMPILED = "apk_decompiled"
```

### 4.3 ClickHouse Tables

| Table | Key | Pattern | Analog |
|-------|-----|---------|--------|
| `code_apk_decompiled_methods_content` | `decompiled_method_hash` | Content (deduplicated) | `code_binja_decompiled_functions_content` |
| `code_apk_decompiled_methods_references` | `(sha256, decompiled_method_hash)` | Reference (per-binary) | `code_binja_decompiled_functions_references` |
| `code_apk_smali_methods_content` | `smali_method_hash` | Content (deduplicated) | `code_binja_disassembled_functions_content` |
| `code_apk_smali_methods_references` | `(sha256, smali_method_hash)` | Reference (per-binary) | `code_binja_disassembled_functions_references` |
| `code_apk_method_similarity_metrics` | `smali_method_hash` | Similarity | `code_binja_function_similarity_metrics` |
| `code_apk_analysis_errors` | `(sha256, class_name, method_name)` | Errors | `function_analysis_errors_binja` |

All tables use `ReplacingMergeTree(analysis_date)` engine, consistent with existing schema.

---

## 5. External Tool Management

### 5.1 JADX

- **Invocation:** `jadx --no-res --no-imports --threads-count 2 --output-dir <tmpdir> <apk_path>`
- **Flags:**
  - `--no-res` — Skip resource decompilation (androguard handles resources)
  - `--no-imports` — Omit import statements for cleaner per-method extraction
  - `--threads-count 2` — Limit threads (same as Binary Ninja worker thread limit)
- **Output:** Java source files in `<tmpdir>/<package>/<Class>.java`
- **Timeout:** Configurable via `JADX_TIMEOUT` env var (default: 600s)
- **Path:** Configurable via `JADX_PATH` env var (default: `jadx`)
- **Error handling:** If JADX fails for a specific APK, the decompiled content tables are skipped but smali analysis continues. Error logged to `code_apk_analysis_errors`.

### 5.2 apktool

- **Invocation:** `apktool d --no-res --force --output <tmpdir> <apk_path>`
- **Flags:**
  - `--no-res` — Skip resource decoding (only want smali)
  - `--force` — Overwrite output directory if exists
- **Output:** Smali files in `<tmpdir>/smali/com/example/ClassName.smali` (one per class, containing all methods)
- **Timeout:** Configurable via `APKTOOL_TIMEOUT` env var (default: 600s)
- **Path:** Configurable via `APKTOOL_PATH` env var (default: `apktool`)
- **Error handling:** Same as JADX — if apktool fails, smali content tables are skipped but decompiled Java analysis continues. Error logged.

### 5.3 Androguard

- **Invocation:** Direct Python API — `AnalyzeAPK(filepath)` returns `(APK, list[DEX], Analysis)`
- **The `Analysis` object provides:**
  - `get_methods()` — All `MethodAnalysis` objects
  - `get_call_graph()` — networkx `MultiDiGraph` of method calls
  - `MethodAnalysis.get_xref_from()` — Who calls this method
  - `MethodAnalysis.get_xref_to()` — What this method calls
  - `MethodAnalysis.get_method()` — Access to `EncodedMethod` for bytecode
- **No timeout needed** — runs in-process, same Python process

---

## 6. Ingestor Integration

In `workers.py:process_binary_file()`, the APK branch will be extended to run `DecompileAPK` after the existing APK extractors:

```python
# Existing APK extractors (from APK_FEATURES_PDD.md)
for module in apk_modules:
    extractor = module(filepath, logger, exporters=exporters, ...)
    extractor.export_data()

# NEW: Code analysis (this PDD)
if "DecompileAPK" in selected_modules or "all" in selected_modules:
    decompiler = DecompileAPK(
        filepath, logger, exporters=exporters,
        index_prefix=index_prefix, filetype="apk",
    )
    decompiler.export_data()
```

The `DecompileAPK` extractor runs with its own timeout (configurable via `APK_DECOMPILE_TIMEOUT`, default: 1800s) using the same daemon-thread pattern as `DecompileBinja`.

---

## 7. New Dependencies

### 7.1 Required (system-level)

| Tool | Installation | Version | License | Purpose |
|------|-------------|---------|---------|---------|
| **JADX** | System package or download from GitHub releases | >= 1.5 | Apache 2.0 | Java decompilation |
| **apktool** | System package or download from GitHub releases | >= 2.9 | Apache 2.0 | Smali disassembly |
| **Java Runtime** | System package (`openjdk-17-jre` or similar) | >= 11 | GPL+CE | Required by JADX and apktool |

### 7.2 Required (Python — already installed)

| Library | Current Version | Usage in this PDD |
|---------|----------------|-------------------|
| `androguard` | >=4.1 | Call graphs, xrefs, method enumeration (already in requirements.txt) |
| `ppdeep` | installed | ssdeep fuzzy hashing (already used by Binary Ninja pipeline) |
| `py-tlsh` | installed | TLSH fuzzy hashing (already used by Binary Ninja pipeline) |
| `mmh3` | installed | MinHash computation (already used by Binary Ninja pipeline) |
| `networkx` | installed via androguard | Call graph representation (transitive dependency) |

### 7.3 No new Python dependencies required

All Python libraries needed are already in `requirements.txt`. The only new system-level dependencies are JADX, apktool, and a Java runtime.

---

## 8. Implementation Phases

### Phase 1 — Core Infrastructure

1. `APKCodeAnalyzer` class — androguard integration (method enumeration, call graph, xrefs, library filtering)
2. `JADXDecompiler` wrapper — subprocess management with timeout, output parsing
3. `ApktoolDisassembler` wrapper — subprocess management with timeout, smali parsing
4. Method-level content extraction and hashing logic
5. Unit tests for all Phase 1 components

### Phase 2 — Extractor and Data Export

6. `DecompileAPK` extractor class (following `DecompileBinja` pattern)
7. ClickHouse table creation functions
8. Multi-table export (`prepare_export_data`) for all 6 tables
9. Integration with `workers.py` dispatch
10. Unit tests for extractor, schema, and export
11. Update `TEST_INDEX.md`

### Phase 3 — Similarity and Obfuscation Analysis

12. Method-level similarity hash computation (ssdeep, TLSH, MinHash on smali)
13. Obfuscation indicator computation per method
14. `code_apk_method_similarity_metrics` table population
15. Unit tests for similarity and obfuscation
16. Update `TEST_INDEX.md`

### Phase 4 — Integration Testing and Hardening

17. End-to-end integration tests with real APK samples (benign + malicious + obfuscated)
18. Edge case handling: multi-DEX, empty DEX, packed APKs, APKs with no user code
19. Performance profiling and timeout tuning
20. Final `TEST_INDEX.md` update

---

## 9. Testing Strategy

### 9.1 Unit Tests

All unit tests mock external tools (JADX, apktool, androguard) and require no system dependencies:

- **Analyzer tests** — Method enumeration, library filtering, call graph extraction, xref parsing
- **JADX wrapper tests** — Subprocess invocation, output parsing, timeout handling, error recovery
- **Apktool wrapper tests** — Same as JADX
- **Hashing tests** — SHA-256 normalization, ssdeep/TLSH computation, MinHash signature generation
- **Extractor tests** — `DecompileAPK.extract()`, `prepare_export_data()`, multi-table schema validation
- **Smali parsing tests** — Method boundary detection, instruction extraction, register counting

### 9.2 Integration Tests

Require JADX, apktool, and Java installed:

- Full pipeline run on known APK samples
- Cross-validate decompiled output against known method signatures
- Verify ClickHouse export column counts and types
- Test with obfuscated APKs (ProGuard/R8 output)

### 9.3 Markers

```python
@pytest.mark.apk           # All APK tests
@pytest.mark.decompile      # All decompiler tests
@pytest.mark.unit           # No external deps
@pytest.mark.integration    # Requires JADX/apktool/Java
```

---

## 10. Configuration

All configuration via environment variables, consistent with existing extractors:

| Variable | Default | Description |
|----------|---------|-------------|
| `JADX_PATH` | `jadx` | Path to JADX binary |
| `JADX_TIMEOUT` | `600` | JADX subprocess timeout (seconds) |
| `APKTOOL_PATH` | `apktool` | Path to apktool binary |
| `APKTOOL_TIMEOUT` | `600` | apktool subprocess timeout (seconds) |
| `APK_DECOMPILE_TIMEOUT` | `1800` | Overall decompilation timeout (seconds) |
| `APK_LIBRARY_PREFIXES` | (see §2.3) | Comma-separated package prefixes to filter |
| `APK_MIN_METHOD_INSTRUCTIONS` | `5` | Minimum smali instruction count to analyze a method |

---

## 11. Open Questions / Future Work

1. **ProGuard/R8 mapping file support** — If mapping files are bundled (rare in malware, common in crash reports), JADX can use them to restore original names. Deferred.
2. **Kotlin-specific analysis** — Kotlin metadata annotations could provide richer type information. Deferred.
3. **Cross-DEX analysis** — Multi-DEX APKs may have cross-DEX method calls. Androguard handles this via `AnalyzeAPK()` which loads all DEX files into a single `Analysis` object. No special handling needed.
4. **JADX as Java library via JPype** — Could eliminate subprocess overhead. Deferred in favor of the proven subprocess pattern, but may be revisited if performance is an issue.

---

## Appendix A — CFG Feature Parity with Binary Ninja Pipeline

**Date:** 2026-03-13
**Status:** Planned (Phase 5)
**Depends on:** Phases 13 complete

### A.1 Motivation

The Binary Ninja pipeline produces a dedicated `code_binja_cfg_functions` table with 17 fields capturing graph topology, structural hashes, and per-block feature vectors. The current APK pipeline computes only basic graph scalars (`block_count`, `edge_count`, `cyclomatic_complexity`, `loop_count`, `max_depth`, `max_fan_out`) and bundles them into the `code_apk_method_similarity_metrics` table alongside fuzzy hashes.

Analysis shows that **all advanced CFG features can be computed from smali** — this is not a limitation of Java bytecode. The existing APK infrastructure already:

- Builds `successors[]` adjacency lists from smali control flow (`smali_cfg.py`)
- Computes per-block ACFG feature vectors using the same 8-category schema as Binary Ninja (`smali_cfg.py:_build_block_features`)
- Normalizes Dalvik opcodes into semantic categories equivalent to LLIL categories (`smali_normalization.py`)

The generic functions in `cfg_features.py` (`compute_topology_hash`, `compute_md_index_topdown/bottomup`, `compute_wl_minhash`, `compute_cfg_feature_tlsh`, `pack_adjacency`) operate on adjacency lists and block feature arrays — they have no Binary Ninja dependency and can be called directly from the APK pipeline.

### A.2 Table Restructuring

Split the current single table into two, mirroring the Binja pattern:

#### `code_apk_method_similarity_metrics` (content-based fuzzy matching)

Retains only fuzzy hashes and instruction-sequence similarity data:

| Column | Type | Change |
|--------|------|--------|
| `smali_method_hash` | FixedString(64) | Unchanged |
| `cyclomatic_complexity` | Nullable(UInt16) | Stays (duplicated in both, same as Binja) |
| `ssdeep_smali` | Nullable(String) | Unchanged |
| `tlsh_smali` | Nullable(FixedString(72)) | Unchanged |
| `minhash` | Array(UInt8) | Unchanged |
| `analysis_date` | DateTime64(3, 'UTC') | Unchanged |

Removed from this table: `block_count`, `edge_count`, `loop_count`, `max_depth`, `max_fan_out` — these move to the CFG table.

#### `code_apk_cfg_methods` (NEW — structural/topological similarity)

Mirrors `code_binja_cfg_functions`:

| Column | Type | Source | Analog in Binja |
|--------|------|--------|-----------------|
| `smali_method_hash` | FixedString(64) | Existing | `disassembled_function_hash` |
| `cfg_topology_hash` | FixedString(16) | NEW — `cfg_features.compute_topology_hash()` | Same |
| `block_count` | UInt16 | Moved from similarity table | Same |
| `edge_count` | UInt16 | Moved from similarity table | Same |
| `instructions_count` | UInt32 | Existing (total Dalvik instructions) | `llil_total_operations` |
| `call_count` | UInt16 | NEW — count of `invoke-*` instructions | Same |
| `cyclomatic_complexity` | UInt16 | Moved from similarity table | Same |
| `loop_count` | UInt8 | Moved from similarity table | Same |
| `max_depth` | UInt16 | Moved from similarity table | Same |
| `max_fan_out` | UInt8 | Moved from similarity table | Same |
| `md_index_topdown` | UInt64 | NEW — `cfg_features.compute_md_index_topdown()` | Same |
| `md_index_bottomup` | UInt64 | NEW — `cfg_features.compute_md_index_bottomup()` | Same |
| `prime_product_smali` | UInt64 | NEW — Dalvik opcode → prime mapping | `prime_product_llil` |
| `cfg_feature_tlsh` | Nullable(FixedString(72)) | NEW — `cfg_features.compute_cfg_feature_tlsh()` | Same |
| `wl_minhash` | Array(UInt8) | NEW — `cfg_features.compute_wl_minhash()`, 128 elements | Same |
| `bb_features` | Array(Array(UInt16)) | Existing (computed, not exported) | Same |
| `cfg_adjacency` | Array(UInt32) | NEW — `cfg_features.pack_adjacency()` | Same |
| `analysis_date` | DateTime64(3, 'UTC') | | Same |

### A.3 Naming Differences from Binja

Two columns are intentionally renamed to reflect what the data actually represents:

- **`prime_product_smali`** (not `prime_product_llil`) — the prime mapping is applied to normalized Dalvik opcodes, not LLIL. Semantically equivalent for APK-to-APK comparison but not numerically comparable to Binja values.
- **`instructions_count`** (not `llil_total_operations`) — counts Dalvik instructions, not LLIL operations. LLIL decomposes machine instructions into sub-operations; Dalvik bytecode is already at a higher abstraction level where one instruction ≈ one operation.

### A.4 Implementation Requirements

| Task | Effort | Notes |
|------|--------|-------|
| Build `predecessors[]` from `successors[]` in `smali_cfg.py` | ~5 lines | Trivial reverse mapping |
| Define `SMALI_OP_PRIMES` mapping | ~30 lines | Map semantic categories from `smali_normalization.py` to same primes used in `cfg_features.py` |
| Wire `cfg_features.py` functions into `smali_cfg.py` | ~40 lines | Call `compute_topology_hash`, `compute_md_index_*`, `compute_wl_minhash`, `compute_cfg_feature_tlsh`, `pack_adjacency` |
| Export `bb_features` (already computed, not exported) | ~5 lines | Add to results dict |
| Count `invoke-*` instructions for `call_count` | ~5 lines | Filter in instruction loop |
| New `code_apk_cfg_methods` table export in `DecompileAPK.py` | ~60 lines | Follow existing export pattern |
| Slim down `code_apk_method_similarity_metrics` export | ~10 lines | Remove moved columns |
| ClickHouse schema for new table | ~30 lines | Mirror `code_binja_cfg_functions` |
| Unit tests | ~100 lines | Test new fields, reuse patterns from `test_cfg_features.py` |

**Total estimated: ~285 lines of code changes.**

### A.5 What Cannot Be Identical Cross-Platform

The `prime_product_smali` values are **not numerically comparable** to `prime_product_llil` from the Binja pipeline. LLIL decomposes native instructions into sub-operations (e.g., one x86 `push` becomes `STORE` + `SET_REG`), while Dalvik bytecode maps 1:1 to semantic categories. The prime products are valid for APK-vs-APK similarity and APK-vs-APK clustering, which is the intended use case.

All other fields (`cfg_topology_hash`, `md_index_*`, `wl_minhash`, `cfg_feature_tlsh`, `bb_features`, `cfg_adjacency`) are computed from the same generic algorithms and are structurally equivalent across platforms.