Vincent Lemaire

# APK Code Analysis — Product Design Document

**Author:** Engineering Team
**Date:** 2026-03-07
**Status:** Draft
**Target:** redb ingestor pipeline
**Depends on:** APK_FEATURES_PDD.md (APK static analysis extractors — implemented)

---

## 1. Overview

This document describes the design for adding **DEX code analysis** (decompilation, disassembly, call graphs, cross-references, and function similarity) to the redb ingestor pipeline. This is the Android equivalent of the Binary Ninja code analysis pipeline that exists for PE and ELF binaries.

### 1.1 Goals

- Decompile and disassemble APK DEX bytecode at the **method level**, producing per-method content and reference records analogous to the Binary Ninja `code_binja_*` tables
- Extract **call graphs and cross-references** (caller/callee relationships) for each method
- Compute **function similarity hashes** (SHA-256, ssdeep, TLSH, MinHash) for method-level clustering and hunting
- **Filter out library/framework code** to focus on user-written application logic — same philosophy as the `is_lib_or_thunk()` filter in Binary Ninja analysis
- Produce **decompiled Java source** (via JADX) and **smali disassembly** (via apktool) for each method, following the content/reference split pattern used by Binary Ninja tables
- Integrate with the existing APK extractor pipeline (runs after the Phase 1 APK extractors from `APK_FEATURES_PDD.md`)

### 1.2 Non-Goals

- **Native .so library analysis** — These are equivalent to external DLLs/shared libraries in PE/ELF. They are catalogued by `APKNativeLibExtractor` but not decompiled. If deep native analysis is needed, the existing ELF pipeline can be used on extracted `.so` files in a future phase.
- **Dynamic analysis / emulation** — Out of scope
- **Full APK repackaging / patching** — We use apktool for disassembly only, not rebuild
- **Inter-procedural data-flow analysis** (e.g., FlowDroid taint tracking) — Future consideration

### 1.3 Relationship to Existing Work

| Existing | New (this PDD) |
|----------|----------------|
| `APK_FEATURES_PDD.md` — APK metadata, manifest, permissions, certificates, DEX summary, resources, native libs | DEX **code-level** analysis: per-method decompilation, disassembly, call graphs, similarity hashes |
| `DecompileBinja` — PE/ELF code analysis via Binary Ninja | `DecompileAPK` — APK/DEX code analysis via androguard + JADX + apktool |
| `code_binja_*` ClickHouse tables | `code_apk_*` ClickHouse tables (same content/reference split pattern) |

---

## 2. Background

### 2.1 DEX Bytecode vs Native Code

| Aspect | PE/ELF (Binary Ninja) | APK/DEX (This PDD) |
|--------|----------------------|---------------------|
| Code format | Machine code (x86, ARM) | Dalvik bytecode (register-based VM) |
| Basic unit | Function (by address) | Method (by class + signature) |
| Disassembly | x86/ARM mnemonics | Smali (Dalvik assembly) |
| Decompilation | Pseudo-C (HLIL) | Java source code |
| Library filtering | `is_lib_or_thunk()` — symbol type | Package prefix filtering (e.g., `android.*`, `androidx.*`, `com.google.*`) |
| Similarity hashing | SHA-256 of normalized disassembly | SHA-256 of normalized smali |

### 2.2 Tool Selection

Three tools are combined to replicate the Binary Ninja analysis depth:

| Tool | Role | Integration | License |
|------|------|-------------|---------|
| **Androguard** (Python library) | Method enumeration, call graphs, cross-references, bytecode access, permissions analysis | Direct Python import — `from androguard.misc import AnalyzeAPK` | Apache 2.0 |
| **JADX** (Java CLI) | High-quality Java decompilation (equivalent to Binary Ninja HLIL) | subprocess (following CAPA/DIE pattern) | Apache 2.0 |
| **apktool** (Java CLI) | Smali disassembly with resource decoding (equivalent to Binary Ninja disassembly) | subprocess (following CAPA/DIE pattern) | Apache 2.0 |

**Why all three:**
- **Androguard** is the analysis engine — it provides call graphs, xrefs, and method enumeration natively in Python. However, its decompiler (DAD) produces lower-quality Java than JADX.
- **JADX** produces the best Java decompilation available. It is the industry standard for Android reverse engineering (47k+ GitHub stars).
- **apktool** produces canonical smali output with decoded resources. While androguard can access bytecode, apktool's smali output is the standard interchange format for Android RE.

### 2.3 Library Filtering Strategy

Native `.so` libraries are **not reverse-engineered** — they are equivalent to external DLLs in PE or shared libraries in ELF, and are already inventoried by `APKNativeLibExtractor`.

For DEX code, we filter out **framework/library packages** to focus on user-written code. This is the Android equivalent of `is_lib_or_thunk()` in the Binary Ninja pipeline.

**Default filter list** (configurable via environment variable `APK_LIBRARY_PREFIXES`):

```
android.*              # Android SDK
androidx.*             # AndroidX support libraries
com.google.android.*   # Google Play Services, Firebase
com.google.firebase.*  # Firebase
com.google.gson.*      # Gson JSON library
com.google.protobuf.*  # Protocol Buffers
kotlin.*               # Kotlin stdlib
kotlinx.*              # Kotlin extensions
org.apache.*           # Apache Commons
com.squareup.*         # OkHttp, Retrofit, Moshi
io.reactivex.*         # RxJava
org.reactivestreams.*  # Reactive Streams
com.facebook.*         # Facebook SDK
com.crashlytics.*      # Crashlytics
io.fabric.*            # Fabric
org.junit.*            # Test frameworks
org.mockito.*          # Test frameworks
```

Methods in filtered packages are still counted in call graph edges (caller/callee arrays) but their content is not stored in content tables. This mirrors how Binary Ninja records calls to library functions in `functions_caller`/`functions_call` arrays without decompiling the library functions themselves.

### 2.4 Packer Detection

Packer/protector detection for APKs uses **DetectItEasy (DIE)**, consistent with how packer detection works for PE/ELF/Mach-O in the existing pipeline. DIE already has signatures for common Android packers (Qihoo 360, Bangcle, Ijiami, Tencent Legu, Baidu, etc.).

The existing `DIEExtractor` runs as a format-agnostic extractor before format-specific analysis and requires no changes.

---

## 3. Architecture

### 3.1 Extractor Class Hierarchy

```
Extractor (redb/extractors/extractor.py)
└── DecompileAPK (NEW — redb/extractors/decompiler/DecompileAPK.py)
    ├── Uses: APKCodeAnalyzer (NEW — redb/extractors/decompiler/apk/analyzer.py)
    │   ├── AndroguardAnalysis — call graphs, xrefs, method enumeration
    │   ├── JADXDecompiler — Java decompilation (subprocess)
    │   └── ApktoolDisassembler — smali extraction (subprocess)
    └── Produces: multi_table ClickHouse export (same pattern as DecompileBinja)
```

**Design rationale:** `DecompileAPK` extends `Extractor` directly (not `APKExtractor`) because it follows the `DecompileBinja` pattern — a standalone extractor with its own analysis engine, rather than an APK metadata extractor that shares a parsed `APK` object. The APK parsing object from androguard is used internally but not shared with other extractors.

### 3.2 Analysis Pipeline Flow

```
APK file
  │
  ├─[1]─► apktool d <apk> ─► smali files on disk (temp dir)
  │
  ├─[2]─► jadx <apk> --no-res ─► Java source files on disk (temp dir)
  │
  └─[3]─► androguard AnalyzeAPK() ─► Analysis object (in-memory)
              │
              ├── Method enumeration ──► filter library packages
              │
              ├── For each user method:
              │     ├── Read smali from apktool output [1]
              │     ├── Read Java source from JADX output [2]
              │     ├── Get xrefs from Analysis object [3]
              │     ├── Compute content hashes (SHA-256 of smali, SHA-256 of Java)
              │     ├── Compute similarity hashes (ssdeep, TLSH of smali)
              │     └── Emit content + reference records
              │
              └── Call graph export ──► caller/callee arrays per method
```

Steps [1], [2], and [3] run in parallel (apktool and JADX as subprocess, androguard in-process). All three must complete before per-method analysis begins.

### 3.3 Content/Reference Split Pattern

Following the Binary Ninja schema pattern exactly:

- **Content tables** — Keyed by `function_hash` (SHA-256 of the method content). Deduplicated: if two APKs share identical method code, only one content record exists.
- **Reference tables** — Keyed by `(sha256, method_hash)`. Links a specific binary to its methods. Contains per-binary metadata (method name, class, address, callers, callees, fuzzy hashes).

This is the same pattern as `code_binja_decompiled_functions_content` / `code_binja_decompiled_functions_references`.

### 3.4 Method-Level Hashing

Hashing is computed at the **method level** for consistency with the Binary Ninja pipeline:

| Hash | Input | Purpose |
|------|-------|---------|
| `decompiled_method_hash` | SHA-256 of decompiled Java source (whitespace-normalized) | Content deduplication, exact match |
| `smali_method_hash` | SHA-256 of smali body (instructions only, no `.method`/`.end method` directives) | Content deduplication, exact match |
| `ssdeep_smali` | ssdeep of smali body | Fuzzy similarity search |
| `tlsh_smali` | TLSH of smali body | Fuzzy similarity search |
| `minhash_smali` | MinHash signature of smali instruction n-grams | LSH-based similarity clustering |

### 3.5 Obfuscation Indicators (per method)

Computed from the smali representation:

- `short_method_name` — Method name is <= 2 characters (a, b, c — typical R8/ProGuard output)
- `short_class_name` — Enclosing class has a single-letter name
- `has_string_encryption` — Method contains `const-string` followed by decryption-pattern calls
- `has_reflection_calls` — Method uses `java.lang.reflect.*` APIs
- `excessive_goto_count` — Number of `goto` instructions exceeds threshold (control flow flattening indicator)

---

## 4. Data Model

### 4.1 New Dataclasses

```python
@dataclass
class APKDecompiledMethodContent:
    """Decompiled Java source for a single method (content table — deduplicated by hash)."""
    decompiled_method_hash: str          # SHA-256 of normalized Java source
    decompiled_method: str               # Full Java method source
    method_type: str                     # "USER" or "LIBRARY"
    has_string_encryption: bool = False
    has_reflection_calls: bool = False
    excessive_goto_count: bool = False


@dataclass
class APKDecompiledMethodReference:
    """Links a specific APK to one of its decompiled methods (reference table)."""
    sha256: str                          # APK hash
    sha1: str
    md5: str
    decompiled_method_hash: str          # FK to content table
    smali_method_hash: Optional[str]     # FK to smali content table
    class_name: str                      # e.g., "com.example.MainActivity"
    method_name: str                     # e.g., "onCreate"
    method_signature: str                # e.g., "(Landroid/os/Bundle;)V"
    method_prototype: str                # e.g., "void onCreate(Bundle)"
    functions_caller: List[str]          # Methods that call this method
    functions_call: List[str]            # Methods called by this method


@dataclass
class APKSmaliMethodContent:
    """Smali disassembly for a single method (content table — deduplicated by hash)."""
    smali_method_hash: str               # SHA-256 of normalized smali body
    smali_method: str                    # Full smali method body
    method_type: str                     # "USER" or "LIBRARY"
    instructions_count: int = 0
    register_count: int = 0
    has_string_encryption: bool = False
    has_reflection_calls: bool = False
    excessive_goto_count: bool = False


@dataclass
class APKSmaliMethodReference:
    """Links a specific APK to one of its smali methods (reference table)."""
    sha256: str
    sha1: str
    md5: str
    smali_method_hash: str               # FK to content table
    decompiled_method_hash: Optional[str] # FK to decompiled content table
    class_name: str
    method_name: str
    method_signature: str
    ssdeep_smali: Optional[str] = None
    tlsh_smali: Optional[str] = None


@dataclass
class APKMethodSimilarityMetrics:
    """Similarity hashes for method-level clustering (keyed by smali hash)."""
    smali_method_hash: str
    cyclomatic_complexity: Optional[int] = None
    ssdeep_smali: Optional[str] = None
    tlsh_smali: Optional[str] = None
    minhash: Optional[List[int]] = None


@dataclass
class APKCodeAnalysisError:
    """Error encountered during method analysis."""
    sha256: str
    class_name: Optional[str] = None
    method_name: Optional[str] = None
    error_location: str = ""             # "jadx", "apktool", "androguard", "analysis"
    error_message: Optional[str] = None
    error_type: Optional[str] = None
```

### 4.2 Tag Enum Addition

```python
# In redb/extractors/enum.py
APK_DECOMPILED = "apk_decompiled"
```

### 4.3 ClickHouse Tables

| Table | Key | Pattern | Analog |
|-------|-----|---------|--------|
| `code_apk_decompiled_methods_content` | `decompiled_method_hash` | Content (deduplicated) | `code_binja_decompiled_functions_content` |
| `code_apk_decompiled_methods_references` | `(sha256, decompiled_method_hash)` | Reference (per-binary) | `code_binja_decompiled_functions_references` |
| `code_apk_smali_methods_content` | `smali_method_hash` | Content (deduplicated) | `code_binja_disassembled_functions_content` |
| `code_apk_smali_methods_references` | `(sha256, smali_method_hash)` | Reference (per-binary) | `code_binja_disassembled_functions_references` |
| `code_apk_method_similarity_metrics` | `smali_method_hash` | Similarity | `code_binja_function_similarity_metrics` |
| `code_apk_analysis_errors` | `(sha256, class_name, method_name)` | Errors | `function_analysis_errors_binja` |

All tables use `ReplacingMergeTree(analysis_date)` engine, consistent with existing schema.

---

## 5. External Tool Management

### 5.1 JADX

- **Invocation:** `jadx --no-res --no-imports --threads-count 2 --output-dir <tmpdir> <apk_path>`
- **Flags:**
  - `--no-res` — Skip resource decompilation (androguard handles resources)
  - `--no-imports` — Omit import statements for cleaner per-method extraction
  - `--threads-count 2` — Limit threads (same as Binary Ninja worker thread limit)
- **Output:** Java source files in `<tmpdir>/<package>/<Class>.java`
- **Timeout:** Configurable via `JADX_TIMEOUT` env var (default: 600s)
- **Path:** Configurable via `JADX_PATH` env var (default: `jadx`)
- **Error handling:** If JADX fails for a specific APK, the decompiled content tables are skipped but smali analysis continues. Error logged to `code_apk_analysis_errors`.

### 5.2 apktool

- **Invocation:** `apktool d --no-res --force --output <tmpdir> <apk_path>`
- **Flags:**
  - `--no-res` — Skip resource decoding (only want smali)
  - `--force` — Overwrite output directory if exists
- **Output:** Smali files in `<tmpdir>/smali/com/example/ClassName.smali` (one per class, containing all methods)
- **Timeout:** Configurable via `APKTOOL_TIMEOUT` env var (default: 600s)
- **Path:** Configurable via `APKTOOL_PATH` env var (default: `apktool`)
- **Error handling:** Same as JADX — if apktool fails, smali content tables are skipped but decompiled Java analysis continues. Error logged.

### 5.3 Androguard

- **Invocation:** Direct Python API — `AnalyzeAPK(filepath)` returns `(APK, list[DEX], Analysis)`
- **The `Analysis` object provides:**
  - `get_methods()` — All `MethodAnalysis` objects
  - `get_call_graph()` — networkx `MultiDiGraph` of method calls
  - `MethodAnalysis.get_xref_from()` — Who calls this method
  - `MethodAnalysis.get_xref_to()` — What this method calls
  - `MethodAnalysis.get_method()` — Access to `EncodedMethod` for bytecode
- **No timeout needed** — runs in-process, same Python process

---

## 6. Ingestor Integration

In `workers.py:process_binary_file()`, the APK branch will be extended to run `DecompileAPK` after the existing APK extractors:

```python
# Existing APK extractors (from APK_FEATURES_PDD.md)
for module in apk_modules:
    extractor = module(filepath, logger, exporters=exporters, ...)
    extractor.export_data()

# NEW: Code analysis (this PDD)
if "DecompileAPK" in selected_modules or "all" in selected_modules:
    decompiler = DecompileAPK(
        filepath, logger, exporters=exporters,
        index_prefix=index_prefix, filetype="apk",
    )
    decompiler.export_data()
```

The `DecompileAPK` extractor runs with its own timeout (configurable via `APK_DECOMPILE_TIMEOUT`, default: 1800s) using the same daemon-thread pattern as `DecompileBinja`.

---

## 7. New Dependencies

### 7.1 Required (system-level)

| Tool | Installation | Version | License | Purpose |
|------|-------------|---------|---------|---------|
| **JADX** | System package or download from GitHub releases | >= 1.5 | Apache 2.0 | Java decompilation |
| **apktool** | System package or download from GitHub releases | >= 2.9 | Apache 2.0 | Smali disassembly |
| **Java Runtime** | System package (`openjdk-17-jre` or similar) | >= 11 | GPL+CE | Required by JADX and apktool |

### 7.2 Required (Python — already installed)

| Library | Current Version | Usage in this PDD |
|---------|----------------|-------------------|
| `androguard` | >=4.1 | Call graphs, xrefs, method enumeration (already in requirements.txt) |
| `ppdeep` | installed | ssdeep fuzzy hashing (already used by Binary Ninja pipeline) |
| `py-tlsh` | installed | TLSH fuzzy hashing (already used by Binary Ninja pipeline) |
| `mmh3` | installed | MinHash computation (already used by Binary Ninja pipeline) |
| `networkx` | installed via androguard | Call graph representation (transitive dependency) |

### 7.3 No new Python dependencies required

All Python libraries needed are already in `requirements.txt`. The only new system-level dependencies are JADX, apktool, and a Java runtime.

---

## 8. Implementation Phases

### Phase 1 — Core Infrastructure

1. `APKCodeAnalyzer` class — androguard integration (method enumeration, call graph, xrefs, library filtering)
2. `JADXDecompiler` wrapper — subprocess management with timeout, output parsing
3. `ApktoolDisassembler` wrapper — subprocess management with timeout, smali parsing
4. Method-level content extraction and hashing logic
5. Unit tests for all Phase 1 components

### Phase 2 — Extractor and Data Export

6. `DecompileAPK` extractor class (following `DecompileBinja` pattern)
7. ClickHouse table creation functions
8. Multi-table export (`prepare_export_data`) for all 6 tables
9. Integration with `workers.py` dispatch
10. Unit tests for extractor, schema, and export
11. Update `TEST_INDEX.md`

### Phase 3 — Similarity and Obfuscation Analysis

12. Method-level similarity hash computation (ssdeep, TLSH, MinHash on smali)
13. Obfuscation indicator computation per method
14. `code_apk_method_similarity_metrics` table population
15. Unit tests for similarity and obfuscation
16. Update `TEST_INDEX.md`

### Phase 4 — Integration Testing and Hardening

17. End-to-end integration tests with real APK samples (benign + malicious + obfuscated)
18. Edge case handling: multi-DEX, empty DEX, packed APKs, APKs with no user code
19. Performance profiling and timeout tuning
20. Final `TEST_INDEX.md` update

---

## 9. Testing Strategy

### 9.1 Unit Tests

All unit tests mock external tools (JADX, apktool, androguard) and require no system dependencies:

- **Analyzer tests** — Method enumeration, library filtering, call graph extraction, xref parsing
- **JADX wrapper tests** — Subprocess invocation, output parsing, timeout handling, error recovery
- **Apktool wrapper tests** — Same as JADX
- **Hashing tests** — SHA-256 normalization, ssdeep/TLSH computation, MinHash signature generation
- **Extractor tests** — `DecompileAPK.extract()`, `prepare_export_data()`, multi-table schema validation
- **Smali parsing tests** — Method boundary detection, instruction extraction, register counting

### 9.2 Integration Tests

Require JADX, apktool, and Java installed:

- Full pipeline run on known APK samples
- Cross-validate decompiled output against known method signatures
- Verify ClickHouse export column counts and types
- Test with obfuscated APKs (ProGuard/R8 output)

### 9.3 Markers

```python
@pytest.mark.apk           # All APK tests
@pytest.mark.decompile      # All decompiler tests
@pytest.mark.unit           # No external deps
@pytest.mark.integration    # Requires JADX/apktool/Java
```

---

## 10. Configuration

All configuration via environment variables, consistent with existing extractors:

| Variable | Default | Description |
|----------|---------|-------------|
| `JADX_PATH` | `jadx` | Path to JADX binary |
| `JADX_TIMEOUT` | `600` | JADX subprocess timeout (seconds) |
| `APKTOOL_PATH` | `apktool` | Path to apktool binary |
| `APKTOOL_TIMEOUT` | `600` | apktool subprocess timeout (seconds) |
| `APK_DECOMPILE_TIMEOUT` | `1800` | Overall decompilation timeout (seconds) |
| `APK_LIBRARY_PREFIXES` | (see §2.3) | Comma-separated package prefixes to filter |
| `APK_MIN_METHOD_INSTRUCTIONS` | `5` | Minimum smali instruction count to analyze a method |

---

## 11. Open Questions / Future Work

1. **ProGuard/R8 mapping file support** — If mapping files are bundled (rare in malware, common in crash reports), JADX can use them to restore original names. Deferred.
2. **Kotlin-specific analysis** — Kotlin metadata annotations could provide richer type information. Deferred.
3. **Cross-DEX analysis** — Multi-DEX APKs may have cross-DEX method calls. Androguard handles this via `AnalyzeAPK()` which loads all DEX files into a single `Analysis` object. No special handling needed.
4. **JADX as Java library via JPype** — Could eliminate subprocess overhead. Deferred in favor of the proven subprocess pattern, but may be revisited if performance is an issue.

---

## Appendix A — CFG Feature Parity with Binary Ninja Pipeline

**Date:** 2026-03-13
**Status:** Planned (Phase 5)
**Depends on:** Phases 1–3 complete

### A.1 Motivation

The Binary Ninja pipeline produces a dedicated `code_binja_cfg_functions` table with 17 fields capturing graph topology, structural hashes, and per-block feature vectors. The current APK pipeline computes only basic graph scalars (`block_count`, `edge_count`, `cyclomatic_complexity`, `loop_count`, `max_depth`, `max_fan_out`) and bundles them into the `code_apk_method_similarity_metrics` table alongside fuzzy hashes.

Analysis shows that **all advanced CFG features can be computed from smali** — this is not a limitation of Java bytecode. The existing APK infrastructure already:

- Builds `successors[]` adjacency lists from smali control flow (`smali_cfg.py`)
- Computes per-block ACFG feature vectors using the same 8-category schema as Binary Ninja (`smali_cfg.py:_build_block_features`)
- Normalizes Dalvik opcodes into semantic categories equivalent to LLIL categories (`smali_normalization.py`)

The generic functions in `cfg_features.py` (`compute_topology_hash`, `compute_md_index_topdown/bottomup`, `compute_wl_minhash`, `compute_cfg_feature_tlsh`, `pack_adjacency`) operate on adjacency lists and block feature arrays — they have no Binary Ninja dependency and can be called directly from the APK pipeline.

### A.2 Table Restructuring

Split the current single table into two, mirroring the Binja pattern:

#### `code_apk_method_similarity_metrics` (content-based fuzzy matching)

Retains only fuzzy hashes and instruction-sequence similarity data:

| Column | Type | Change |
|--------|------|--------|
| `smali_method_hash` | FixedString(64) | Unchanged |
| `cyclomatic_complexity` | Nullable(UInt16) | Stays (duplicated in both, same as Binja) |
| `ssdeep_smali` | Nullable(String) | Unchanged |
| `tlsh_smali` | Nullable(FixedString(72)) | Unchanged |
| `minhash` | Array(UInt8) | Unchanged |
| `analysis_date` | DateTime64(3, 'UTC') | Unchanged |

Removed from this table: `block_count`, `edge_count`, `loop_count`, `max_depth`, `max_fan_out` — these move to the CFG table.

#### `code_apk_cfg_methods` (NEW — structural/topological similarity)

Mirrors `code_binja_cfg_functions`:

| Column | Type | Source | Analog in Binja |
|--------|------|--------|-----------------|
| `smali_method_hash` | FixedString(64) | Existing | `disassembled_function_hash` |
| `cfg_topology_hash` | FixedString(16) | NEW — `cfg_features.compute_topology_hash()` | Same |
| `block_count` | UInt16 | Moved from similarity table | Same |
| `edge_count` | UInt16 | Moved from similarity table | Same |
| `instructions_count` | UInt32 | Existing (total Dalvik instructions) | `llil_total_operations` |
| `call_count` | UInt16 | NEW — count of `invoke-*` instructions | Same |
| `cyclomatic_complexity` | UInt16 | Moved from similarity table | Same |
| `loop_count` | UInt8 | Moved from similarity table | Same |
| `max_depth` | UInt16 | Moved from similarity table | Same |
| `max_fan_out` | UInt8 | Moved from similarity table | Same |
| `md_index_topdown` | UInt64 | NEW — `cfg_features.compute_md_index_topdown()` | Same |
| `md_index_bottomup` | UInt64 | NEW — `cfg_features.compute_md_index_bottomup()` | Same |
| `prime_product_smali` | UInt64 | NEW — Dalvik opcode → prime mapping | `prime_product_llil` |
| `cfg_feature_tlsh` | Nullable(FixedString(72)) | NEW — `cfg_features.compute_cfg_feature_tlsh()` | Same |
| `wl_minhash` | Array(UInt8) | NEW — `cfg_features.compute_wl_minhash()`, 128 elements | Same |
| `bb_features` | Array(Array(UInt16)) | Existing (computed, not exported) | Same |
| `cfg_adjacency` | Array(UInt32) | NEW — `cfg_features.pack_adjacency()` | Same |
| `analysis_date` | DateTime64(3, 'UTC') | | Same |

### A.3 Naming Differences from Binja

Two columns are intentionally renamed to reflect what the data actually represents:

- **`prime_product_smali`** (not `prime_product_llil`) — the prime mapping is applied to normalized Dalvik opcodes, not LLIL. Semantically equivalent for APK-to-APK comparison but not numerically comparable to Binja values.
- **`instructions_count`** (not `llil_total_operations`) — counts Dalvik instructions, not LLIL operations. LLIL decomposes machine instructions into sub-operations; Dalvik bytecode is already at a higher abstraction level where one instruction ≈ one operation.

### A.4 Implementation Requirements

| Task | Effort | Notes |
|------|--------|-------|
| Build `predecessors[]` from `successors[]` in `smali_cfg.py` | ~5 lines | Trivial reverse mapping |
| Define `SMALI_OP_PRIMES` mapping | ~30 lines | Map semantic categories from `smali_normalization.py` to same primes used in `cfg_features.py` |
| Wire `cfg_features.py` functions into `smali_cfg.py` | ~40 lines | Call `compute_topology_hash`, `compute_md_index_*`, `compute_wl_minhash`, `compute_cfg_feature_tlsh`, `pack_adjacency` |
| Export `bb_features` (already computed, not exported) | ~5 lines | Add to results dict |
| Count `invoke-*` instructions for `call_count` | ~5 lines | Filter in instruction loop |
| New `code_apk_cfg_methods` table export in `DecompileAPK.py` | ~60 lines | Follow existing export pattern |
| Slim down `code_apk_method_similarity_metrics` export | ~10 lines | Remove moved columns |
| ClickHouse schema for new table | ~30 lines | Mirror `code_binja_cfg_functions` |
| Unit tests | ~100 lines | Test new fields, reuse patterns from `test_cfg_features.py` |

**Total estimated: ~285 lines of code changes.**

### A.5 What Cannot Be Identical Cross-Platform

The `prime_product_smali` values are **not numerically comparable** to `prime_product_llil` from the Binja pipeline. LLIL decomposes native instructions into sub-operations (e.g., one x86 `push` becomes `STORE` + `SET_REG`), while Dalvik bytecode maps 1:1 to semantic categories. The prime products are valid for APK-vs-APK similarity and APK-vs-APK clustering, which is the intended use case.

All other fields (`cfg_topology_hash`, `md_index_*`, `wl_minhash`, `cfg_feature_tlsh`, `bb_features`, `cfg_adjacency`) are computed from the same generic algorithms and are structurally equivalent across platforms.