Post cover image

August 14, 2026

From Smali to Frida: How Codex Accelerated My Android Security Assessment

By Raj

9 min read

This is my first article on Medium, and I wanted to start with something that closely reflects the kind of work I enjoy most. Drum roll, please: 🥁🥁🥁 Diving into mobile security controls and figuring out how they actually work. 😅

During a recent authorized Android security assessment, I was testing an application protected by multiple layers of security controls, which is fairly standard for fintech and banking applications. What caught my attention, was something unusual; the Frida Server running on my Android device kept getting killed, along with the application itself.

Then a thought crossed my mind 🤔 Why not bring AI into the workflow and see if it could help me solve the problem?

Instead of simply asking it to generate a generic "root detection bypass" or an all-in-one Frida script, I used Codex alongside my usual mobile pentesting workflow, feeding it decompiled APKs, Smali code, Frida output, ADB results, Logcat logs, and other runtime evidence.

The idea wasn't to ask AI for a universal bypass. Instead, I treated Codex as part of an iterative security-testing workflow:

  1. Inspect the application artifacts.
  2. Form a hypothesis about the detection or enforcement path.
  3. Instrument the application and collect runtime evidence.
  4. Revise the hooks based on what has actually happened.
  5. Repeat until the result was stable and reproducible.

Scope and Anonymization 🎯

The application, vendors, package identifier, network hosts, class names, library names, hashes, and native offsets mentioned in this article have been anonymized.

Two distinct third-party components were relevant during the assessment:

  • A commercial mobile protection SDK provided the primary Runtime Application Self-Protection (RASP) controls.
  • A third-party observability SDK included an independent root-detection mechanism used for device and crash telemetry. It was not the primary RASP component.

Test Environment 🧪

My test environment included:

  • A rooted Android 15 test device
  • A systemless root-management framework (Magisk) with an application denylist
  • Device-integrity-related test modules
  • A renamed Frida Server binary running from a non-default location
  • Decompiled base and split APKs
  • Matching Frida client and server versions
  • A local HTTPS interception proxy

Protecting the Frida Transport 🛰️

First things first, the main headache 🫩 that I realized was that simply renaming the Frida Server binary was not enough.

An application can still detect instrumentation by probing localhost ports or inspecting process, memory, and runtime artifacts.

Instead of exposing a TCP listener directly on the Android device, I ran the renamed Frida Server using an abstract Unix socket and forwarded the connection through ADB:

adb shell su -c "/data/local/tmp/<renamed-server> -D -l unix:/data/local/tmp/<socket-name>.sock"
adb forward tcp:<local-port> localabstract:/data/local/tmp/<socket-name>.sock

This allowed my workstation to communicate with Frida through a locally forwarded port without leaving a device-side TCP listener for the application to probe.

It did not make Frida completely invisible, but it removed one obvious detection surface and gave me a cleaner starting point for analyzing the application's anti-instrumentation behavior.

Giving Codex useful evidence 🔎

I provided Codex with:

  • Decompiled APK directories
  • The application identifier
  • Frida runtime output
  • Android Logcat and process-death records
  • Read access to the current installed APK set
  • Access to the connected test device

That context was essential. Obfuscated applications rarely expose descriptive names such as RootDetector or terminateApplication. A reliable assessment must correlate multiple weak signals rather than trust a single class name.

Codex searched the smali for relevant Android APIs and strings, including:

Debug.isDebuggerConnected
Debug.waitingForDebugger
adb_enabled
development_settings_enabled
System.exit
Process.killProcess
finishAffinity
finishAndRemoveTask
Certificate pinning failure
checkServerTrusted

It also inspected the application's native protection and root-telemetry libraries.

Starting from the installed application 📦

The first step was to identify the installed package metadata and extract its exact split set:

adb shell pm path <application-id>
adb pull <base-apk-path>
adb pull <architecture-split-path>
adb pull <language-split-path>
adb pull <density-split-path>

I recorded the installed version, version code, target SDK, ABI, split paths, and cryptographic hashes. I then decompiled this extracted set and used it as the sole source for static analysis.

This established a clean rule for the rest of the work: every obfuscated class mapping and native offset had to be derived from the installed build and then confirmed at runtime.

Root and debugger detection 🛡️

Static and runtime analysis identified multiple detection layers. This is where the real chase began. 🕵️

Java debugger APIs

The application invoked:

Debug.isDebuggerConnected();
Debug.waitingForDebugger();

The Frida script forced both methods to return false.

Developer options and ADB

The protection logic read the following Android settings through both global and secure settings providers:

adb_enabled
development_settings_enabled

Targeted hooks returned 0 only for these security-relevant keys.

Build properties

Suspicious build properties were normalized to production-like values:

None

Root artifacts

The application checked for common root artifacts using both Java file APIs and native libc functions. Representative categories included:

  • su binaries
  • Root-management directories
  • Systemless-root files
  • Root-framework modules
  • BusyBox and superuser applications

The bypass covered Java methods such as File.exists(), File.canRead(), and File.canExecute(), along with native functions such as open, openat, access, stat, and fopen.

Runtime testing confirmed commands that searched for su or queried sensitive system properties. The script intercepted matching Runtime.exec() and ProcessBuilder calls and made them fail cleanly.

Root-management packages

Package-manager lookups for common root, superuser, hooking, and framework management applications were filtered. Direct queries behaved as though the package did not exist, while installed-package lists had matching entries removed.

Independent telemetry-side root detection

The observability SDK performed a separate root assessment. Its checks included:

  • Test build tags
  • Root binary locations
  • su command execution
  • Insecure build properties
  • A native JNI root check

These checks appeared to enrich device and crash telemetry rather than drive the primary application shutdown path. The Java decision methods and native JNI result were nevertheless forced to report a non-rooted device for consistent testing.

Tracing the Java enforcement path 🧵

The commercial protection SDK initialized through the application startup graph and registered an obfuscated threat listener.

It would have been tempting to disable SDK initialization completely. Static analysis showed that other components expected the SDK state to exist, however, and suppressing initialization could cause unrelated failures.

The safer strategy was to let the SDK initialize and intercept its result and enforcement layers.

An obfuscated Kotlin coroutine with a specific discriminator performed the shutdown sequence:

Activity.finishAffinity()
ActivityManager.AppTask.finishAndRemoveTask()
System.exit()

The first hook intercepted the coroutine's public invoke() entry point, but the application still exited. That was the clue I needed. 🧩

To understand why, Codex instrumented the native bridge used by Runtime.exit() and captured the Java stack. The stack showed that Kotlin was resuming the coroutine directly through invokeSuspend(), bypassing the invoke() hook.

The corrected script hooked both paths and returned kotlin.Unit only for the shutdown discriminator. Other variants of the shared obfuscated class continued normally.

This targeted approach was safer than disabling every method in the class or globally suppressing all calls to System.exit().

Mapping the native termination routine ⚙️

After the Java shutdown path was blocked, the application progressed further but later attempted to send a fatal signal to its own process.

The caller originated inside the native protection library. Codex used the runtime return address and ARM64 disassembly to identify:

  • The start of the enforcement function
  • Its legitimate clean-return branch
  • The self-termination call
  • A second termination primitive reached after the signal call

The enclosing function was replaced with a clean return.

Because this address was specific to one application and protection-library build, the script included a module-size guard. If a different library version is loaded, the hook refuses to apply the offset.

For a production assessment, a stronger fingerprint could include the library hash or a verified instruction signature around the target function.

Concealing instrumentation artifacts 🥷

The application inspected /proc and native process state for evidence of instrumentation. The bypass therefore covered more than the Frida Server name.

Relevant surfaces included:

  • /proc/self/maps
  • /proc/self/status
  • TracerPid
  • Loaded-module enumeration
  • Thread names
  • Frida and Gum strings
  • Native memory searches
  • Debugger attachment through ptrace
  • Process termination functions

The native hooks included:

open / openat / fopen
read / fgets / readlink
strstr / strcasestr / memmem
dl_iterate_phdr
pthread_getname_np / prctl
ptrace
kill / tgkill / raise

Where possible, filtering was restricted to calls originating from known detector modules. Caller-scoped hooks reduce the risk of changing unrelated application behavior.

Analyzing the SSL-pinning layers 🔐

Once the root and instrumentation bypass was stable, I moved to TLS validation, the next layer of the puzzle. 🔐

The application used several overlapping mechanisms:

  • SSLContext trust managers
  • Android Conscrypt validation
  • Android Network Security Config pins
  • An R8-obfuscated OkHttp certificate pinner
  • A payment SDK's custom trust manager
  • SDK-level SSL-pinning configuration
  • WebView certificate-error handling
  • Native X.509 verification

Finding the current OkHttp pinner

Searching the extracted smali for the text Certificate pinning failure revealed the application's obfuscated OkHttp pinner. Its method signature corresponded to the obfuscated equivalent of:

check(hostname, peerCertificatesProvider)

The hook replaced only that exact signature with an immediate successful return. No class was hooked merely because its name resembled a known mapping; the expected signature and pinning-failure code both had to be present.

Trust manager replacement

The script registered a custom X509TrustManager and supplied it whenever the application initialized an SSLContext.

Conscrypt and Network Security Config

Hooks covered Conscrypt's chain-verification paths and Android's internal Network Security Config pin checks.

Hostname verification

A permissive HostnameVerifier was installed for test traffic using HttpsURLConnection.

Payment SDK pinning

The bundled payment SDK implemented its own trust manager and certificate-pin validation helper. Both paths were intercepted, including overloads that received a hostname.

SDK configuration flag

The application exposed an SDK-wide SSL-pinning option. Its getter was forced to return false, while attempts to enable the option were rewritten to keep it disabled.

Native certificate verification

When available, native verification exports were replaced with successful results:

X509_verify_cert       → success
SSL_get_verify_result  → no verification error

The script initialized Frida's Java bridge before installing these native SSL hooks. During this assessment, loading a large set of native hooks first caused ART class-resolution failures; reversing the order produced reliable Java hook installation on this Android build.

Runtime validation 🧪

Then came the moment of truth: the final launch loaded the root/debug and SSL scripts together. ✅

frida -H 127.0.0.1:<local-port> `
  -f <application-id> `
  -l .\frida_root_debug_bypass.js `
  -l .\frida_ssl_pinning_bypass.js

Runtime logs confirmed that the following controls were exercised:

  • Java debugger checks
  • ADB and developer-setting checks
  • Root file and package lookups
  • Root-related command execution
  • /proc map and status inspection
  • The Java shutdown coroutine
  • The native termination routine
  • The observability SDK's root detector
  • Conscrypt certificate validation
  • Network Security Config pins
  • Hostname verification
  • The application's obfuscated OkHttp pinner
  • The payment SDK's certificate validation
  • Native X.509 verification

The root and instrumentation configuration remained alive through the full validation interval, passing both early enforcement events and the delayed instrumentation-probe window. The combined SSL test also processed real application traffic through the interception environment.

Finally, both Frida agents were eternalized in the application process and the process remained alive through repeated health checks.

What Codex contributed 🤖

Codex was most valuable when continuously correlating multiple forms of evidence:

  • Searching large, obfuscated smali trees
  • Comparing application releases
  • Verifying class mappings and native offsets against current artifacts
  • Generating and editing Frida hooks
  • Checking JavaScript and PowerShell syntax
  • Running controlled tests against the device
  • Interpreting Frida and Logcat output
  • Mapping runtime addresses back to native code
  • Revising hooks when the first approach failed
  • Maintaining the assessment document alongside the scripts

The human role remained equally important:

  • Defining and enforcing the authorized scope
  • Preparing the rooted device and proxy environment
  • Deciding which behavior was safe to modify
  • Reviewing hooks for unintended business-logic impact
  • Distinguishing protection enforcement from ordinary telemetry
  • Interpreting the security significance of the results

The productive model was not "AI produced a bypass." It was "AI accelerated an evidence-driven testing loop."

Lessons learned 💡

Build identity is a prerequisite

Before analyzing obfuscated code, extract and fingerprint the installed package. Class mappings and native offsets are meaningful only for the exact build from which they were derived.

Runtime evidence should guide static analysis

The decisive clues came from runtime behavior: Java stacks, native caller addresses, process-death timing, and detector logs. These observations narrowed a large obfuscated codebase to a small number of relevant paths.

Hook the result or enforcement layer

Preventing a protection SDK from initializing can destabilize components that depend on its state. Allowing initialization and intercepting the resulting decision, callback, or enforcement action is often safer.

Prefer caller-scoped termination protection

Globally disabling System.exit(), kill(), or raise() can conceal genuine application failures. Caller-scoped hooks and verified enforcement-function patches are less disruptive.

Renaming Frida Server is not concealment

Applications can detect instrumentation through ports, /proc, memory maps, module enumeration, thread names, and active probes. Transport configuration and in-process concealment must be considered together.

SSL bypass and traffic interception are separate tasks

Removing certificate pinning does not configure the test proxy, route device traffic, or install a test CA. Those remain separate environment requirements.

AI-generated hooks still require verification

A hook can be syntactically correct and conceptually wrong. Each important hook should be tied to static evidence, runtime invocation, and observable behavior.

Defensive recommendations 🧱

This assessment reinforces that client-side protections are valuable friction, but they should not be treated as an absolute trust boundary.

Development teams should:

  • Enforce sensitive authorization decisions on the server.
  • Treat device-integrity verdicts as one risk signal rather than the sole authorization control.
  • Bind important requests to authenticated sessions and transaction context.
  • Monitor unusual combinations of device, session, and behavioral signals.
  • Avoid relying on one predictable application-exit path.
  • Verify that disabling telemetry or RASP does not expose privileged features.
  • Keep secrets and irreversible authorization decisions out of the client.
  • Reassess protection behavior after every SDK or application update.
  • Include bypass-resistance testing in the mobile release process.

Conclusion 🚀

And now for the million-dollar question 💵: Will AI eventually replace humans in mobile security?

If you ask me, the answer is no. Not yet, at least. Human oversight is still essential for defining the scope, preparing the test environment, interpreting unexpected behavior, and determining whether something has actually gone wrong.

None

In this assessment, Codex significantly accelerated the work, but the successful outcome came from an iterative process rather than one-shot code generation.

The process required extracting the installed APKs, mapping obfuscated Java controls, tracing coroutine execution, locating the native termination routine, building SSL-pinning hooks, and validating everything against real runtime behavior.

My takeaway is simple: AI is most useful in security engineering when it is grounded in authentic artifacts, controlled experiments, runtime evidence, and careful human review. AI can accelerate the investigation, but for now, humans remain firmly in the loop.