Post cover image
A decade of working for others, an year of working for the love of subject ❤

September 20, 2026

Hunting TOCTOU Races and Memory-Safety Bugs in NVIDIA’s Native Toolchain

Titsec’s log from the NVIDIA Public Bug Bounty (via Intigriti): what I actually ran, what happened when I filed it, and the methodology…

By Titsecop

11 min read

Titsec's log from the NVIDIA Public Bug Bounty (via Intigriti): what I actually ran, what happened when I filed it, and the methodology behind both. Some of what I found in this line of work is still open with the vendor — that part stays out of this post.

I've spent a chunk of this year doing native/local vulnerability research against NVIDIA's stack under their public bounty program on Intigriti — started 2026–06–23. Not web hunting: this is source-level auditing and dynamic PoC-building against C and Go codebases that run with real host privilege, scoped AV:Local. Two findings from that work are closed and paid, and I can walk through those with real dates and real commands. A separate line of memory-safety research against NVIDIA's compiler-toolchain CLI tooling is still sitting with their team, unresolved — I'm not naming the tools, the bug class specifics, or anything else about it here. Publishing that before a fix ships isn't research, it's just handing out a weapon.

What I can give you is the actual process, because the process is the reusable part.

The two closed findings

H-1 — TOCTOU in libnvidia-container's mount core

Class: a time-of-check/time-of-use race in the mount-setup path of libnvidia-container's C core — same general family as the publicly-disclosed CVE-2024–0132 container-escape class (more on that below, since it's fully public and makes a better teaching example than my own writeup of my own bug).

Timeline, as it actually happened:

  • 2026–06–23, 5:21 PM — filed on Intigriti.
  • 2026–06–23, 10:40 PM — closed Informative same night. The triager disputed my version-dating.
  • 2026–07–02 → 2026–07–08 — rebuttal thread, clarified the exact version/config I'd tested against.
  • 2026–07–04 — built and posted a live-exploit proof against the real, unmodified production binary, zero physical GPU hardware required to demonstrate it. First honest run: 11/100 trials.
  • 2026–07–04, 10:11 PM — triager replies the PoC doesn't reproduce on their end.
  • 2026–07–05 → 2026–07–23 — I go find out why. Three environment/harness gaps on my side, fixed one at a time, re-verified at 4/50 trials — lower raw count, but a consistent rate once the harness was actually correct, which matters more than the raw number. Posted a corrected, portable harness.
  • 2026–07–28 → 2026–08–20 — no further movement through normal channels, so I escalated directly to psirt@nvidia.com. They acknowledged and said they'd raised it with the Intigriti triage side.
  • 2026–08–21 → 2026–08–22 — reopened, then closed Informative again.
  • 2026–08–22 — I pushed back with a statistical argument (11/100 and 4/50 with a fixed harness isn't noise) and asked for a larger trial run on their side instead of taking my word for it.
  • 2026–08–25, 12:16 PM — status flips from Informative back to Triage.
  • 2026–08–25, 9:41 PM — closed as Duplicate of NVIDIA-G9G72YJS, severity cut from the Critical I'd filed at down to Medium, bounty $1,500.

Two months, two Informative closes, a PSIRT escalation, and a rewritten harness before it landed as a paid duplicate. I'm including that whole arc on purpose — the useful lesson here isn't the bug, it's that a real race condition against a real binary can still get closed twice before triage catches up to it, and the thing that turned it around wasn't a better payload, it was a more reliable reproduction and a statistical argument for why 11/100 wasn't noise.

L-1 — chmod hook follows symlinks

Class: a symlink-follow issue in a chmod-related container-toolkit hook. Filing date isn't precisely recorded in my own notes (somewhere between 2026–06–24, when I initially held off submitting it over reachability concerns, and its resolution). Closed 2026–07–03 as Duplicate of NVIDIA-0LEYP7D7, no bounty.

I'm keeping the root-cause detail on both at the class level above. They closed as duplicates of two existing internal reports — NVIDIA-G9G72YJS and NVIDIA-0LEYP7D7 — that I didn't file myself, so the deeper technical writeup on those specific root causes belongs to whoever did file them, not to me.

Environment & tooling — what I actually run this out of

This isn't web hunting, so the tooling list is different from a Burp+ffuf setup. One lab environment, a specific stack of utilities each step below actually depends on, and a hard rule about where any of this runs.

The VM

I never test against my host machine. A race harness or a fuzzer crash can leave a box in a bad state, and a container-escape PoC is, by definition, trying to break an isolation boundary — you don't want to find out the hard way that it worked a little too well. Everything here runs inside a disposable Linux VM:

brew install lima
limactl start --name=nvlab template://ubuntu-lts
limactl shell nvlab

Lima gets you a real Linux VM (not a container) on a Mac host in a couple of minutes, with the project directory auto-mounted. Good enough for this class of work when you don't have spare bare-metal — the TOCTOU class here lives in the mount/container-setup logic, not inside the GPU driver itself, so you don't need physical GPU hardware in the loop to audit or race it. If a later step needs real hardware, swap this for bare-metal or a cloud GPU instance and keep everything else the same.

Base toolchain, inside the VM:

sudo apt update
sudo apt install -y build-essential git curl ripgrep gdb strace ltrace \
    tmux python3-pip python3-venv pkg-config bmake libelf-dev
  • ripgrep (rg) — the resolve-then-use sweep in Step 2 lives or dies on how fast you can re-run it across a big C/Go tree.
  • gdb / strace / ltrace — for confirming, syscall by syscall, exactly where the check happens and where the use happens, before you build a race harness around a guess. strace -f -e trace=stat,lstat,open,mount,rename,symlink <target> is usually the first command I run against anything new.
  • tmux — a race harness needs at least two terminals running concurrently (the stalled victim process in one pane, the filesystem swap in another); tmux panes beat juggling separate SSH sessions.
  • bmake, libelf-dev — specific to building libnvidia-container itself; its Makefile expects BSD-make semantics in places, and WITH_LIBELF=yes (used in Step 1) needs the system libelf headers present.

Container-runtime pieces

To drive the toolkit end-to-end instead of only reading the source, you need a real container runtime underneath it, plus the packaged toolkit alongside your own from-source build so you can diff behavior directly rather than reading two commits side by side:

curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
    sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update && sudo apt install -y nvidia-container-toolkit

Race-harness pieces

python3 -m venv ~/venvs/fuse
source ~/venvs/fuse/bin/activate
pip install fusepy
sudo apt install -y fuse3 libfuse3-dev

fusepy is what the stall_fuse.py harness in Step 3 is built on. You also want unshare/nsenter (from util-linux, installed by default on Ubuntu) whenever you're reasoning about mount-namespace boundaries directly instead of only through the container runtime's own CLI.

Fuzzing / memory-safety pieces

sudo apt install -y clang lld llvm
git clone https://github.com/AFLplusplus/AFLplusplus
cd AFLplusplus && make distrib && sudo make install
  • clang — you want a reasonably current Clang for -fsanitize=address,undefined,fuzzersupport and AFL++'s afl-clang-fast instrumentation mode; the stock GCC on most distros won't give you libFuzzer at all.
  • AFL++ built via make distrib, not plain make — that target pulls in afl-clang-fast, qemu_mode, and the extra triage utilities (afl-tmin, afl-cmin) used in the crash-triage step later; the bare build skips them.
  • Past a handful of unique crashes, casr (cargo install casr, needs a Rust toolchain) auto-clusters crashes by root cause and gives a rough exploitability estimate — worth adding once you need to prioritize which crash to triage first instead of reading every ASan report by eye.

None of this is exotic. This is close to the full environment for both lanes of the work below — a disposable Linux VM, a normal build toolchain, strace/gdb to confirm timing before you automate it, FUSE to force a race deterministically, and AFL++/libFuzzer plus sanitizers for the memory-safety side. The research is what takes the time, not the setup.

The TOCTOU methodology, worked in full on a case that IS public

Since Wiz Research already published the complete technical breakdown of CVE-2024–0132, I'll use it as the worked example instead of my own private candidates. This is the exact shape of bug I was hunting for, and Wiz's writeup is detailed enough to teach the whole pattern honestly.

The bug, per Wiz's own disclosure: libnvidia-container (pre-1.17.4) resolves a set of library paths with find_library_paths(), which globs /usr/local/cuda/compat/lib*.so.* — that's the check. Later, mount_files() iterates that already-resolved list and mounts root + paths[i]into the container — that's the use. Between the two, an attacker who controls the container image's filesystem can substitute what those paths resolve to: first a directory shadows an expected library file on one mount pass, which changes what the next resolution step sees; then a symlink dropped at the next expected name — pointing at ../../../../../../ — gets walked by the kernel when the second mount pass reaches it. Read-only host root mounted into the container is still enough to reach docker.sock and go from there to full host takeover. Fixed in 1.17.4.

That's a textbook two-pass check-then-use gap, and it's the pattern I go looking for everywhere I work on this class of software.

Step 1 — pin the source, don't guess at a binary

Real commands from my own build, on a Lima VM (nvlab, Ubuntu aarch64, 4 vCPU / 6GB RAM / 20GB disk, gcc-15 + Go 1.26.4):

git -C libnvidia-container describe --tags
# v1.19.1
# pinned exact source commit for this audit pass:
# 0d1d7494ba63cbba3627c229486811f7ae48868f
make WITH_LIBELF=yes
# without this flag the vendored elftoolchain build fights modern GNU ld's
# -soname syntax; WITH_LIBELF=yes routes it through the system libelf instead
apt install libtirpc-dev
cp /usr/include/tirpc/rpc/*.h /usr/include/rpc/
make LDLIBS=-ltirpc
# Sun-RPC headers were stripped from modern glibc; libnvidia-container's
# build still expects them at /usr/include/rpc, so you either vendor them
# back in or link against libtirpc and copy the headers manually
nvidia-container-cli --version
# 1.19.1+20.g0d1d749   <- confirms you built what you think you built

That last line matters more than it looks — I burned real time on H-1's first Informative close arguing about exactly which version I'd tested, because I hadn't nailed down git describe output as evidence up front. Get the version string into your report before anyone asks for it.

Step 2 — sweep for the resolve-then-use idiom, mechanically

You're grepping for pairs: something that resolves a path (stat, lstat, realpath, readlink, a cached glob result) and, anywhere later in the same control flow, not necessarily the next line, something that acts on that same value in a privileged way (open, mount, chmod, chown, unlink, or the value getting handed to a subprocess/hook).

# find every resolve-site
rg -n '\b(stat|lstat|realpath|readlink|glob)\s*\(' --type=c
# then for each candidate variable, find every later use in the same file
rg -n '\b(open|mount|chmod|chown|unlink|symlink)\s*\(' --type=c

That's crude on its own — the real work is reading each pair by hand and asking "does anything reachable by an attacker sit between these two calls." But it's how you generate the candidate list before you spend real time on any single one. If the codebase has shipped a TOCTOU CVE before (and container runtimes usually have — see also runc's CVE-2019-5736), pull the fix commit first and grep every other function for the same idiom the fix only patched in one place. Vendor fixes are almost never as surgical as the report that prompted them.

Step 3 — stop relying on timing luck, force the race deterministically

A PoC that wins "eventually, in a loop" is weak evidence, and it's exactly the kind of thing that gets you a "doesn't reproduce" reply from a triager — which is what happened to me on 2026–07–04. The fix isn't a faster loop, it's removing timing from the equation entirely: stall the check syscall under your control, swap the filesystem state while it's stalled, then let it resume into the use step pointed somewhere else. FUSE is the standard way to do the stall. Here's the generic shape of that harness (illustrative, not tied to any specific target's code):

#!/usr/bin/env python3
# stall_fuse.py — a minimal FUSE stub that sleeps inside getattr() for one
# specific path, so a second process gets a guaranteed window to swap
# what that path resolves to before the victim's stat() call returns.
import os, time, errno
from fuse import FUSE, FuseOSError, Operations
STALL_TARGET = "/watched/target.so"
STALL_SECONDS = 2.0
class StallFS(Operations):
    def getattr(self, path, fh=None):
        if path == STALL_TARGET:
            time.sleep(STALL_SECONDS)   # <-- the deterministic window
        st = os.lstat("." + path)
        return {k: getattr(st, k) for k in (
            'st_mode','st_ino','st_dev','st_nlink','st_uid',
            'st_gid','st_size','st_atime','st_mtime','st_ctime')}
    # readdir/open/read passthrough omitted for brevity
if __name__ == '__main__':
    FUSE(StallFS(), '/mnt/stallfs', foreground=True, allow_other=True)
# terminal 1: mount the stall filesystem, trigger the victim flow against it
python3 stall_fuse.py &
run-victim-tool --target /mnt/stallfs/watched/target.so
# terminal 2: during the STALL_SECONDS window, win the race every time
ln -sfn /evil/redirect /mnt/stallfs/watched/target.so

Every run, 100% of the time, instead of 11/100. That's the difference between "here's a theory" and "here's a reproduction," and it's the single change that would have saved three weeks of back-and-forth on H-1.

Step 4 — build the reachability matrix before you get excited

Before treating anything as a finding: what privilege level does the victim process run at, does the vulnerable path require a non-default flag or config, does it fire on every invocation or only some admin subcommand, and — critically for a container runtime — is there even an attacker-controllable filesystem in play at the moment the check happens (a container image you control is a very different precondition than a local user racing another local user's files). This is where most candidates die, and it's supposed to be where most candidates die.

Memory-safety testing — the methodology, without the findings

Compiler-adjacent CLI tooling — linkers, demanglers, anything that parses attacker-shaped symbol or object data — is a classic, under-tested memory-corruption surface, because everyone treats it as "just a build tool" when it often runs with real trust and consumes data an attacker can shape (a package, a precompiled object, a crafted input file).

Harness shape (libFuzzer, generic target signature):

// fuzz_harness.c
#include <stdint.h>
#include <stddef.h>
// LLVMFuzzerTestOneInput is the entry point libFuzzer drives directly —
// isolate just the parsing/decoding function so you pay no process-
// startup cost per iteration.
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    if (size < 4) return 0;
    parse_target_format(data, size);   // the function under test
    return 0;
}

Build with sanitizers on — this is non-negotiable, a crash without ASan just tells you it crashed, a crash with ASan tells you what kind of memory violation it was and roughly where:

clang -fsanitize=address,undefined,fuzzer -g -O1 \
    -o fuzz_harness fuzz_harness.c target_parser.c
mkdir -p corpus
# seed with REAL samples of the format, not empty/random input —
# coverage-guided fuzzing converges far faster from realistic seeds
cp /usr/lib/real-sample-inputs/* corpus/
./fuzz_harness -max_len=4096 -jobs=4 corpus/

Or the AFL++ persistent-mode equivalent when the target's build system fits that better:

AFL_USE_ASAN=1 afl-clang-fast -o target_afl target_parser.c
afl-fuzz -i corpus/ -o findings/ -- ./target_afl @@

Triage, once it finds something:

# minimize the crashing input first — you want the smallest repro
afl-tmin -i findings/crashes/id:000001 -o min_crash -- ./target_afl @@
# then run it once under ASan directly and read the report
./fuzz_harness min_crash

From the ASan report, the question that actually matters for severity: is this an out-of-bounds read(often "just" info disclosure or a crash) or a write (much closer to exploitable)? Is the out-of-bounds offset attacker-influenced, or is it a fixed one-past-the-end that never moves regardless of input? Dedupe every unique crash by its actual faulting stack before you count anything, and reproduce each one manually with a hand-built minimal input outside the fuzzer before it goes anywhere near a report — fuzzer output is a lead, not evidence.

I ran this exact process — harness, sanitizers, corpus, triage — against a set of NVIDIA's compiler-toolchain CLI binaries. It surfaced results that are currently still with NVIDIA's team, unresolved. That's as far as this post goes on that thread: no tool names, no function names, no crash details, no repro inputs. If a fix ships and a disclosure window opens, that's a different post.

What actually mattered

Looking back at the H-1 timeline, the thing that moved it from a second Informative close to a paid duplicate wasn't a sharper payload — it was making the race deterministic instead of probabilistic, and having a version-pinned build with git describe output ready before anyone asked. On the memory-safety side, the thing that separates a real finding from fuzzer noise is exactly the same discipline: reproduce it by hand, outside the tool that found it, before you believe it yourself.

Reference: Wiz Research — NVIDIA AI Vulnerability: Deep Dive into CVE-2024–0132

None