Post cover image

September 24, 2026

The Bug That Reads and Writes: Reverse-Engineering vCenter’s DCE/RPC Heap Overflow (CVE-2024–38812)

A network-only exploitation study of VMware vCenter’s directory service; from Ghidra decompilation to a build-constant that turns a…

By Anthony Cihan

12 min read

A network-only exploitation study of VMware vCenter's directory service; from Ghidra decompilation to a build-constant that turns a "reliable DoS" into reliable pre-auth RCE.

TL;DR

  • CVE-2024–38812 is a CVSS 9.8 pre-auth heap overflow in libdcerpc.so, the DCE/RPC runtime shared by vCenter's vmdird, vmcad, and vmafdd daemons. It's on the CISA KEV list with confirmed in-the-wild exploitation, but at the time of this research no public no-login RCE proof-of-concept existed, and no writeup had fully disassembled why it works.
  • The root cause is a single unvalidated NDR conformance field (lower) that the array interpreter multiplies straight into a memcpy destination pointer. It yields a controlled-offset, controlled-size, controlled-data relative heap write during [in] unmarshalling — pre-authentication.
  • The finding I'm proudest of: the same vulnerable function runs in the marshal (send) path too. The unbounded lower shift, mirror-imaged, is an arbitrary relative read into the RPC response — an ASLR-defeat leak living inside the "write" CVE that every public analysis missed. I proved it live by leaking libdcerpc's ELF header onto the wire.
  • The most instructive part of the project was being wrong in public and correcting it. A carefully-run experiment concluded "unauthenticated blind impact is a reliable DoS, not RCE." That conclusion was true for the way I was firing. Recognising that the exploitation displacement is a build-constant — derivable offline, not a per-target secret — inverted the result to reliable network-only RCE.
  • Defensively, the denial-of-service failure mode is disk-driven and self-recoverable (core-dump partition exhaustion), which gives blue teams a clean, high-fidelity detection and mitigation story.

1. Why vCenter's RPC stack is worth the effort

vCenter Server is the control plane for a VMware estate. Compromise it and you're one step from the managed ESXi hosts, which is exactly why the 2024 DCE/RPC vulnerabilities became a ransomware favorite. Three network CVEs landed in two advisories:

| CVE | VMSA | Class | Mechanism | CVSS |
|-----|------|-------|-----------|------|
| CVE-2024-37079 | 2024-0012 | A | DCE/RPC association **auth-trailer integer underflow** → heap overflow | 9.8 |
| CVE-2024-37080 | 2024-0012 | A | Same class, `ALTER_CONTEXT` sibling path | 9.8 |
| CVE-2024-38812 | 2024-0019 | B | NDR **conformant/varying-array** pointer arithmetic → relative heap write | 9.8 |

All three are unauthenticated and reachable by binding a valid interface UUID on one of three TCP endpoints and sending a crafted PDU:

| Port | Service | Daemon |
|------|---------|--------|
| 2012 | VMware Directory Service | `vmdird` |
| 2014 | VMware Certificate Authority | `vmcad` |
| 2020 | VMware Authentication Framework | `vmafdd` |

The advisories rate all three 9.8. What the advisories don't tell you is how much daylight there is between "9.8 pre-auth RCE" as a primitive and "9.8 pre-auth RCE" as something an unauthenticated attacker can reliably reach with no leak, no login, and no oracle. Making that determination was the whole project.

The single biggest advantage: this code is open source

libdcerpc.so is not bespoke VMware code. It descends from Likewise Open (later PBIS / BeyondTrust AD Bridge), whose DCE/RPC runtime is a fork of the OSF DCE 1.1 reference implementation. The NDR stub interpreter, the rpc_ss_ndr_* family where the bug lives, is therefore readable C on GitHub. Every function Ghidra decompiled could be mapped back to ndrui.c, cnsassm.c, and friends. That collapses the reversing effort from "figure out this blob" to "confirm this known code against this specific build." If you take one methodology lesson from this article: identify your target's upstream lineage before you start staring at decompiler output.

2. A 90-second NDR primer

Connection-oriented DCE/RPC over TCP starts every PDU with a common 16-byte header (version, packet type, flags, data representation, fragment length, auth length, call id). To reach the two bug classes you only need two body types.

BIND body, where Class A lives, carries a presentation-context list:

u16 max_xmit_frag
u16 max_recv_frag
u32 assoc_group_id
u8  n_context_elem     ← the count Class A abuses
...
p_cont_elem_t[n_context_elem]

REQUEST stub, where Class B lives, carries NDR-marshalled parameters. The relevant NDR concept is the conformant varying array. A variable-length array is serialized with metadata before its elements:

  • a conformance header, the maximum element count (max_count, call it Z);
  • a variance header, an offset (lower) and an actual count.

The interpreter uses these counts to size an allocation and to place elements. Controlling them is the entire game.

3. Class A: a clean crash that teaches restraint (CVE-2024-37079/37080)

Class A is the "fast win" and a good warm-up, so it's worth understanding even though it ends in a dead end for RCE.

The naive theory, "send a huge context list and overflow", is wrong. The negotiate routine has a size gate:

result_list_size = 24 * n + 4
if (result_list_size > large_frag_size - base_header):   # 4096-byte frag
    return 0x1beef                                        # rejected

That caps you at n = 169 contexts. The real bug is in the auth trailer path that runs afterward. With n = 169, the BIND_ACK response buffer is filled to ~4092 of 4096 bytes, and then rpc__cn_assoc_process_auth_tlr computes the remaining space for an NTLMSSP challenge:

auth_len = large_frag_size - (header_size + 8);   // 4096 - 4100 = 0xFFFFFFFC

An integer underflow. That 0xFFFFFFFC (~4 GB) sails through the NTLMSSP formatter's bounds check:

if (challenge_len <= *auth_value_len) {              // 166 <= 0xFFFFFFFC → passes
    memcpy(auth_value, challenge, challenge_len);      // ~166 bytes written past the 4096 buffer
}

producing a ~170-byte heap overflow from a single ~7500-byte packet, no grooming required. Reproducible crash of vmcad on every attempt.

Why Class A dies here:

I pushed hard on weaponising it and reached a firm negative, which is itself a result worth publishing:

  • The overflow content is server-generated, not attacker-controlled: The memcpy source is the NTLMSSP Type-2 Challenge: a fixed signature, a random 8-byte server nonce, and server config strings. The only attacker-influenced field (the masked Negotiate flags) lands inside the fragbuf's own trailing slack and never reaches the neighbouring chunk. You smash the adjacent object with a random nonce; a crash, never a controllable overlap.
  • No info-leak either: The underflow inflates the response length, so the reply does transmit ~170 bytes past the buffer — but the inflated length equals the write extent, so you read back your own writes, not un-written adjacent heap.

Verdict: Class A is a clean, reliable, single-packet pre-auth DoS and nothing more on this build. Knowing exactly why a promising avenue is dead is what lets you stop spending time on it.

4.1 Class B: the real target (CVE-2024-38812)

The NDR interpreter reaches a conformant-varying array (type code 0x17) and walks this chain (source names from ndrui.c):

rpc_ss_ndr_unmar_interp  (case 0x17)
  → unmar_Z_values         reads max_count Z from the wire
  → unmar_range_list       reads offset A + actual_count B; sets lower=A, upper=A+B  (NO validation)
  → alloc_storage          malloc sized from Z
  → u_var_or_open_arr      checks (upper - lower) <= Z  ← partial, and useless
  → contiguous_elt         ← THE BUG
  → unmar_by_copying       memcpy(dest, wire_data, count * elem_size)

The vulnerable arithmetic in rpc_ss_ndr_contiguous_elt is, in essence:

array_addr += element_size * range_list->lower;   // lower is straight off the wire, unbounded

The only guard anywhere on the path checks (upper - lower) <= Z. An attacker simply sets lower to a large displacement, keeps upper - lower small (so it passes), and makes Z large enough to pass its own check. lower is never itself bounded.

4.2 The primitive

Set the four wire fields and you get a fully-parameterised relative heap write:

| Wire field | Control | Effect |
|---|---|---|
| `max_count` (Z) | full uint32 | `malloc(Z * elem_size)`, you size the allocation (heap shaping) |
| offset (A → `lower`) | full uint32, unchecked | `dest = alloc_base + A * elem_size`, you pick the displacement |
| actual_count (B) | uint32, must be ≤ Z | `memcpy` length = `B * elem_size`, you pick the write size |
| element data | B·elem_size bytes | `memcpy` source, you pick the bytes |

For a wchar_t array, elem_size = 2. This is ASLR-agnostic for nearby heap objects: the displacement is relative to the allocation, so you don't need to know absolute addresses to corrupt a neighbour.

I confirmed it under GDB exactly as the decompilation predicted. Firing A = 0x41414141 displaced the memcpy destination by 0x82828282 (~2 GB) into unmapped memory:

Thread 17 "vmafdd" received signal SIGSEGV
0x…  memcpy+31: movups %xmm0,(%rdi)
  rdi = alloc_base + 0x82828282     ← faulting destination (= A * 2)
  rsi = <wire data from the socket> ← fully controlled source
  rdx = 0x20                        ← write size (B * 2)
#1 rpc_ss_ndr_unmar_by_copying     ndrui.c:655
#2 rpc_ss_ndr_u_var_or_open_arr    ndrui.c:1197
...
#5 vmafdd opnum 3 stub
#6 rpc__cn_call_executor

The Ghidra offsets matched the live return address to the byte. Static RE and runtime were now one artifact.

4.3 From write to code execution

Two working RCE variants, both against the fixed-address advantage that the daemons are non-PIE (ET_EXEC) and import system(), so system@plt sits at a constant address (e.g. 0x41AA80 in vmdird):

  • GOT overwrite: request 1 overwrites GOT[free] with system(); request 2 stashes a shell command in the array that NDR cleanup will free(), turning free(array) into system(cmd). Confirmed uid=0 on vmafdd.
  • Receive-fragbuf callback hijack (the cleaner path): partial-PDU grooming parks a field of 0x1050 receive-fragbufs in front of the NDR allocation. The overflow overwrites a co-located fragbuf's dealloc callback pointer (+0x18) with system@plt, and writes the command string at fragbuf+0x00, which is the RDI argument at call time. Closing the groom connections frees the fragbuf → (*(fragbuf+0x18))(fragbuf) == system(cmd).

I verified the callback-hijack path purely over the network with a connect-back: a ≤23-byte curl HOST:PORT payload, with the target's own inbound HTTP request to my listener as self-authenticating proof of execution. On a 4-CPU appliance it landed 4/4 when the displacement was supplied.

That parenthetical, "when the displacement was supplied", is where things get interesting.

5. The novel bit: the CVE reads as well as it writes

Every public analysis of CVE-2024-38812 describes the write. Here is the part nobody had documented:

rpc_ss_ndr_contiguous_elt is shared by the marshal (send) interpreter too.

On the outbound path, rpc_ss_ndr_m_var_or_open_arr → contiguous_elt → rpc_ss_ndr_marsh_by_copying performs:

memcpy(response_buffer, array_base + element_size * lower, (upper - lower) * element_size);

Same unbounded lower, now applied to the source of a copy that goes into the response. And the marshal path has no bound on lower at all, not even the toothless upper - lower ≤ Z check the unmarshal side has. That is an arbitrary relative read that exfiltrates memory onto the wire. Same root cause, opposite direction, the ASLR-defeat leak that people assumed had to come from some separate bug is sitting inside the RCE bug itself.

I proved it live and non-destructively. With range_list->lower set so contiguous_elt pointed at libdcerpc's load base, a benign pre-auth GetHeartbeatStatus call (vmafdd opnum 39, which marshals wchar service-name strings through exactly this interpreter) returned a response that carried libdcerpc's ELF header — 7f 45 4c 46 … straight onto the wire. Module base disclosed; ASLR defeated. I then demonstrated two independent corruption→leak targets (the executor-arena range_list->lower, and a registry pszServiceName pointer) that make a real remote attacker reach the same read.

The remaining barrier for a turnkey network-only leak is heap co-location, not mechanism, the write and the object you want to read have to land in the same glibc arena, which is a function of concurrency and load. On a minimal 3-service idle lab that co-location is rare; on a continuously-loaded production appliance (more connections than the arena cap of 8 × ncpus, forcing threads to share arenas) it is exactly the condition. I measured a 16-connection burst producing 5 shared write marshal arenas, including the main-heap arena that holds the registry. So the leak is production-gated, not mechanism-gated, which is the reason these bugs manifest as full RCE against real fleets rather than against a stripped lab.

6. Being wrong well: the DoS→RCE reversal

This is the part of the project I'd want a junior researcher to read.

For the vmdird callback hijack, one value was treated as the hard, per-instance-random unknown: the forward displacement D0 from the NDR allocation to a co-located groom fragbuf. Believing that value required a pre-auth oracle, I built a "machine-gunner" that sweeps the displacement space blind, and ran a disciplined experiment to measure it:

  • 6 trials, fresh daemon each, cores enabled, pure-network, auto-classified → RCE 0/6, DoS 6/6.

Two earlier runs had converged to RCE (~235 salvos), but crash-light retests swept past that point (272,400 salvos) with zero hits, and a gold-standard snapshot-revert control also DoS'd. So the early "successes" were outliers. I wrote the conclusion: "unauthenticated blind impact is a reliable DoS, not autonomous RCE". I even recorded the DoS failure mode (see §7). That was a correct conclusion… about blind wide-candidate sweeping.

Then something changed. Re-measured with a fixed, simple groom recipe (120 partial-PDU fragbufs + 12 warm-up requests), the displacement isn't a per-instance secret at all, it's a build-constant of the recipe: lower = 0x820. Co-location was 30/30. A single hardcoded 0x820 shot popped a freshly-restarted vmdird 6 of 8 times (~75%) across independent instances and ASLR layouts, with every miss benign (the daemon survived 8/8). Because the offset is relative (NDR→fragbuf) and recipe-fixed, ASLR only shifts the absolute arena base, so an attacker derives 0x820 offline from the same shipping build, no login, no oracle, no per-target recon.

That inverts the headline. The 0/6 DoS was the cost of firing out-of-region candidates you don't need. Fire the one in-region build-constant instead:

  • Reliable pre-auth, network-only RCE: no leak, no oracle, no login. ~75% first-shot, benign misses, so 1–3 paced shots exceed 98%. uid=9899(vmdird) execution confirmed on the wire.

The wide-blind DoS remains the degenerate, mis-tuned case; the correct exploit simply doesn't fire out-of-region. The lesson isn't "I was wrong", it's that the same primitive can be a DoS or an RCE depending entirely on one modelling assumption, and rigorous experiments are only as good as the assumption they encode.

The main caveat with the result: every reliability number above is measured against freshly-restarted lab appliances. The build-constant is recipe-relative and ASLR-invariant (a strong argument that it generalizes), but a continuously-loaded production appliance's baseline arena state is a variable I did not test and which could shift the recipe's displacement or first-shot rate.

7. The safe reset: turning a liability into a tool

A subtle engineering detail makes the PoC dependable. The build-constant displacement is only valid against a fresh arena. Hammer one target and server-side CLOSE-WAIT groom buildup slowly drifts the arena, dropping the hit rate. The fix is counter-intuitive: deliberately crash the service to reset it.

An out-of-region relative write faults vmdird; the Likewise supervisor (lwsmd) auto-restarts a fresh instance in ~1.5–3 s, in which lower = 0x820 is valid again. The critical constraint, learned the hard way, is that this must fire exactly once:

CRITICAL: never re-fire. lwsmd auto-restarts vmdir in ~1.5-3s, so a re-fire would crash the restarted instance and the 2nd consecutive crash trips lwsmd's autorestart throttle → hard stop → and the service stays dead even though the caller only asked for one reset.

So the exploit's own crash primitive doubles as a heap-state reset that restores exploitation reliability, but respecting the supervisor's throttle keeps it a single clean reset rather than a self-inflicted denial of service.

8. Defensive takeaways (the part your blue team wants)

This research is only worth publishing if defenders get more out of it than attackers. The good news: the failure modes are loud and the mitigations are cheap.

  • The DoS is disk-driven and self-recoverable. Every crash writes an up-to-2 GB core to /storage/core. A sustained sweep fills the partition, at which point a crash can no longer restart cleanly and the service enters a crash-loop meltdown with all ports down. This is partition exhaustion, not persistent data corruption, clear the cores and the service restarts. Mitigation: bound/rotate core dumps for the RPC daemons, or give them a dedicated size-capped core partition.
  • High-fidelity detection: rapid growth of /storage/core/core.{vmdird,vmcad,vmafdd}.* and repeated watchdog/lwsmd respawns in /var/log/vmware/<daemon>/ are near-unambiguous indicators of exploitation attempts against these endpoints. Alert on both.
  • A positive hardening observation: on this build, liblber NUL-terminates decoded LDAP values, which closes the classic errorMessage-%s-over-read leak class on port 389. Worth preserving in future builds. I confirmed three separate external leak surfaces were negative precisely because of details like this.
  • Patch, obviously. The U3p→U3t diff isolates the Class B fix to an integer-overflow guard in rpc_ss_ndr_allocation_size (the dimension-product), lower-bound sign checks in contiguous_elt, and bounds in u_var_or_open_arr, ~20 of 1485 functions changed. That diff corroborates the array-conformance root cause.

Segmenting management-plane RPC (2012/2014/2020) away from anything an attacker can reach remains the single highest-value control: these are unauthenticated endpoints doing memory-unsafe parsing.

9. Methodology recap

If you're reproducing work like this, the order that worked best:

  1. Establish lineage. Trace the binary to its open-source ancestor (Likewise Open / OSF DCE 1.1) and read the real C.
  2. Reachability first. Confirm every service accepts a BIND with the right interface UUID before touching bugs.
  3. Static RE ↔ source ↔ runtime. Map Ghidra offsets to source line numbers, then confirm against a live GDB back trace. When the return address matches to the byte, you know your model is right.
  4. Chase primitives, then reliability, separately. "Can I write?" and "can I reliably write in this spot with no oracle?" are different questions with different experiments.
  5. Run adversarial experiments against your own conclusions. The snapshot-revert control and the crash-light retests are what caught the outlier "successes."
  6. Model assumptions are important. The entire DoS↔RCE flip was one assumption (D0 is per-instance) being replaced by a better one (D0 is a build-constant). Test and confirm your assumptions.
  7. Publish the corrections. The reversal is more useful to the next researcher than the polished result.

10. Closing

CVE-2024-38812 is a small bug, one missing bound on one wire field, with a large blast radius. Reversing it end to end surfaced three things the public record didn't have: that the same function is an arbitrary read as well as a write (a self-contained ASLR defeat), that the "per-instance random" exploitation displacement is actually a build-constant derivable offline, and that the unauthenticated impact flips between DoS and RCE on a single modelling assumption. The defensive corollary is encouraging: the noisiest attacker behaviors are exactly the ones that are cheapest to detect and bound.

Patch to U3t. Watch your core partition. And when your careful experiment tells you something is impossible, check whether it's the bug that's the limit or your assumption about it.