June 10, 2026

From Synthea to ICD-10: Building a Medical Coding ML Pipeline on GCP

How a 16-point F1 jump came from fixing the data, not the model

By Raju Subba

8 min read

How a 16-point F1 jump came from fixing the data, not the model

The hook

Every time a patient is discharged from a U.S. hospital, someone has to turn what happened to them into codes. The pneumonia, the chest X-ray, the IV antibiotics — each becomes an ICD-10 code, and those codes are what insurers pay against. Get them right, the hospital is reimbursed correctly. Get them wrong, you get denied claims, audits, and revenue that quietly evaporates.

This work — reading a clinical note and assigning the right diagnosis codes — is called medical coding, and it's a real industry. The U.S. medical coding market is worth roughly $24 billion in 2025, sitting inside a larger problem: administrative spending is up to a quarter of all U.S. health expenditures, with billing and coding among the biggest drivers. Most of it is still done by human coders, one note at a time.

If you squint, it's a textbook ML problem: free text in, a structured label set out. Supervised, multi-label, and abundant. So why isn't it solved?

I set out to explore that by building an end-to-end pipeline — synthetic records in BigQuery, discharge notes generated by Gemini, a classifier on top, and a live API on Cloud Run. Along the way the model went from a micro F1 of 0.821 to 0.981. And here's the part worth your attention:

That jump had almost nothing to do with the model. No transformer, no heroic tuning. The gain came entirely from understanding what the data was doing, and fixing the problem at the data layer instead of the model layer. That's the story.

The synthetic data strategy

Every healthcare ML project hits the same wall first: you can't just download real discharge notes. Clinical text is among the most protected data there is — access means an IRB submission, a data-use agreement, de-identification review, and usually an institutional affiliation. Months before your first line of model code. And you can't put any of it in a public repo, so nobody can reproduce your work.

So I went synthetic, in two stages.

Synthea for the medical reality. Synthea is an open-source patient simulator that generates fully synthetic but clinically coherent records, and every condition it emits comes already tagged with a real ICD-10 code. I generated 1,091 patients across 10,755 encounters into BigQuery. That handed me the hardest part of a coding dataset for free: trustworthy labels. The codes aren't a coder's noisy guess — they're ground truth by construction.

Gemini for the narrative. Synthea gives you structured data — tables of conditions and encounters — not the free-text note a coder reads. So for each encounter I fed its ICD-10 codes to Gemini 2.5 Flash on Vertex AI and had it write a plausible discharge note. Codes in, clinical prose out. The result looks like the real task — narrative text labeled with codes — with zero real patient data, safe for a public repo.

The payoff is exactly what a portfolio project wants: no IRB, fully reproducible, and a pipeline that exercises the whole workflow — generation, warehousing, a managed LLM, feature engineering, deployment.

One asterisk to hold onto, though: the notes are generated from the codes, so predicting codes from notes means reversing a transformation Gemini just performed — and TF-IDF is very good at catching the breadcrumbs it leaves. This synthetic task is easier than the real one. I'll account for exactly how much in the caveats.

Building the baseline

With the dataset in BigQuery, the modeling setup was deliberately boring. I pulled the encounter notes, vectorized them with TF-IDF — word 1–2grams plus character 3–5grams (char_wb) to catch both terminology and the fragments of medical morphology — and trained a OneVsRest wrapper around logistic regression, one binary classifier per ICD-10 code. No deep learning, no embeddings. The whole thing fits in a few lines of scikit-learn.

This first grouping — one note per patient, with all of that patient's codes attached — gave me 779 development notes across 94 codes. The first honest pass landed at 0.685 micro F1. After cleaning up the label space and tuning the vectorizer and regularization, it climbed to 0.821 ± 0.019 in cross-validation, holding at 0.826 on the test set.

For a linear bag-of-words model on clinical-style text, that's a respectable number — and it raised the obvious question: time to reach for a transformer?

I didn't, and the reasoning matters. Before assuming the model was the ceiling, I broke the score down code by code. The failures weren't spread evenly — they clustered hard on codes the model had barely seen in training. That's a data-support problem, not a representation problem. A BERT-based model gives you richer features, but richer features don't help a classifier that's only seen a code three times. Swapping architectures would have been treating the wrong layer of the stack.

So the transformer went on the backlog, and I turned to the failures themselves — which is where the project got interesting.

Diagnosing the failure

A single micro-F1 number hides everything interesting. So I dropped to the per-code level and asked a blunter question: which codes is this model actually getting right, and which is it failing on?

The answer was stark. Thirteen codes had an F1 of exactly 0.000 — the model never scored a single correct prediction on them. And they had one thing in common: every one had a support of 11 examples or fewer. Worse, ten of the thirteen had zero positive predictions at all. The model wasn't guessing wrong on these codes; it had quietly learned never to predict them. When a code shows up three times in training, the loss-minimizing move is to always say "no" — you'll be right the overwhelming majority of the time. The classifier had correctly optimized its way into being useless on the rare tail.

When I bucketed every code by how many training examples it had and averaged the F1 within each bucket, the relationship was impossible to miss:

Training examples per codeMean F11–50.26251+0.857

The model's skill on a code was almost entirely a function of how often it had seen it. That's the whole story of the 0.821 ceiling in one table.

And it reframes the transformer question completely. No architecture fixes three training examples. Swap logistic regression for BERT and the well-supported codes might tick up a little, but the thirteen dead codes stay dead — there's simply nothing to learn from. The bottleneck wasn't how the model represented the text. It was that the data starved half the label space.

Which pointed at the fix. If the problem is support, the solution isn't a better model — it's more examples per code. And the way to get them was sitting in how I'd grouped the notes in the first place.

Fixing it at the data layer

The original setup grouped notes by patient — one note carrying every code that person had ever accumulated. That sounds reasonable until you look at what it does to the data. A patient note bundled together a lifetime of conditions: 4.2 codes per note on average, and only 779 notes to go around. Lots of labels, few examples each, a starved tail. Exactly the support problem from the last section.

So I regrouped at the level the task actually happens: the encounter. One note per hospital visit, scoped to just the codes from that visit. The shift did two things at once. Each note got simpler and more realistic — 1.6 codes per note instead of 4.2, much closer to how a real discharge note reads. And because patients have multiple encounters, the dataset roughly doubled: 1,581 development notes instead of 779. More notes, fewer codes each, and crucially, more examples behind every individual code.

The tail codes felt it immediately. S35 (vascular injury) went from an F1 of 0.000 at support 4 to a perfect 1.000 at support 14 — same model, same features, same everything. The only thing that changed was that the code now appeared often enough to be learnable.

Across the board, micro F1 jumped from 0.821 to 0.972 ± 0.005 in cross-validation, landing at 0.981 on the test set:

DatasetDev notesCodesMicro F1 (CV)Test F1Patient-grouped779940.821 ± 0.0190.826Encounter-grouped1,581740.972 ± 0.0050.981

That's a 16-point gain with zero changes to the model. Same TF-IDF, same logistic regression. The entire improvement came from choosing a unit of analysis that gave each code enough support to be learned — and, not coincidentally, one that better matched the real-world task. The lesson I keep coming back to: when a model plateaus, the most valuable question isn't "what's a better model?" but "what is my data actually representing?"

Deployment

A model that lives in a notebook isn't an engineering project — it's a result. So the last step was putting it behind a real API and a real deploy pipeline.

The serving layer is a FastAPI app exposing a single /predict endpoint: POST a note, get back the predicted ICD-10 codes. It's wrapped in a Docker image and runs on Cloud Run, which suits this workload well — it scales to zero when idle (a portfolio API gets no traffic most of the time) and spins up on demand, so it costs essentially nothing to keep live. Calling it looks like this:

curl -X POST "https://medical-coding-ml-api-403853829362.us-central1.run.app/predict" \
  -H "Content-Type: application/json" \
  -d '{"text": "Patient presents with acute bronchitis, productive cough..."}'
{
  "predictions": [
    {"code": "J20", "description": "Acute bronchitis"}
  ]
}

The part I'm happiest with is the CI/CD. A single push to main triggers a GitHub Actions workflow that takes it the rest of the way to production:

  • Download the trained model artifacts from GCS
  • Build the Docker image (amd64)
  • Push to Artifact Registry
  • Deploy the new revision to Cloud Run

Start to live in about two minutes, no manual steps.

One decision worth calling out: the model artifacts are pulled from GCS during the workflow, not baked into the Docker image or committed to the repo. That keeps the image lean and the repo clean, and it means retraining and redeploying are decoupled — I can ship a new model by dropping it in a bucket, without touching the application code. Small choice, but it's the kind of thing that separates "I trained a model" from "I can operate one."

What this isn't

I'd rather you trust this project than be impressed by it, so here's the honest accounting of what a 0.981 F1 does and doesn't mean.

The synthetic notes are easy on purpose — and that flatters the model. This is the big one. Gemini writes each note from the code list, so the vocabulary that signals a diagnosis is reliably present in the text. TF-IDF then reads that vocabulary right back out. The model is, in effect, reversing a transformation Gemini just performed. Real clinical notes don't cooperate like that: they're full of abbreviations, negations ("no evidence of MI"), copy-pasted boilerplate, and findings described without ever naming the diagnosis. On real notes, this exact pipeline would score meaningfully lower. The number is real for the task I built; the task is easier than the real world.

Part of the 16-point jump was the task getting simpler. The encounter model predicts 1.6 codes per note versus 4.2 for the patient model. That's not only better support — it's a genuinely easier prediction problem. So the 0.821 → 0.981 gain is a mix of two things: better per-code support (the real, transferable insight) and a lighter task (a partial freebie). Both are true. I'd never claim the whole 16 points as pure modeling virtue.

Some codes are rare in a way no data strategy fixes. Regrouping helped codes that were under-sampled but present. It does nothing for codes that barely exist in the source population — F32 (major depressive disorder) appears in just 7 encounters in my entire Synthea set. You can't engineer support out of data that was never generated. In a real deployment, those tail codes are exactly where a model quietly fails and a human coder still earns their salary.

None of this undoes the core lesson — that the binding constraint was data support, not model architecture. But it does scope it. What I built is a clean demonstration of how to find and fix a data-layer bottleneck, on a task honest enough to show the effect clearly. It is not a system I'd point at a real patient's chart tomorrow.

What's next

The honest caveats double as a roadmap — each one points at the next experiment.

A transformer baseline, finally. I deferred BERT because the bottleneck was data support, not representation — but now that the well-supported codes are genuinely well-supported, the question is live again. A clinical transformer (ClinicalBERT or similar) is most likely to earn its keep exactly where TF-IDF is weakest: notes that imply a diagnosis without naming it. That's the experiment that tells me whether richer features buy anything once support is no longer the limit.

Real notes, via MIMIC-III. The single biggest leap would be moving off synthetic data onto MIMIC-III — real, de-identified ICU notes with real coding noise. It requires credentialed access and a data-use agreement, which is its own milestone, but it's the only way to find out how much of that 0.981 survives contact with abbreviations, negation, and copy-paste. I expect a sobering number. That's the point.

Code and full pipeline: github.com/rajusubbaprojects/medical-coding-ml