There's a recurring tension that shows up in most of the projects I work on. It's easy to mistake for unclear requirements or bad engineering, but that's not quite right. It comes from something deeper — a mismatch between what teams expect to define upfront and what can only be discovered through iteration.

This tends to happen in domains that are still being actively understood. Not just solved, but shaped. The kind of work where you're not only answering questions but figuring out which questions even matter. That includes most data science, most knowledge infrastructure, and a surprising amount of what companies call analytics, modeling, or engineering.

The tension shows up like this: teams want the "right" system built immediately, but they can't provide stable requirements. Not because they're withholding information, but because the information doesn't exist yet. They don't yet know what they need, and they can't know until they start interacting with the process of building tools and watching its surface come into focus.

What follows from this is a pattern of misalignment. The shape of the data gets tied too early to the questions being asked. The pipeline gets built around assumptions that haven't been tested. The interface starts to reflect decisions that were never meant to be permanent. And when these concerns collapse into each other, the tools become brittle, the understanding becomes shallow, and the practices become unreliable.

At its core, this isn't a technical problem. It's a framing problem. Teams treat evolving knowledge as if it were fixed, and treat early decisions as if they were final. The real hard work is recognizing that some parts of the system are meant to stabilize, while others are meant to stay fluid — and building an architecture that can tell the difference.

I've been thinking about this as three layers: the System, the Framework, and the Views. Not as rigid components, but as a way of keeping different kinds of decisions from collapsing into each other. Because once they collapse, you lose the ability to evolve any one part without destabilizing the rest.

Many disciplines already do this within their best practices implicitly. But it's been valuable for me to spell it out and I'm hoping it is for others as well.

Three Layers

The diagram above is the whole argument in one frame. The relationship between the three layers is not a hierarchy and not a pipeline — it's a loop, with each layer informing the others and the real work happening at the boundaries.

The System is the raw territory. The actual market, the actual organization, the actual ecosystem you're trying to understand. Everything that exists, whether or not you can see it. You never work directly with the system. You work with what you've captured of it, and the capture is always partial.

The Framework is the ontology — the layer meant to stabilize. It is the commitment your team makes about what exists in the world you're building for: what counts as a thing, what properties things have, what relates to what. It can be thought of as "the mental model of your tools". When a product team decides that "user" is a row in a table with an email address and a timestamp, they've made an ontological commitment. These look like technical decisions because they're expressed in technical language but they are fundamental claims about what we think exists in the world and what the team thinks matters about it.

The View is the projection — the layer meant to stay fluid. It is the deliberate act of hiding most of the framework so that a particular question can be answered clearly. A view is not a summary. It's a semantic commitment about what a particular person, asking a particular question, should see and should not see. Views change often, because the questions change often. The framework beneath them is supposed to hold still.

The discipline is to hold the three layers as separate concerns even though they're constantly talking to each other. A view people actually use teaches you something about what the framework was missing. A framework you actually live with teaches you something about what the system contains that you weren't capturing. The loop runs in both directions, all the time. But if you collapse the layers — if the dashboard is welded to the schema, if the schema is welded to one specific question — you lose the ability to revise any one of them without breaking the others.

Most of the failures I've watched in knowledge infrastructure come down to this collapse. The symptoms look technical. The cause is structural.

Different People, Different Projections

One reason the layers have to stay separate is that different people need radically different things from the same underlying reality.

Think about who is actually in the room. 1. Domain Expert: the executive, the sales lead, the warehouse operator who all have lived inside the system long enough to have strong intuitions about how it behaves. 2. The Infrastructure Architect: There's the person who translates those intuitions into a formal structure — the schema designer, the engineer, the architect, whoever turns a mental model into something a computer can reason about. 3. The Analyst: There's the person who uses that structure to describe specific cases — the modeler, the analyst, the person writing queries inside the framework they were handed. 4. The Decision Maker: And there's the person who consumes the outputs — the executive, the operator, the policymaker, someone who needs an answer they can act on and doesn't care how it was produced.

Each of these people needs something different from the same system. If you try to serve all of them with a single view, you fail all of them. The domain expert feels the dashboard has been dumbed down and naive. The architect feels it's been over-specified. The analyst finds it clunky and not flexible enough to answer their questions. The decision maker finds it unreadable.

The three-layer architecture is what lets you serve them without pretending their needs are the same. The domain expert and the translator negotiate at the Framework layer. The modeler works across Framework and View. The consumer lives entirely in the View. Each can revise their own layer without forcing everyone else to rebuild.

If we don't enforce clean separations between their concerns, we cannot build reliable tools for any of them

What a View Hides

This is the first thing I want people to understand about views: what a view hides is more important than what it shows. A subway map hides almost everything about a city — distances, streets, neighborhoods, hills — and keeps only the topology of connections. It's wrong in almost every quantitative sense, and exactly right for the question how do I get from here to there by train. A balance sheet hides almost everything about a business and keeps only financial positions at a moment in time. Each is a deliberate lie, and each is useful because of what it refuses to show.

The name I've been using recently for this concept is structured lossyness. Every useful representation and view is lossy on purpose, and the structure of the loss is the craft. When I design a view well, I'm not summarizing the framework — I'm making deliberate decisions about what to suppress so that what remains can be seen clearly.

There is no neutral view. The only views that work are the ones that take a position ; that commit to a question, a reader, and a scale at which the answer is meant to be valid.

Where the Misalignment Actually Lives

Every individual on a team carries a working mental model of the domain. It shapes what they choose to capture, what they ignore, and what assumptions they treat as safe defaults. You can think of this as an implicit framework — a way of deciding what is knowable, what matters, and how to act on it.

In practice, these implicit frameworks aren't written down as clean ontologies. They show up in fragments: an Excel macro someone trusts, a dashboard everyone references, a spreadsheet that's become "source of truth" by habit.

The issue isn't that these frameworks exist. They have to. The issue is that they remain implicit. When that happens, coordination starts to break down in ways that feel operational but are actually conceptual. The same data gets entered multiple times, slightly differently each time. Metrics don't line up across teams. Generating a report becomes a reconciliation exercise. Delays creep in between when something happens and when anyone can see it in a number.

From an engineering lens, this shows up as schema drift, brittle pipelines, and business logic scattered across transformations. But underneath that, it's simpler: different people are working with different ideas of what the system is.

Some friction here is useful. Redundancy and delay can act as a kind of quality assurance check, forcing validation before things get locked in. But a lot of it isn't. A lot of it comes from not having a shared way to describe what entities exist, what relationships matter, and what assumptions are being made.

Every team has a framework like this, whether they've named it or not. Teams that don't externalize it don't avoid complexity — they just push it into coordination overhead, where it becomes harder to see and harder to fix.

Tools as Epistemic Instruments

We usually think of tools as things that automate work. But when you look closely, they do something more fundamental.

Every tool makes decisions about the domain — what counts as a valid entity, what gets tracked and what gets ignored, what relationships are first-class, what assumptions are baked in. These are ontological choices, whether we call them that or not.

And you don't get to avoid making them. You cannot ship a vague concept. You cannot compile an ambiguity. The moment you try to build something real, ambiguity stops being an option.

That's why building tools often feels like it "teaches" you about the domain. It's not magic — it's constraint. You're forced to turn vague ideas into concrete structures. From an engineering point of view, this is the shift from informal reasoning to schema design, from loose assumptions to typed interfaces, from ad hoc workflows to reproducible pipelines.

The code isn't just execution. It's where these decisions get fixed into something others can see, use, and question.

And once that happens, misalignments that were previously invisible start to surface.

What's changed recently is the cost of doing this. With better tooling and AI, it's cheap to spin up schemas, fast to build pipelines, easy to generate interfaces. The bottleneck is no longer can we build this? It's do we actually know what we're building? Can we agree on what entities exist? Can we define them clearly enough to encode? Can we make our assumptions visible and revisable?

A tool, in this framing, isn't an automation layer. It's a coordination surface — a place where a team's understanding of the domain gets made concrete enough to work with. AI lowers the cost of building, but more importantly, it lowers the cost of externalizing thought.

And as building becomes ubiquitous, the gap that used to be hidden by the difficulty of engineering starts to show. When everyone can ship, the difference between good engineering and good ontology becomes even more apparent. The teams that used to win on execution are losing ground to the teams that win on clarity, because execution is no longer the hard part.

A companion to an earlier essay of mine, The Shadow of Compression, on why every act of understanding is an act of compression, and every compression is a bet.