Chad Sanderson

The data that causes the worst failures rarely sits at such a boundary.

• It is the field two teams away that nobody thought to flag, feeding a model whose owner never knew it existed.
• What teams actually need is provenance — where a value came from, what logic shaped it, what depends on it — and that record is a property of the code that produced the data, recovered by reading the code and keeping the map current.
• Contracts then sit on top of the map for a few lines rather than being spread thin across thousands of boundaries.
• Sanderson frames this as a trust problem rather than a regulation problem, and points out that coding agents change code they do not fully understand, quietly rewriting provenance faster than anyone can reconstruct it after the fact.

Sanderson describes spending years putting contracts in front of banks and enterprises that wanted them everywhere, and still considering that work worthwhile for the handful of fields where everyone knows a break takes the business down. The failures that cost the most came from fields no one understood well enough to write a contract for.

A year before writing the contracts post, Sanderson would have told teams to start with data contracts. The new version of the Shift Left Data Manifesto explains what changed: provenance first, read from the code where data is born, with contracts moved to the top of that map.

Read original · Original 2

It may show that a metric came from a particular table, which is traceable without complex tooling, but it cannot answer what a value means, who produced it, or how it was produced, because those systems have no context on where the data came from or why it has the shape it does.

• That context has to come from the source systems — not the PostgreSQL databases that stored the data but the application code further upstream.
• Sanderson estimates teams get less than half of the information needed to make lineage useful beyond data teams.
• The obvious alternative, asking producers to label data or attach metadata at generation, runs into two problems: every producer in the company has to buy in, and the historical data and older systems responsible for the largest data movement still have to be accounted for.

A field named userId is just a string in a table, but in the source code it is visible as an extraction from a credit-score API response, which gives it immediate functional meaning.

• Version control also shows who wrote the logic that produces and modifies the data, and change impact can be inferred before a dashboard breaks.
• Sanderson says this approach was built at Gable, runs in production on millions of lines of code at large companies, and borrows techniques from security and QA to trace how variables flow without running the application.

Read original · Original 2 · Original 3

Data, in the technological sense, is the output of one system available for consumption by another: an API response, an event on a Kafka topic, a PostgreSQL record, a file in object storage, a queue message, an application log, a SQL query result.

• All follow the same pattern of producing information that others consume, trace across systems, and transform.
• The common split — anything upstream is a software problem, anything downstream is a data problem — breaks down because a Java change that causes an outage for an API consumer and a dbt change that breaks a dashboard are the same kind of failure.
• If that is right, then quality rules, lineage, governance, and contracts should apply across all applications of data, not just the final stage of the supply chain.
• Sanderson's conclusion is that software teams do not think about data problems day in and day out while data teams do, and that applying data engineering principles at the organization level could be one of the highest-return investments a business can make.

Sanderson lists the outputs — API responses, Kafka events, Postgres records, S3 files, queue messages, log entries, SQL results — to show they are one category, then uses the Java-versus-dbt comparison to argue data teams should own the biggest data problems company-wide rather than only what happens inside the warehouse.

Read original · Original 2

As systems grow, code changes accumulate, services multiply, execution paths become implicit, transformations scatter across the codebase, and the original developers leave.

• The system may still run, but teams no longer know how it works, what depends on what, where critical behavior is defined, or what a change will break.
• This is deliberately separated from tech debt: debt is an explicit trade-off that a team knows it will have to handle later, while entropy occurs whether or not scalable decisions are made, because it is about the accuracy of a team's understanding over time.
• Sanderson treats poor change management, long migration timelines, and slow bug triage as restatements of the same problem, and says AI both suffers from entropy and accelerates it when nothing keeps track of what AI changes in a codebase.

Sanderson compares the situation to asking a junior engineer to refactor a business-critical service without telling them that twenty other teams depend on it. AI needs the same context about what a codebase does across the business before it can safely modify anything.

Read original

It usually arrives from compliance or risk after an audit or regulatory inquiry.

• Answering it honestly means pulling the most senior engineers off their work and putting them on archaeology: tracing a single value backwards through services, reading git history, and interviewing whoever still remembers how the pipeline worked two years ago.
• Sanderson says this runs for weeks and burns hundreds of hours of expensive people, and the resulting map is already wrong when it is finished because the code kept changing while it was documented; a different field and a different examiner bring the same exercise back next quarter.
• The answer does not live in the warehouse or the catalog, the argument goes, because a regulated value is computed in application code, in a service, in a transformation that shipped three releases ago.
• Teams that stopped paying this tax captured field-level lineage from source code as part of continuous integration, so every release carries its inputs and transformation history and the next inquiry becomes a lookup rather than a project.

The cost Sanderson describes is specific: senior engineers tracing one value through multiple services, git history, and past colleagues, for weeks at a time, repeatedly, with each finished map stale on arrival.

Read original · Original 2

Models are strong on binary questions around common pattern recognition — does this function exist, does this query exist, does a relationship exist — because they are trained on a large corpus of what things are and what describes them.

• They are also strong in the messy middle, where a problem has some degree of subjectivity and several acceptable answers, as in greenfield coding where there is no correct approach, only trade-offs.
• They are weak on non-binary outcomes with many variables, such as explaining the architecture of a complex ecosystem, where the output looks correct but is never fully correct, and they are weak on objective questions with one definitive answer, where the more specific and business-relevant the question becomes, the more likely the model lacks the specificity to answer accurately and consistently.
• Sanderson also describes AI as lazy: when it meets a pattern it does not recognize, it assumes one it does understand for efficiency, so a system might be right ninety percent of the time with no way to tell which ten percent is wrong, which makes the output unusable.
• The conclusion is that models should handle subjective problems and deterministic tools should handle objective ones, with both feeding each other.

Driving is subjective, with no single right answer about how to move the wheel or handle conditions, which is why Sanderson calls AI an exceptional driver. Navigation is computational and does have a right answer — the most efficient path between two points — with an exponential number of combinations, which is why people use GPS.

Read original

Monitoring tells you what happened and contracts tell you what should happen, but quality depends on whose point of view you take: data can be correct from the producer's perspective and incorrect from the consumer's, and a contract was a way to bridge that gap rather than a guarantee.

• Independent agents creating and consuming data raise new questions — who decides what the data means, what happens when agents take dependencies on each other, how anyone knows when an agent is making an incorrect assumption, and how far off an answer is when it is wrong.
• Without a shared agreement on meaning, Sanderson says, data does not merely have no value, it has negative value, because being led to the wrong conclusion would have been better avoided.
• Contracts cannot solve this on their own, but visibility derived from the true source of the data, the code, can give both people and AI the context on how data should be used, where it goes, and what depends on it.

Sanderson asks what stops a cascade of failures that have nothing to do with common quality metrics and everything to do with meaning, once agents are producing and consuming data and taking dependencies on each other.

Read original · Original 2

It answers the question the rest of the work builds on: what replaced data contracts at the front of the queue?

Sanderson says that a year earlier they would have told teams to start with data contracts, and that they no longer lead with that.

• Contracts still do one real thing well: they take a boundary everyone already agrees is critical and turn it into an agreement enforced automatically on every change.
• The data that causes the worst failures is almost never that boundary.
• It is the field two teams away that nobody thought to flag, feeding a model whose owner never knew it existed.
• The post argues that what teams actually need is provenance — where a value came from, what logic shaped it, what depends on it — and that provenance is a property of the code that produced the data, read from that code and kept current as the code changes.
• Contracts then sit on top of that map for the few lines worth holding, rather than being spread thin across thousands of boundaries.
• He notes that banks and regulated industries feel the gap first because they are already being asked to prove where a number came from, but frames it as a trust story rather than a regulation story, and points to coding agents rewriting provenance faster than anyone can reconstruct it after the fact.

For brands planning a campaign

For a webinar on end-to-end lineage, this is the frame that carries the session: not that contracts were wrong, but that contracts answer a narrower question than lineage does. A presenter who spent years putting contracts into banks and has publicly changed position gives the talk a credible spine and an obvious point for debate.

Sanderson defines system entropy as the gradual loss of understanding, predictability, and control in a software system as it evolves.

• As systems grow, code changes accumulate, services multiply, execution paths become implicit, transformations scatter across the codebase, and original developers leave.
• The system may still run, but teams no longer know how it works, what depends on what, where critical behaviour is defined, or what a change will break.
• Sanderson separates this from tech debt: debt is an explicit trade-off that will need handling later, while entropy happens whether or not scalable decisions are made, because it is about the accuracy of a team's understanding over time.
• He argues that poor change management, long migration timelines, slow bug triage, and failed data and AI tool adoption are mostly restatements of the same problem, and that AI both suffers from entropy and accelerates it when nothing tracks what AI changes in a codebase.

For brands planning a campaign

Gives a webinar a problem the audience already feels without using a product category as the villain. Useful for platform and engineering leaders who do not buy 'data lineage' language but do recognise onboarding failures, long refactors, and dependency surprises.

It is the best single scene-setting example in the set.

The most expensive question in a regulated bank, Sanderson writes, is five words: where did this field come from?

• It usually arrives from compliance or risk after an audit or regulatory inquiry.
• Answering it honestly pulls the most senior engineers off their work and puts them on archaeology — tracing one value backwards through services, reading git history, and interviewing whoever still remembers how the pipeline worked two years earlier.
• Sanderson says this runs for weeks and burns hundreds of hours of expensive people, and the map is wrong as soon as it is finished because the code kept changing while it was documented.
• It repeats the next quarter with a different field.
• The answer does not live in the warehouse or the catalog, he argues, because a regulated value is computed in application code, in a service, in a transformation that shipped three releases ago.
• Teams that stopped paying this tax captured field-level lineage from source code as part of CI, so each release carries its inputs and transformation history and the next question becomes a lookup.

For brands planning a campaign

The strongest concrete example for a financial-services audience: it names the internal trigger (compliance or risk after an audit), the cost (weeks of senior engineering time), and the fix (lineage captured in the build). It works as an opening story before the wider lineage argument.

Sanderson argues that data lineage consistently fails to deliver because it captures less than half of the actual data journey.

• The typical picture is data leaving a black-box application, landing in a database, being modelled in a warehouse, and feeding dashboards and machine-learning models.
• That stretch, he says, is the cleanest and easiest part of the flow to explain, and focusing there loses ownership, meaning, and impact.
• Ownership is lost because the database administrator is not the owner; meaning is lost because semantic intent cannot be read from rows in a table; impact is lost because change management becomes reactive, with breakages discovered only after a dashboard fails.
• His claim is that data begins its journey in code, and that code holds the missing answers: version control shows who wrote the logic that produces and modifies the data, a field like userId is just a string in a table but visible as an extraction from a credit-score API response in code, changes to schema or logic can be detected before pipelines break, and the consumer graph can be made explicit.
• He says this is what Gable built and that it runs in production on millions of lines of code at large companies, borrowing techniques from security and QA to trace how variables flow without running the application.

For brands planning a campaign

Makes the beyond-the-warehouse thesis concrete and shows there is something to demonstrate, not just argue. The credit-score API example transfers directly to a slide and gives a webinar a visual: code-level flow rather than table-to-table diagrams.

Sanderson argues that data engineering organisations cripple their own influence by restricting their scope to the warehouse and lake.

• Data, in the technological sense, is any output of one system available for consumption by another: an API response, an event on a Kafka topic, a Postgres record, a file in object storage, a queue message, an application log, a SQL query result.
• All of those follow the same pattern of producing information that others consume, trace across systems, and transform.
• He rejects the split where anything upstream is a software problem and anything downstream is a data problem, pointing out that a Java change breaking an API consumer and a dbt change breaking a dashboard are the same kind of failure.
• If that is true, then data quality rules, lineage graphs, governance practices, and contracts should apply across all applications of data rather than the final stage of the supply chain.
• He argues software teams do not think about data problems daily while data teams do, and that applying data engineering principles at the organisation level could be one of the highest-return investments a business can make.

For brands planning a campaign

Supplies the org-design half of a webinar: the pitch to data leaders about owning upstream systems, not just the warehouse. Useful for a 'who should own this' segment that moves the conversation from tooling to remit.

It is the most direct available statement of his position on AI in production.

Sanderson splits production experience with LLMs into good and bad.

• Good: binary questions about common patterns — does this function exist, does this query exist, does a relationship exist — because models are trained on a large corpus of what things are and what describes them; and subjective work in the 'messy middle', such as greenfield coding where there is no correct answer, only trade-offs.
• Bad: non-binary outcomes with many variables, where the output looks correct but never fully is, and objective questions with a single right answer and no interpretation, where the more specific the question the more likely the model lacks the specificity to answer accurately and consistently.
• He also calls AI lazy, explaining that when it meets a pattern it does not recognise it assumes one it does understand for efficiency, so a system might be right 90% of the time with no way to tell which 10% is wrong, making the output unusable.
• His metaphor: AI is an exceptional driver because driving is subjective, but it is not exceptional at navigation, which is computational and has a right answer, which is why we use GPS.
• The conclusion is that LLMs should solve subjective problems and deterministic tools should solve objective ones, with both feeding each other.

For brands planning a campaign

Grounds the audience questions and the claim that AI-generated code needs deterministic lineage underneath it. The driver-versus-navigator metaphor is a ready-made slide for explaining where automation should and should not be trusted.

A newcomer gets the size of the gap here.

Sanderson defines data flow as how data moves through software systems before winding up downstream in a lake, warehouse, or analytics dashboard.

• Most teams do not understand theirs, which makes a whole class of questions impossible to answer without large human effort: what does this data mean, who created it, when was it changed, who consumes it, is it the actual source of truth or a derivation, how is it being transformed.
• None of those answers come from the data itself, because data carries no information about its origin.
• He then addresses the common suggestion that producers should label data or attach metadata at generation: every producer in the company has to buy into the process, and historical data plus the systems responsible for the largest and most essential data movement still have to be accounted for.
• He describes a mapped enterprise codebase where nodes are functions that ingest, transform, or egress data, and uses it to argue that the missing map is why so many data initiatives fail — preventing quality issues is hard when the source is multiple hops from the destination, migrations run long when dozens of connections are unknown, and PII cannot be traced when the full journey is undocumented.
• His estimate is that data teams operate with less than 50% of the information required to implement most data management and governance practices.

For brands planning a campaign

Establishes the size of the gap early in a webinar and handles the standard objection that better producer metadata will fix lineage. Pairs naturally with the code-level example in the lineage post to move from the problem to the demonstrated approach.

Contracts still matter, but in this view they belong on top of that map, holding the few lines worth holding rather than being spread across every boundary. The argument is grounded in experience putting contracts into banks and enterprises: the data that causes the most damage is usually the field two teams away that nobody understood well enough to write a contract for.

Traces a single regulated value backwards through services and git history to show what the current answer costs in senior engineering time.

Does not dismiss data contracts. Treats them as genuinely useful at agreed critical boundaries, just not as a map of origin.

For brands planning a campaign

This is the strongest spine for a webinar on end-to-end lineage beyond the warehouse. The practical question to build the session around is: when a number looks wrong, can you show where it came from by reading the code, faster than a catalog or warehouse search can explain it?

Examples: Source 1 · Source 2 · Source 3 · Source 4

Sanderson says this leaves teams unable to answer ownership, semantic meaning, and proactive impact questions, and estimates such tools cover less than half of the information needed to make lineage valuable beyond data teams.

For brands planning a campaign

Useful counterpoint for a webinar that promises to go beyond warehouse lineage. Panelists can be asked to name a specific decision that downstream-only lineage cannot support, such as change impact before a release.

Examples: Source 1 · Source 2 · Source 3

He separates it from tech debt, arguing entropy happens even when sound technical decisions are made. He applies it to data and AI adoption: AI needs context about what a codebase does before it can safely modify it, and without a record of what AI changes, entropy accelerates.

For brands planning a campaign

Gives a data platform a broader framing than tool comparison: the session can argue that lineage and code context are what make AI engineering at scale possible, rather than a governance chore.

Examples: Source 1

Data, in this telling, is any output of one system consumed by another: an API response, a Kafka event, a Postgres record, an S3 file, a queue message, a log line, a SQL result. A change to a Java file that breaks an API consumer and a change to a dbt file that breaks a dashboard are treated as the same class of problem.

For brands planning a campaign

Supports a vendor-neutral opening: end-to-end lineage is not a new product category so much as the same data problem applied to the whole supply chain. Useful if the webinar audience includes software and platform teams, not only analytics engineers.

Examples: Source 1 · Source 2

He notes that data carries no information about its own origin, that asking producers to label more solves only part of the problem, and that historical data and older systems still need accounting.

For brands planning a campaign

Good segment for a webinar demo. Ask what a fully mapped enterprise codebase actually looks like and how a platform team would prioritize the first slice.

Examples: Source 1 · Source 2

LLMs are strongest on binary outcomes around common pattern recognition (does this function, query, or relationship exist) and on subjective problems with a spectrum of acceptable answers.

• They are weakest on non-binary questions with many variables and on objective questions with one definitive answer, where specificity matters.
• The comparison: AI is an exceptional driver because driving is subjective, but it is not exceptional at navigation, which is a computational problem, which is why we use GPS.
• The conclusion is that LLMs solve subjective problems and deterministic tools solve objective ones, with both feeding each other.

Categorizes production LLM experience by outcome type rather than by model capability.

For brands planning a campaign

A practical way to keep an AI-and-lineage webinar concrete. It gives panelists a shared vocabulary for deciding what should be automated probabilistically and what should be deterministic.

Examples: Source 1

Monitoring tells you what happened and contracts tell you what should happen, but with no shared agreement on meaning, data can carry negative value if it leads to the wrong conclusion.

For brands planning a campaign

Sharpens the webinar's urgency argument: agent-driven changes rewrite provenance continuously, which makes code-level lineage a prerequisite rather than a nice-to-have.

Examples: Source 1 · Source 2

Answering it honestly pulls senior engineers off their work to trace a value backwards through services and git history, and the resulting map is stale as soon as it is finished.

• The answer does not live in the warehouse or the catalog, he argues, because a regulated value is computed in application code.
• Teams that stopped paying this tax moved lineage into the build, so each release carries its inputs and transformation history.

For brands planning a campaign

A strong concrete hook for a webinar aimed at financial services buyers. It frames code-level lineage as audit preparation rather than an analytics optimization.

Examples: Source 1 · Source 2

They cannot answer where a value came from, what logic shaped it, or what depends on it. He also notes that quality is point-of-view dependent, so data can be correct for a producer and incorrect for a consumer, and that contracts were a mechanism for bridging that gap rather than a guarantee.

For brands planning a campaign

Useful for a session that must not sound like a contract vendor's pitch. It lets the host position contracts and code-level provenance as complementary controls with different jobs.

Examples: Source 1 · Source 2

In one post he frames the topic as "what we've built at Gable" and says the tooling runs in production on millions of lines of code; in another he signs off by saying he is excited to share "what we've learned and found here at Gable" and notes the company is hiring. The posts do not state a job title, start date, or ownership stake.

View example

His posts explain data lakehouse architecture and ask manufacturers to think about how much collected data is actually used, which overlaps with Sanderson's concern that data's value depends on whether teams understand where it came from and what it means.

Grasso ranges across digital transformation, smart manufacturing, and partner conversations, often in short explainer or interview-clip format. Sanderson stays closer to the engineering mechanics of lineage, contracts, and code-level provenance.

Read original · Original 2

That is the same territory Sanderson covers when arguing that a value's origin is lost without a record from the code that produced it.

Beddows approaches it through broad conceptual essays, historical analogies, and policy examples. Sanderson grounds the argument in production systems, contracts at banks, and specific tooling failures.

Read original · Original 2

That echoes Sanderson's point that data without context on its origin and meaning can mislead rather than inform.

Reeve applies that idea to pay transparency and compensation strategy for HR and total-rewards leaders. Sanderson applies it to software systems, lineage, and data engineering.

Read original · Original 2

The material covers why a data lake and a data lakehouse are different in practice, with retail and manufacturing examples, and it also shows a working interview format: a conversation with Barry Elliott from Rockwell Automation used to ask why manufacturers still struggle to turn collected data into decisions. That suits a co-hosted or partner-produced session aimed at a broad data and IT audience rather than one narrow engineering room.

The tradeoff is depth and originality.

• Grasso explains categories that are already widely understood, such as flexible storage versus managed tables, or whether using roughly 40 percent of collected manufacturing data should be read as a gap or as progress.
• A platform or risk leader leaves with vocabulary, not a method they can apply to a specific field.
• For a brief built around code-level lineage and what happens upstream of the warehouse, Grasso's posts stop at the analytical layers and do not address the origin-of-the-number problem at the centre of the Chad Sanderson fit.
• One post states a partnership with Rockwell Automation, so a buyer should expect partner-supported content conventions.

Examples: Source 1 · Source 2

The posts name sources-sought and RFI processes, laboratory-to-market ecosystems, and the Pentagon Papers, and they ask how data should be assessed for completeness, age, provenance, and the assumptions behind its use. That provenance framing overlaps directly with the lineage question and gives a policy or innovation audience a way into it.

The tradeoff is abstraction and currency.

• Beddows writes long numbered posts built on analogies, including Galileo's telescope, the canary in the coal mine, and Kenny Rogers on knowing when to fold, and the questions are posed rather than answered with a repeatable method.
• A buyer who wants the mechanics of tracing a field through application code, identifying real owners, and capturing lineage during the build will not find them here.
• The supplied posts are also the oldest in this comparison, from January and February 2025, so they are weaker evidence of what this audience is discussing now.

Examples: Source 1 · Source 2

The posts make a focused case that compensation data has a shelf life, that salary surveys and real-time labour market data each carry different signals, and that publishing a salary range without the logic behind it leaves employees unable to understand how pay decisions are made. A vendor selling compensation or HR technology would get a ready-made conversation, including a stated interview with Cary Sparrow on the People/AI Strategy Forum and a clear route to a follow-up resource.

For the data-lineage webinar the tradeoff is disqualifying.

• The reviewed posts contain no data engineering, lineage, governance, or regulated reporting material, so choosing Reeve would mean rewriting the brief rather than swapping one speaker into it.
• Reeve is also the wrong pick where the buyer wants technical depth from someone who builds the tooling, and would fit only if the campaign goal changed from reaching platform and risk leaders to reaching compensation and HR leaders.

Examples: Source 1 · Source 2