[{"data":1,"prerenderedAt":268},["ShallowReactive",2],{"blog-why-you-cannot-simply-use-an-llm-to-navigate-a-decision-tree":3},{"id":4,"title":5,"body":6,"description":12,"extension":257,"meta":258,"navigation":262,"path":263,"seo":264,"sitemap":265,"stem":266,"__hash__":267},"blog\u002Fblog\u002Fwhy-you-cannot-simply-use-an-llm-to-navigate-a-decision-tree.md","Why You Cannot Simply Use an LLM to Navigate a Decision Tree",{"type":7,"value":8,"toc":236},"minimark",[9,13,16,21,24,27,32,35,39,42,45,47,51,54,57,60,64,67,70,74,77,80,84,87,90,93,96,99,103,106,108,111,114,117,120,123,125,129,140,143,147,150,153,156,165,169,172,179,185,191,197,203,209,211,215,218,221,224,226,229],[10,11,12],"p",{},"There is a gap between how conversational AI feels and how structured processes actually work. Language models handle natural speech effortlessly. They remember what you said three messages ago, they let you correct yourself, and they never force you down a rigid path. Decision trees, on the other hand, are predictable, auditable, and legally defensible. For years, teams building configurators, advisory workflows, or diagnostic interviews had to choose one or the other. This article is about why that tradeoff exists, why naive attempts to combine both fall apart, and what a different structural approach makes possible.",[14,15],"hr",{},[17,18,20],"h2",{"id":19},"the-pre-llm-era-decision-trees","The Pre-LLM Era: Decision Trees",[10,22,23],{},"Before language models entered the picture, structured processes were built as decision trees. A user answers a question, the answer determines the next question, and the tree branches accordingly. That model has real strengths. Every possible path through the system is known before a single user ever touches it. You can test every branch, trace every decision, audit every outcome, and prove to a regulator exactly what happened and why. In industries like insurance, healthcare, or financial services, that kind of traceability is not optional.",[10,25,26],{},"The weaknesses are just as real, though, and they tend to be underestimated until you are deep in a project.",[28,29,31],"h3",{"id":30},"you-cannot-speak-to-a-decision-tree","You Cannot Speak to a Decision Tree",[10,33,34],{},"A decision tree expects structured input at each step. A dropdown, a radio button, a number field. It has no way to interpret \"actually, I meant the other one\" or \"let me go back and change what I said two steps ago.\" Natural language is simply outside the model.",[28,36,38],{"id":37},"going-back-is-harder-than-it-looks","Going Back Is Harder Than It Looks",[10,40,41],{},"Backtracking in a decision tree is not a feature you add, it is a separate engineering problem. Once a user has moved forward through several nodes, the decisions made along the way have side effects. Later questions may only exist because of earlier answers. If a user wants to change something early on, the answers that depended on that earlier value may now be invalid. You cannot simply reset one node. You have to figure out which downstream data is now inconsistent and clean it up.",[10,43,44],{},"Developers solve this by writing additional logic that mirrors the structure of the tree itself. I call this additional layer of code a shadow graph, because it replicates, in code, the branching logic that already exists in the visual tree.",[14,46],{},[17,48,50],{"id":49},"the-present-what-llms-seem-to-fix-and-where-they-actually-break","The Present: What LLMs Seem to Fix and Where They Actually Break",[10,52,53],{},"When language models became capable enough to hold structured conversations, the appeal was immediate. You replace the rigid tree with a prompt, feed user messages into the model, and get structured output back. The visual tree is gone. The shadow graph is gone. The interface is just a chat window.",[10,55,56],{},"And to be fair, language models are genuinely impressive at this. Backtracking happens naturally. A user can say \"wait, I want to change my earlier answer\" and the model adjusts. The conversation feels fluid in a way a decision tree never did.",[10,58,59],{},"But the simplicity is an illusion.",[28,61,63],{"id":62},"the-shadow-graph-does-not-disappear","The Shadow Graph Does Not Disappear",[10,65,66],{},"When you move a complex decision process into a language model, the logic that was previously in the tree has to go somewhere. It ends up in the system prompt, in structured output schemas, or in a set of validation rules that run after the model responds. The complexity is still there. It has been compressed and moved out of sight. Now it lives in a text file that no domain expert can inspect visually, that no tester can walk through node by node, and that no compliance officer can point to in an audit.",[10,68,69],{},"The shadow graph returned. It just became harder to read.",[28,71,73],{"id":72},"determinism-and-hallucination-are-two-different-problems","Determinism and Hallucination Are Two Different Problems",[10,75,76],{},"Teams working in regulated industries sometimes argue that determinism is achievable with language models if you use a fixed temperature and a fixed model version. That is technically correct and practically misleading. Deterministic output means the model produces the same response for the same input every time. It does not mean the response is correct. The model can hallucinate deterministically. The same wrong answer, every time, is not a feature.",[10,78,79],{},"Beyond that, real conversations never have the same input twice. A different phrasing, a different conversation history, or a different piece of context added mid-session is enough to produce different output. With enough variation in input, predicting output becomes very difficult.",[28,81,83],{"id":82},"the-problems-compound","The Problems Compound",[10,85,86],{},"Several other issues compound on top of each other when you run complex structured processes through a language model.",[10,88,89],{},"The context window grows with every exchange. A long configurator session or a multi-step interview accumulates text, and every backtrack adds more. At some point the model is reasoning over a very large block of text, and early parts of that text may receive less attention than recent ones.",[10,91,92],{},"Jailbreaks and distraction remain possible. A user who argues with the prompt, who asks off-topic questions at the wrong moment, or who finds a phrasing that confuses the model's output format can break the process in ways that are difficult to anticipate and test.",[10,94,95],{},"When the model provider changes a model, or when you switch providers entirely, behavior changes. A process that worked reliably under one model may fail silently under another, and those failures may not surface immediately.",[10,97,98],{},"If the AI service becomes unavailable, the entire process stops. There is no fallback mode.",[28,100,102],{"id":101},"mitigating-these-problems-brings-back-the-shadow-graph","Mitigating These Problems Brings Back the Shadow Graph",[10,104,105],{},"The natural response to these risks is to build a deterministic harness around the language model. Validate its output. Enforce a schema. Track the state of the conversation separately so the model cannot lose track of what has already been decided. That harness is, again, a shadow graph. And now you have two representations of your process that have to be kept in sync: the logic in the prompt and the logic in the harness. When the process changes, both need to be updated together.",[14,107],{},[17,109,5],{"id":110},"why-you-cannot-simply-use-an-llm-to-navigate-a-decision-tree",[10,112,113],{},"At this point you might ask: why not keep the decision tree as the source of truth and use the language model only to navigate it? Let the model interpret user messages and translate them into the structured inputs the tree expects.",[10,115,116],{},"The problem is that a standard decision tree is linear. It moves forward. If a user's response to question seven implies that their answer to question three was wrong, the tree has no native way to handle that. The model would have to tell the user to start over, or you would have to build, again, a shadow graph that tracks the conversation state and handles invalidations.",[10,118,119],{},"The very thing that makes decision trees valuable, their strict forward structure, is exactly what prevents a language model from navigating them naturally. You cannot easily combine the two without writing the logic layer that bridges them, and that logic layer is the problem you were trying to avoid.",[10,121,122],{},"What is needed is not a better prompt or a smarter harness. What is needed is a process representation that can hold a consistent state, propagate the consequences of any change through itself automatically, and remain readable to both a human and a language model at any point in the session. That is a different structural primitive altogether.",[14,124],{},[17,126,128],{"id":127},"the-future-reactive-state-graphs","The Future: Reactive State Graphs",[10,130,131,132,139],{},"Reactive Graph Sequencing, the technology behind GraVersal, is described in detail in ",[133,134,138],"a",{"href":135,"rel":136},"https:\u002F\u002Fgraversal.io\u002Fblog\u002Freactive-graph-sequencing-the-technology-behind-wanderer",[137],"nofollow","this earlier post",". The short version: a reactive state graph holds the current state of every node. When a value changes anywhere in the graph, the graph re-traverses itself and produces a new valid sequence. Nodes that depend on an invalidated value become inactive. New paths that are now reachable become active. The graph always reflects a consistent current state.",[10,141,142],{},"That structure changes what a language model can do.",[28,144,146],{"id":145},"the-llm-as-a-graph-navigator","The LLM as a Graph Navigator",[10,148,149],{},"A language model can be given read access to the full graph topology and the current state of the run. It can see which nodes are active, which questions are currently in scope, what values have been set, and which branches are reachable from the current position.",[10,151,152],{},"When the model wants to change a value, it does not rewrite the conversation history. It calls a tool that sets a value on a node in the graph. The graph reacts. It re-sequences itself, activates or deactivates branches, and returns a diff to the model. The model sees exactly what changed: which nodes became active, which fell out of scope, which constraints were satisfied, which were violated.",[10,154,155],{},"The model cannot do anything the graph does not allow. It cannot skip required fields, activate branches that have no valid path, or produce an inconsistent state. The graph enforces its own rules regardless of what the model tries to do.",[10,157,158,159,164],{},"An example of this pattern in practice is the ",[133,160,163],{"href":161,"rel":162},"https:\u002F\u002Fgraversal.io\u002Fflows\u002FTalking-to-Your-Own-Flow-An-Ice-Cream-Configurator-That-Explains-Itself-4utogag8sr9xrxey14yx5mjligci37ll",[137],"ice cream configurator",", where a language model can read the full graph and the current run state, answer questions about why certain options are or are not available, and guide a user through the configurator in natural language, while the graph itself controls what actually happens.",[28,166,168],{"id":167},"additional-structural-advantages","Additional Structural Advantages",[10,170,171],{},"Several things become possible in this setup that were not possible before.",[10,173,174,178],{},[175,176,177],"strong",{},"No history required."," The graph holds the current state. You do not need to keep the full conversation history in the context window to know where the user is. After six or eight messages, the history can be trimmed. The system prompt always imports the current graph state, which is the only ground truth the model needs.",[10,180,181,184],{},[175,182,183],{},"Distinguishable outputs."," Messages from the language model and outputs from the graph are separate things produced by separate components. You can label them differently in the interface. A compliance officer can see which statements came from a deterministic process and which came from a probabilistic model. That distinction matters legally and practically.",[10,186,187,190],{},[175,188,189],{},"LLMs become optional and interchangeable."," Because the graph holds the state and enforces the logic, the language model is a navigation aid, not the process itself. If the AI service is unavailable, the process can continue through a traditional interface. If you switch model providers, the graph continues to work. The model just has a different voice.",[10,192,193,196],{},[175,194,195],{},"Co-editing becomes natural."," A customer partially completes a complex insurance application or product configuration, then calls support the next day. The staff member who picks up the call may have no deep product knowledge. In a traditional setup, that person either reads from a script or escalates. With a reactive graph, they open the same session the customer left behind. The graph shows the current state clearly: what has been answered, what is still open, which branches are active. The staff member can continue the session directly, guided by the graph structure, without needing to understand every dependency behind it. And if the customer calls again later and wants to change something from the first session, the graph handles the downstream invalidation automatically. The staff member does not need to know which follow-up questions are now affected. The graph does. This means the graph functions as shared operational infrastructure for the AI assistant, the customer, and the human team simultaneously. It is not a tool only the AI uses.",[10,198,199,202],{},[175,200,201],{},"Hallucinations become structurally contained."," This is the central promise for regulated industries and it deserves to be stated precisely. Language models will continue to hallucinate. That is not a solvable problem in the near term, and any architecture that depends on the model never being wrong is brittle by design. The reactive graph takes a different position: it does not try to prevent hallucinations, it removes their path to the process state. A model can say something incorrect in a message. But it cannot write an incorrect value into the graph unless the graph accepts it as valid. If the model tries to set a field to an impossible value, or activate a branch that has no valid preconditions, the graph rejects it and returns an error the model can reason about. What the user might see in a chat message and what actually happens to the configuration or the advisory record are two separate things, produced by two separate systems with different guarantees. That separation is what makes the approach defensible in regulated contexts. You do not need to certify the language model. You need to certify the graph.",[10,204,205,208],{},[175,206,207],{},"Spontaneous restructuring becomes possible."," Because the graph is data, it can be modified, extended, or partially replaced without touching a codebase. New subgraphs can be loaded dynamically during a session. And because there is no shadow graph to maintain in parallel, a change to the process is a change to one thing, not two.",[14,210],{},[17,212,214],{"id":213},"what-this-means-for-regulated-industries","What This Means for Regulated Industries",[10,216,217],{},"The core requirement in regulated contexts is not that a system be intelligent. It is that a system be auditable, predictable, and explainable. A reactive state graph satisfies those requirements directly. Every state transition is logged. Every active branch is visible. Every outcome can be traced back to the sequence of values that produced it.",[10,219,220],{},"The language model operates within that structure. It can hallucinate. Every language model can. But when the graph is the harness, a hallucination has no path to the process state. The model might say something incorrect, but it cannot set an incorrect value that the graph would accept as valid. The consequences of a hallucination are limited to the message the user sees, not to the state of the process.",[10,222,223],{},"That is a meaningful shift. Not the elimination of hallucination, but the structural containment of its effects.",[14,225],{},[10,227,228],{},"The choice is no longer between a system that is predictable but rigid and one that is conversational but unpredictable. The graph handles predictability. The model handles conversation. Keeping those responsibilities separate, and connecting them through a well-defined interface, is what makes both work.",[10,230,231,232],{},"Title image: ",[133,233,234],{"href":234,"rel":235},"https:\u002F\u002Funsplash.com\u002Fde\u002Ffotos\u002Fjemand-zeichnet-auf-einem-tablet-an-seinem-schreibtisch-vhZ8K5Np9mk",[137],{"title":237,"searchDepth":238,"depth":238,"links":239},"",2,[240,245,251,252,256],{"id":19,"depth":238,"text":20,"children":241},[242,244],{"id":30,"depth":243,"text":31},3,{"id":37,"depth":243,"text":38},{"id":49,"depth":238,"text":50,"children":246},[247,248,249,250],{"id":62,"depth":243,"text":63},{"id":72,"depth":243,"text":73},{"id":82,"depth":243,"text":83},{"id":101,"depth":243,"text":102},{"id":110,"depth":238,"text":5},{"id":127,"depth":238,"text":128,"children":253},[254,255],{"id":145,"depth":243,"text":146},{"id":167,"depth":243,"text":168},{"id":213,"depth":238,"text":214},"md",{"date":259,"author":260,"headerImage":261},"2026-09-27","Chris","\u002Fimages\u002Fblog\u002Fflowchart.jpg",true,"\u002Fblog\u002Fwhy-you-cannot-simply-use-an-llm-to-navigate-a-decision-tree",{"title":5,"description":12},{"loc":263},"blog\u002Fwhy-you-cannot-simply-use-an-llm-to-navigate-a-decision-tree","DafOCsv-0i_5GFOqDYawmbyxmLPOS8WW9dobzhVNWDQ",1790888815707]