{"id":"tool-loop","level":"model","name":"Tools, memory & agent loop","shortName":"Tools, memory & agent loop","role":"Connect model inference to actions and evidence.","description":"A model can propose a structured tool call, but a surrounding runtime must validate it, execute the tool, and return the result. The next model invocation can use that observation. Search, code execution, file operations, and external memory are therefore parts of a larger system around the neural network. Permissions, state tracking, error handling, and verification determine what the system can do reliably. A retrieved document and a model parameter are different kinds of memory.\n\nAn observable tool loop needs a contract for effects as well as returned text. The durable incident experience now executes proposals through schema validation, authority, exact human review, service and receipt recording. Its fixed workflow and bounded branching policy are authored code. Their visible decisions demonstrate runtime boundaries without claiming a language model generated them. A future real-model adapter must identify its model/configuration and preserve the same runtime checks.\n\nA model policy can propose a structured action, but a separate environment contract determines whether it is valid and authorized. The tool response becomes an observation, not an instruction with authority over the system. A plausible final sentence does not establish that the requested state transition occurred. The connected automation experience supplies bounded actions and verifiable outcomes; the lifecycle decoder itself generates symbols and does not call external tools or receive extra training from a tool response.","mechanism":["Interpret the task and generate a response or proposed action.","Execute an allowed tool through the runtime and capture its result.","Use observations to revise the plan, continue, or finish.","A proposed call remains separate from permission and the external effect; the caller observes the returned result or explicit uncertainty.","After a write commits but its response is lost, a receipt or idempotent retry resolves the original action before another dependent action starts.","Log the policy input and proposed structured action.","Validate schema, capability, preconditions and budget before any state mutation.","Record the actual observation and independently score the desired state change."],"specs":[{"label":"Boundary","value":"Model ↔ runtime ↔ environment","context":"Tools execute outside the weight matrices."},{"label":"Persistence","value":"Application dependent","context":"External memory differs from a context window."}],"interfaces":[{"name":"User and observations","direction":"in","signal":"Context plus task state"},{"name":"Runtime","direction":"out","signal":"Structured calls or final response"}],"tradeoff":"More tools expand reach while increasing dependencies and failure modes. Useful autonomy requires measurable completion and recovery behavior.","whyAI":"Advanced assistants combine learned inference with software and infrastructure.","related":["output-head","serving","evaluation","post-training","astra","durable-actions-idempotency","tool-contracts-schemas","human-review-cancellation","probability-calibration-abstention","judge-rubric-agreement","retrieval-supported-generation","test-time-search-verification","model-system-regression-contract"],"sources":[{"title":"Schick et al. — Toolformer","url":"https://arxiv.org/abs/2302.04761"},{"title":"Yao et al. — ReAct","url":"https://arxiv.org/abs/2210.03629"},{"title":"Temporal Activities","url":"https://docs.temporal.io/activities"},{"title":"MCP tools","url":"https://modelcontextprotocol.io/specification/2026-07-28/server/tools"},{"title":"ReAct: Synergizing Reasoning and Acting in Language Models","url":"https://arxiv.org/html/2210.03629v3"}],"scale":"Logical structure · mapped onto physical hardware","kind":"tool-loop","misconception":"Tool access is a system capability, not a new physical neuron inside the model.","check":{"question":"What actually executes a model-proposed tool call?","options":["A surrounding runtime or application","The embedding table itself","A silicon atom"],"answer":0,"explanation":"The model emits a proposal; software carries out the operation and returns observations."},"branch":"Agent systems","subbranch":"Actions and observations","science":[{"title":"A closed loop changes context and the environment","equation":"aₜ ∼ πθ(·|hₜ); oₜ = Tool(aₜ); hₜ₊₁ = Update(hₜ,aₜ,oₜ)","explanation":"The model conditions an action proposal a on available history h. A runtime executes an allowed tool and returns an observation o. The next invocation receives updated context and can change its next action. This creates feedback with the world while ordinary inference keeps the trained parameters θ fixed.","assumptions":"This is a schematic sequential loop, not a hidden implementation. Tools may fail, be asynchronous, or change external state; the runtime enforces permissions and can decline actions. Context management may summarize rather than retain all history."},{"title":"A model policy with validated observations and actions · worked relation","equation":"s_{t+1}=\\begin{cases}T(s_t,a_t)&V(s_t,a_t)=1\\\\s_t&V(s_t,a_t)=0\\end{cases}","explanation":"In an authored inventory environment, stock starts at 3. A proposed reserve(2) passes a bound of available stock and commits stock=1. Retrying the same committed action with the same idempotency key returns its prior receipt rather than subtracting again. A fresh reserve(2) fails because only one remains. Saying “reserved” after that rejection is an answer failure even if its JSON syntax is valid.","assumptions":"Authored finite inventory contract, not an actual external order. Model generation, validator execution and learning updates have separate state boundaries."}],"history":[{"year":2022,"title":"ReAct interleaves language-model reasoning and actions","significance":"Yao and colleagues explored alternating reasoning and actions that gather observations from external environments. The preprint appeared in 2022 and the conference version at ICLR 2023.","source":{"title":"Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models","url":"https://arxiv.org/abs/2210.03629"}}],"prerequisites":["output-head","serving"],"learningObjective":"Track which state changes in a tool-assisted inference loop and account for its sequential latency.","engineeringExample":"Synthetic loop: first model call takes 0.8 s, a permitted tool takes 0.3 s, and the follow-up model call takes 0.7 s. With sequential execution, elapsed time is 1.8 s before overhead. Tool output changes the next call’s context; it does not by itself update the model’s learned weights.\n\nIn the commit-timeout case, reserving two FILTER-A units leaves two of the original four units before the caller has any successful response. The fixed workflow retries the same keyed action; the authored policy looks up the receipt. Both can finish with one reservation and one work order. Neither can infer that the timeout meant no effect occurred.\n\nIn an authored inventory environment, stock starts at 3. A proposed reserve(2) passes a bound of available stock and commits stock=1. Retrying the same committed action with the same idempotency key returns its prior receipt rather than subtracting again. A fresh reserve(2) fails because only one remains. Saying “reserved” after that rejection is an answer failure even if its JSON syntax is valid.","evidenceNotes":"The model–action–observation pattern follows the linked Toolformer/ReAct context. Times are synthetic; parallel tools, failure, retries and context management alter timing. The runtime validates permissions and results rather than treating a proposal as completed work.\n\nThe executable extension is a fictional local environment with two authored policies. It adds no claim of real LLM agency, MCP conformance or production durability. Model parameters, invocation context and the workflow journal remain separate objects.\n\nMode: worked example. Authored finite inventory contract, not an actual external order. Model generation, validator execution and learning updates have separate state boundaries. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.","checks":[{"question":"What does a lost tool response establish about whether the write committed?","options":["It establishes no effect","It leaves the outcome uncertain until checked","It automatically authorizes a second logical write"],"answer":1,"explanation":"Transport failure does not determine external state; check the original keyed effect."},{"question":"What establishes successful reservation after a tool call?","options":["A fluent sentence saying the reservation succeeded.","The presence of a syntactically valid action proposal alone.","A validated committed receipt and the corresponding environment state."],"answer":2,"explanation":"A validated committed receipt and the corresponding environment state. Authored finite inventory contract, not an actual external order. Model generation, validator execution and learning updates have separate state boundaries."}],"labId":"library","scope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism.","investigationId":"execution","labScope":"Use the execution investigation for the shared tensor-to-memory-to-result thread; no current standalone calculator models this lesson’s full mechanism."}