{
  "schemaVersion": 1,
  "id": "atlas-lesson-search-value-policy-boundary",
  "title": "Search, value estimates and selected actions",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Explains how a system can spend additional computation selecting an action after receiving an observation.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/search-value-policy-boundary/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "contrasting-ai-methods",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "The workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. Finite deterministic perfect-information game, alternating players and a declared fixed heuristic. This is minimax teaching search, not AlphaZero or a learned planning system.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/search-value-policy-boundary-fd0c4782ffb031c0.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#search-value-policy-boundary",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "da39d87d82a1be195b2a6862e25132ade19423498b8d578d10622a4d7a42159e",
  "contentLength": 6366,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-search-value-policy-boundary.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-search-value-policy-boundary.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Search, value estimates and selected actions",
      "start": 39,
      "end": 496
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 496,
      "end": 1669
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1669,
      "end": 1697
    },
    {
      "id": "section-4",
      "title": "Search, value estimates and selected actions",
      "start": 1697,
      "end": 2789
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2789,
      "end": 3145
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 3145,
      "end": 3741
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3741,
      "end": 4367
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 4367,
      "end": 4669
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4669,
      "end": 4698
    },
    {
      "id": "section-10",
      "title": "What changes when lookahead depth increases in this specimen?",
      "start": 4698,
      "end": 5282
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 5282,
      "end": 5778
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 5778,
      "end": 6064
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 6064,
      "end": 6087
    },
    {
      "id": "section-14",
      "title": "Related lessons",
      "start": 6087,
      "end": 6366
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#search-value-policy-boundary"
    },
    {
      "title": "Silver et al. · AlphaZero",
      "url": "https://arxiv.org/abs/1712.01815"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/search-value-policy-boundary-fd0c4782ffb031c0.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Evaluation & the AGI frontier",
      "url": "https://semiaifoundry.com/learn/atlas/evaluation/"
    },
    {
      "title": "Tools, memory & agent loop",
      "url": "https://semiaifoundry.com/learn/atlas/tool-loop/"
    },
    {
      "title": "Loss, gradients & optimizer",
      "url": "https://semiaifoundry.com/learn/atlas/loss-optimizer/"
    }
  ],
  "recordHash": "0d102679915cc216228d399f37241d3d68c407cbba877f063efd50ceacccfb44",
  "contentFormat": "text/markdown",
  "content": "contrasting-ai-methods · Atlas v1.3.0\n\n# Search, value estimates and selected actions\n\nExplains how a system can spend additional computation selecting an action after receiving an observation.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#search-value-policy-boundary)Mathematical and data space; dimensions are not physical distance\n\nLearning objectiveTrace legal game actions through a bounded alternating-player search and distinguish terminal evidence from a truncated heuristic.\n\n## The mechanism\n\nA policy selects an action. Search evaluates possible continuations before that selection, using a transition model and a value estimate at the explored boundary. In this rule-defined tic-tac-toe specimen, legal moves change a nine-cell board, players alternate, X maximizes and O minimizes a score. Terminal wins receive exact ±100 and draws receive zero. At a truncated leaf, an explicitly authored open-line heuristic supplies a provisional value. Deeper search may replace that guess with a different consequence, at a cost in visited nodes. The heuristic is not learned. Research systems such as AlphaZero combine learned value/policy functions with search; their training and search results are not reproduced by this small example.\n\n1.  Read the board, infer the next player from legal move counts and enumerate legal actions.\n2.  Apply each action to a separate board and alternate the maximizing/minimizing player.\n3.  Use exact terminal outcomes when available; at the selected depth, use the declared heuristic instead.\n4.  Choose the action under the resulting values and count the visited states. Preserve the original board for comparison.\n\n## Science and assumptions\n\n### Search, value estimates and selected actions\n\n$$\nV_d(s)=\\begin{cases}u(s),&s\\text{ terminal},\\\\h(s),&d=0,\\\\\\max_a V_{d-1}(T(s,a)),&X\\text{ to move},\\\\\\min_a V_{d-1}(T(s,a)),&O\\text{ to move}.\\end{cases}\n$$\n\nUse terminal utility, or a boundary heuristic when depth is zero, otherwise alternate maximizing and minimizing over legal transitions.\n\nSymbols and units\n\ns,a\n\nBoard state and legal action\n\ndiscrete state / action\n\nd\n\nRemaining lookahead depth\n\nplies\n\nT\n\nRule-defined transition\n\nfunction\n\nu,h,V\n\nTerminal utility, authored heuristic and search value\n\nscore\n\nThe displayed position has X at opposite corners and O in the center, so O moves next. Compare lookahead depths 1 through 5 and inspect each legal action’s value and visited-node count. A separate exact test uses an immediate X winning row and verifies that the winning move has value +100 regardless of the nonterminal heuristic.\n\nAssumptions Finite deterministic perfect-information game, alternating players and a declared fixed heuristic. This is minimax teaching search, not AlphaZero or a learned planning system.\n\n## Engineering example\n\nThe displayed position has X at opposite corners and O in the center, so O moves next. Compare lookahead depths 1 through 5 and inspect each legal action’s value and visited-node count. A separate exact test uses an immediate X winning row and verifies that the winning move has value +100 regardless of the nonterminal heuristic.\n\n## Distinctions and tradeoffs\n\nSearching at inference time is not the same operation as updating learned parameters. An authored heuristic must not be presented as a trained value function.\n\nMore search can improve a decision when the transition/value assumptions are useful, but consumes time and can amplify a wrong model. More visited nodes are not evidence of better real-world reasoning.\n\nInference-time selection, planning and tool policies can involve additional computation beyond one forward pass. Their benefits require task-specific verification and explicit environmental semantics.\n\n## Specifications and interfaces\n\nTeaching specimen\n\nSearch and value\n\nThe displayed position has X at opposite corners and O in the center, so O moves next. Compare lookahead depths 1 through 5 and inspect each legal action’s value and visited-node count. A separate exact test uses an immediate X winning row and verifies that the winning move has value +100 regardless of the nonterminal heuristic.\n\nEvidence boundary\n\nExplicit computed example\n\nFinite deterministic perfect-information game, alternating players and a declared fixed heuristic. This is minimax teaching search, not AlphaZero or a learned planning system.\n\n### Interfaces\n\nInputs and convention Input\n\nRead the board, infer the next player from legal move counts and enumerate legal actions.\n\nComputed result and scope Output\n\nTrace legal game actions through a bounded alternating-player search and distinguish terminal evidence from a truncated heuristic.\n\n## Check your understanding\n\n### What changes when lookahead depth increases in this specimen?\n\n1.  The evaluated continuation tree and computation spent selecting an action\n2.  The fixed heuristic is automatically trained\n3.  The rules of the environment change\n\nAnswer and reasoning\n\nThe evaluated continuation tree and computation spent selecting an action\n\nThe evaluated continuation tree and computation spent selecting an action. Finite deterministic perfect-information game, alternating players and a declared fixed heuristic. This is minimax teaching search, not AlphaZero or a learned planning system.\n\n## Evidence and scope\n\nThe workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. Finite deterministic perfect-information game, alternating players and a declared fixed heuristic. This is minimax teaching search, not AlphaZero or a learned planning system.\n\nOpen Mathematical foundations for the bounded executable mechanism and its record. General numerical labs are an additional reference library.\n\n## Sources\n\n-   [Silver et al. · AlphaZero](https://arxiv.org/abs/1712.01815)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/search-value-policy-boundary-fd0c4782ffb031c0.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Related lessons\n\n-   [Evaluation & the AGI frontier](https://semiaifoundry.com/learn/atlas/evaluation/)\n-   [Tools, memory & agent loop](https://semiaifoundry.com/learn/atlas/tool-loop/)\n-   [Loss, gradients & optimizer](https://semiaifoundry.com/learn/atlas/loss-optimizer/)"
}