{
  "schemaVersion": 1,
  "id": "atlas-lesson-management-controller",
  "title": "Baseboard management controller",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Monitors and controls hardware through a management path separate from the main workload.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/management-controller/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Operations and reliability",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "This is an ideal detection-time model, not a BMC service guarantee. Transient faults, missed polls, thresholds, retries and alert processing alter detection; the model and management networks remain separate functions.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/management-controller-0258525821be0014.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#management-controller",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "93019569faff4379bc6ba5124a01c9baf183df8e625cddad1f7646055241c6b2",
  "contentLength": 6174,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-management-controller.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-management-controller.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Baseboard management controller",
      "start": 43,
      "end": 438
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 438,
      "end": 1598
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1598,
      "end": 1626
    },
    {
      "id": "section-4",
      "title": "Polling cadence limits fault detection delay",
      "start": 1626,
      "end": 3078
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 3078,
      "end": 3351
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 3351,
      "end": 3909
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3909,
      "end": 4249
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 4249,
      "end": 4458
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4458,
      "end": 4487
    },
    {
      "id": "section-10",
      "title": "Why keep management traffic separate from training tensors?",
      "start": 4487,
      "end": 5040
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 5040,
      "end": 5454
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 5454,
      "end": 5782
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5782,
      "end": 5805
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5805,
      "end": 5887
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 5887,
      "end": 6174
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#management-controller"
    },
    {
      "title": "NVIDIA DGX GB rack systems networking guide",
      "url": "https://docs.nvidia.com/dgx/dgxgb200-user-guide/networking.html"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/management-controller-0258525821be0014.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Host CPU",
      "url": "https://semiaifoundry.com/learn/atlas/host-cpu/"
    },
    {
      "title": "Compute baseboard and tray",
      "url": "https://semiaifoundry.com/learn/atlas/compute-baseboard/"
    },
    {
      "title": "Power supply and power shelf",
      "url": "https://semiaifoundry.com/learn/atlas/power-supply/"
    },
    {
      "title": "Network interface adapter",
      "url": "https://semiaifoundry.com/learn/atlas/network-adapter/"
    }
  ],
  "recordHash": "31024f529afbc9d642172166cfad879023d4542adca9a222482c272618ce543e",
  "contentFormat": "text/markdown",
  "content": "Operations and reliability · Atlas v1.3.0\n\n# Baseboard management controller\n\nMonitors and controls hardware through a management path separate from the main workload.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#management-controller)Small board controller and rack-wide logical management paths\n\nLearning objectiveRelate polling interval to ideal fault-detection delay without confusing monitoring with the model data path.\n\n## The mechanism\n\nThe BMC is a small management computer that remains useful when the main operating system is unavailable. It collects sensor readings, exposes hardware status, and supports operations such as power control and remote maintenance through a separate network. In large AI installations, this out-of-band view helps operators locate failing components and coordinate recovery. It should not be confused with the CPU running the model service or a NIC carrying training tensors. Management connections reach equipment such as compute trays, power devices and switches to report health and coordinate control, separately from workload data traffic. A healthy management path does not mean the workload is healthy, but it provides the observations and controls needed to diagnose the physical system.\n\n1.  The controller collects hardware sensor readings and device status through management interfaces.\n2.  It exposes that information over an out-of-band network independent of the main application data path.\n3.  Authorized operators or automation use the observations and controls to diagnose faults, change power state, or coordinate recovery.\n\n## Science and assumptions\n\n### Polling cadence limits fault detection delay\n\n$$\n\\mathbb E[T_{\\mathrm{detect}}]=\\frac{\\Delta t}{2},\\qquad0\\le T_{\\mathrm{detect}}<\\Delta t\n$$\n\nWith uniform fault-onset phase, mean wait to the next poll is half the polling interval; the ideal wait lies between zero and one interval.\n\nThe source writes a strict maximum below the interval. A continuous uniform wait has supremum Δt, so this states the intended per-event range (almost surely) rather than an unattained maximum. Actual observation/transport delays are excluded.\n\nSymbols and units\n\nT\\_detect\n\nWaiting time from persistent fault onset to the next poll\n\ns\n\nΔt\n\nFixed interval between ideal polls\n\ns\n\nE\\[T\\_detect\\]\n\nMean detection wait over uniformly distributed onset phase\n\ns\n\nΔt is the interval between equally spaced sensor checks. If a persistent fault starts at a uniformly random phase between checks, its average wait to the next observation is half the interval. Faster polling improves detection opportunity but produces more telemetry and processing. Event-driven alerts can supplement polling, while recovery time after detection remains a separate part of availability.\n\nAssumptions An ideal persistent, immediately observable fault with uniformly distributed start time, perfect polling, and zero collection/transport delay. Brief faults, missed checks, thresholds, retries, and alert pipelines change the result. This is a monitoring model, not a vendor BMC guarantee.\n\n## Engineering example\n\nSynthetic monitor polls every 10 s. If a persistent, immediately visible fault starts uniformly between polls, average time to the next poll is 5 s and the delay approaches at most one 10 s interval before collection and alert transport are added.\n\n## Distinctions and tradeoffs\n\nA healthy BMC connection does not prove the GPU workload or operating system is healthy. It provides a separate way to observe and control the hardware.\n\nA separate controller improves observability and recovery, but adds firmware and network components to maintain. Management access and workload traffic should have clearly defined roles.\n\nThousands of accelerators require coordinated monitoring and repair. Management controllers help turn a collection of chips into infrastructure that can stay available over long jobs.\n\n## Specifications and interfaces\n\nNetwork role\n\nOut-of-band management\n\nDistinct from the data and GPU communication networks.\n\nObserved state\n\nPower, temperatures, and faults\n\nExact sensors and controls depend on the device.\n\nDGX management endpoints\n\nCompute trays, switches, and power shelves\n\nDocumented rack management network scope.\n\n### Interfaces\n\nSensors Input\n\nTemperatures, power readings, and hardware status\n\nManagement network Output\n\nTelemetry and event reporting\n\nControl requests Input\n\nAuthorized maintenance and power operations\n\n## Check your understanding\n\n### Why keep management traffic separate from training tensors?\n\n1.  Management should replace all high-bandwidth GPU communication.\n2.  A distinct path helps operators observe and recover hardware when the main workload path fails.\n3.  The BMC must calculate all model attention scores.\n\nAnswer and reasoning\n\nA distinct path helps operators observe and recover hardware when the main workload path fails.\n\nOut-of-band management improves access to hardware state during failures or maintenance without making it part of the main numerical data path.\n\n## Evidence and scope\n\nThis is an ideal detection-time model, not a BMC service guarantee. Transient faults, missed polls, thresholds, retries and alert processing alter detection; the model and management networks remain separate functions.\n\nUse the manufacturing investigation to trace compatible inputs, assembly and operating constraints; no current standalone calculator models this lesson’s full mechanism.\n\n## Sources\n\n-   [NVIDIA DGX GB rack systems networking guide](https://docs.nvidia.com/dgx/dgxgb200-user-guide/networking.html)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/management-controller-0258525821be0014.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Host CPU](https://semiaifoundry.com/learn/atlas/host-cpu/)\n\n### Related lessons\n\n-   [Compute baseboard and tray](https://semiaifoundry.com/learn/atlas/compute-baseboard/)\n-   [Power supply and power shelf](https://semiaifoundry.com/learn/atlas/power-supply/)\n-   [Network interface adapter](https://semiaifoundry.com/learn/atlas/network-adapter/)"
}