{"id":"tree-ensemble-baseline","name":"A tree ensemble that learns residual corrections","shortName":"Learn a tree ensemble","role":"Provides a contrasting learned prediction mechanism that does not use token attention.","description":"A regression stump divides one feature at a threshold and predicts a value on either side. An additive ensemble can fit successive stumps to the current residuals, gradually correcting its earlier predictions. The workbench actually searches thresholds between training values, computes residual means in both leaves, chooses the lowest squared residual error and applies shrinkage before the next round. The reserved examples never fit the thresholds or leaf values. This simple squared-loss mechanism illustrates boosting; it does not reproduce the many algorithms and engineering choices in production XGBoost. The separate reversed-relationship challenge exposes why fitting the training pattern does not guarantee robustness to a changed task.","mechanism":["Initialize predictions at the mean training target and calculate each residual.","For every candidate threshold, partition training rows, fit left/right residual means and compute squared error.","Choose the best stump and add its shrunken leaf contributions to the ensemble. Repeat on the remaining residuals.","Evaluate reserved examples and a separately defined reversed-label relationship with the same fitted ensemble."],"learningObjective":"Fit a sequence of regression stumps, inspect their contributions and evaluate both a reserved task and a changed relationship.","engineeringExample":"Six training points map negative values to −2 and nonnegative values to 2. The first fitted split is −0.5. With shrinkage 0.5, its contributions are −1 and 1. Four rounds leave a reserved-set MSE of 0.015625, while reversing the target relationship produces MSE 15.015625. Both results come from the same actually fitted stumps.","evidenceNotes":"The workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. One numeric feature, squared loss, finite candidate splits and constant leaves; no regularized second-order XGBoost implementation or performance claim.","specs":[{"label":"Teaching specimen","value":"Learn a tree ensemble","context":"Six training points map negative values to −2 and nonnegative values to 2. The first fitted split is −0.5. With shrinkage 0.5, its contributions are −1 and 1. Four rounds leave a reserved-set MSE of 0.015625, while reversing the target relationship produces MSE 15.015625. Both results come from the same actually fitted stumps."},{"label":"Evidence boundary","value":"Explicit computed example","context":"One numeric feature, squared loss, finite candidate splits and constant leaves; no regularized second-order XGBoost implementation or performance claim."}],"interfaces":[{"name":"Inputs and convention","direction":"in","signal":"Initialize predictions at the mean training target and calculate each residual."},{"name":"Computed result and scope","direction":"out","signal":"Fit a sequence of regression stumps, inspect their contributions and evaluate both a reserved task and a changed relationship."}],"tradeoff":"More rounds can fit more detail, but complexity and learning rate affect generalization and computation. A task baseline should be evaluated under the same data boundaries as a larger model.","whyAI":"AI applications often use structured/tabular prediction. Understanding this mechanism broadens the atlas beyond the assumption that all useful learning is a decoder transformer.","misconception":"A good score on one held-out distribution does not imply resistance to a reversed or otherwise changed input/output relationship.","check":{"question":"What data choose the stump threshold in this experiment?","options":["The reversed-relationship challenge labels","Only the training rows and their current residuals","The final reported evaluation score"],"answer":1,"explanation":"Only the training rows and their current residuals. One numeric feature, squared loss, finite candidate splits and constant leaves; no regularized second-order XGBoost implementation or performance claim."},"prerequisites":["ablation-reproduction-design"],"related":["loss-optimizer","evaluation","host-cpu"],"sources":[{"title":"Chen and Guestrin · XGBoost","url":"https://arxiv.org/abs/1603.02754"}],"level":"model","scale":"Mathematical and data space; dimensions are not physical distance","kind":"model","branch":"contrasting-ai-methods","subbranch":"Learn a tree ensemble","workstream":"W11","visualFamily":"mathematical","labId":"library","labScope":"Open Mathematical foundations for the bounded executable mechanism and its record. General numerical labs are an additional reference library.","science":[{"title":"A tree ensemble that learns residual corrections","equation":"Fit a stump to current residuals and add a scaled contribution to the existing prediction function.","explanation":"Six training points map negative values to −2 and nonnegative values to 2. The first fitted split is −0.5. With shrinkage 0.5, its contributions are −1 and 1. Four rounds leave a reserved-set MSE of 0.015625, while reversing the target relationship produces MSE 15.015625. Both results come from the same actually fitted stumps.","assumptions":"One numeric feature, squared loss, finite candidate splits and constant leaves; no regularized second-order XGBoost implementation or performance claim."}]}