The mechanism
A linear map transforms input directions into output directions. Its rank counts independent output directions, not entries or bytes. A rank-restricted map can be stored or updated through smaller factors, but removing a direction changes the function whenever a relevant input contains that component. The specimen constructs a symmetric two-dimensional map from orthogonal unit vectors u and v, with singular values 3 and σ₂. Dropping the v term gives a rank-one approximation. Rotating the basis changes the visible entries while preserving the amount of discarded Frobenius norm. This makes a useful prerequisite for understanding factorized weights and adapters without claiming that a small matrix-error metric determines a task’s quality.
- Choose two orthogonal unit vectors and scale their outer products by the two singular values.
- Add the two outer products to construct the exact map; retain only the larger term for the rank-one map.
- Subtract the maps and inspect an input along the dropped direction. Its smaller response is lost.
- Rotate the basis while holding singular values fixed. Compare changed entries with invariant discarded norm.
Science and assumptions
Linear maps, rank and retained directions
The exact map adds two orthogonal outer products; dropping the second leaves Frobenius error equal to its nonnegative singular value.
Symbols and units
- A,A₁
- Exact and rank-one maps
dimensionless
- u,v
- Orthogonal unit vectors
dimensionless
- σ₂
- Second singular value, from zero to three
dimensionless
For u=[1,0], v=[0,1] and σ₂=1, A=diag(3,1) while A₁=diag(3,0). Input [0,1] maps to [0,1] under A and [0,0] under A₁. The Frobenius error is 1. Rotating both directions by 30° changes every matrix entry but leaves this error at 1.
Assumptions Real symmetric two-dimensional constructed map with orthonormal u and v and 0 ≤ σ₂ ≤ 3. This is not an arbitrary-matrix SVD solver.
Engineering example
For u=[1,0], v=[0,1] and σ₂=1, A=diag(3,1) while A₁=diag(3,0). Input [0,1] maps to [0,1] under A and [0,0] under A₁. The Frobenius error is 1. Rotating both directions by 30° changes every matrix entry but leaves this error at 1.
Distinctions and tradeoffs
A low-rank adapter is a choice of trainable parameterization. It is not by itself a pretraining, supervised or preference-learning objective.
Reduced factor size can lower trainable state or arithmetic, but task-relevant information may occupy the removed subspace. Task evaluation remains necessary.
Embedding geometry, compressed operators and low-rank parameter updates rely on structured directions. Rank and parameter count are related through a specific representation, not by a universal quality law.
Specifications and interfaces
- Teaching specimen
- Rank and directions
For u=[1,0], v=[0,1] and σ₂=1, A=diag(3,1) while A₁=diag(3,0). Input [0,1] maps to [0,1] under A and [0,0] under A₁. The Frobenius error is 1. Rotating both directions by 30° changes every matrix entry but leaves this error at 1.
- Evidence boundary
- Explicit computed example
Real symmetric two-dimensional constructed map with orthonormal u and v and 0 ≤ σ₂ ≤ 3. This is not an arbitrary-matrix SVD solver.
Interfaces
- Inputs and convention Input
- Choose two orthogonal unit vectors and scale their outer products by the two singular values.
- Computed result and scope Output
- Identify the direction lost by a rank-one approximation and distinguish rank from matrix size.
Check your understanding
What stays unchanged when this orthogonal basis is rotated?
- Every displayed matrix entry
- The singular values and the Frobenius norm of the discarded component
- The usefulness of the approximation on every task
Answer and reasoning
The singular values and the Frobenius norm of the discarded component
The singular values and the Frobenius norm of the discarded component. Real symmetric two-dimensional constructed map with orthonormal u and v and 0 ≤ σ₂ ≤ 3. This is not an arbitrary-matrix SVD solver.
Evidence and scope
The workbench executes the declared small mechanism. Numeric examples are authored and independently checked, not measurements from the cited research. Real symmetric two-dimensional constructed map with orthonormal u and v and 0 ≤ σ₂ ≤ 3. This is not an arbitrary-matrix SVD solver.
Open Mathematical foundations for the bounded executable mechanism and its record. General numerical labs are an additional reference library.