Independent research · 2026
The Price of Nestedness in Group-Robust PCA
Working paper · Manuscript not yet public
Brandon Baek
Proofs and certified experiments · Public overview
One code. Every size. Every group.
Many AI systems compress data into a short code and add more numbers when a bigger budget is available, so the first few numbers have to work on their own. This is called a nested representation; Matryoshka embeddings work this way. My paper asks what that convenience costs when one code has to serve several populations, and we judge it by the group it serves worst.
Best at each size is not the same as best for all sizes.
If each size could choose its own directions, it would pick whatever protects the worst-off group at that size. A nested code cannot: the size-2 code has to keep the size-1 direction. The price of nestedness compares the nested code’s worst-group error with the best separate code, size by size.
Worked out by hand to explain the idea. It is not a result from the paper.
Share of each group’s signal the code keeps
Prove it first, then check it on real data.
The paper bounds how large the price of nestedness can get, builds examples that show when some price is unavoidable, and separates the cost of nesting itself from the cost of imperfect optimization. It then audits fitted data and pretrained nested text-embedding models with certificates: guaranteed ranges for the true cost, not just point estimates.
Bound
How large can the price get, however many groups or sizes there are?
Construct
When is some price impossible to avoid?
Certify
How large is it on real data, with guaranteed error bars?
Working paper
The manuscript, its exact bounds, and its numerical results are not public yet. This page explains the question and a hand-worked toy example; it does not report the paper’s findings.
Smaller should not mean unfair.
Nested codes let one model serve a phone and a server, a quick search and a careful one. If shrinking the code quietly shortchanges one group, that cost should be measured and bounded, not assumed away. The same question runs through all of my research: how little can a system use and still keep its promise to everyone?
