The Money-Making Black Box

Machine learning has transformed vision, language, games — even the prediction of a protein’s folded shape — and every one of those victories quietly relied on gifts financial markets withhold: abundant data, stable rules, a legible signal, and a target that does not move because it was predicted. Medium-horizon asset return forecasting is starved of all four, which is why the sophistication of a model says so little about whether it will survive a live market. Asset managers, ever keen on the cutting edge, keep adopting these methods in the pursuit of alpha — sometimes successfully, often not. We argue that what divides the models that endure from those that vanish is less the power of the box than the rigor built around and into it — governance by construction, not as an afterthought: controls and economic constraints designed into the model from the outset, uncertainty quantified so the system knows when to abstain, frictions and capacity priced before the first trade, and every claimed edge made auditable. Borrowing medicine’s discipline, where trust rests on a protocol of evidence and vigilance rather than on a fully understood mechanism, we suggest that investors should shift the burden of proof from fully explaining the model to governing it — and close with the questions an allocator should ask before believing that any machine has learned to beat the market.
A note on terminology, before we begin. We deliberately avoid the oversold phrase “artificial intelligence.” We do not yet understand what natural intelligence is, let alone artificial; what we actually do is far more modest and concrete — statistical learning: the fitting of functions to noisy samples and the optimization of policies against noisy rewards — and we prefer the term that names it honestly and carries none of the fashionable mystique. Where “AI” appears in these pages, it denotes the popular category and its marketing, not a capability we claim to possess. We will also assume the reader, particularly one arriving from machine learning, is reasonably familiar with finance terminology such as the Sharpe ratio.For the uninitiated reader, the finance industry uses this term loosely, and the same word often names different quantities in different rooms, so we state ours. The Sharpe ratio is return per unit of risk — the signal-to-noise ratio of a return series, and the nearest thing finance has to an effect size: , with r the strategy’s return over one period, rf the risk-free rate over the same period, E[·] the mean, and σ(·) the standard deviation of that same excess return. Two conventions vary. The numerator is normally an excess return over a risk-free benchmark; the version that omits the subtraction, or that measures excess over some other declared benchmark rather than cash, is commonly called an information ratio — properly, active return over tracking error, the standard deviation of that same active return — and the two names are not used consistently. The figure also depends on the period it is measured over: a daily one is conventionally annualized by multiplying by √252, the trading days in a year, an annualization that assumes returns serially uncorrelated across days, and so overstates the figure wherever they are positively autocorrelated — smoothed marks, illiquid positions, a book whose gains arrive in runs. Averaging matters too. The arithmetic mean is not what an investor earns; the compounded rate is , and for small returns . That last term is the volatility drag, and it is why two strategies with the same average return and different volatilities do not finish the decade in the same place.
A note, too, on the marks in the margin. The claims of this essay are not all of equal epistemic strength, and it seemed more transparent to admit it. To avoid apologizing continually for insufficient evidence, we propose three marks annotating the text they face, each grading its central claim: a ✓ where the claim is well established within the scope stated; an ~ where the evidence is mixed or scope-dependent and we are taking a side in a live debate; and a ? where the claim is a conviction of ours — reasoned, we would argue, but not yet compelled by evidence, and stated so that it can be proven wrong. We treat this gradation not as ornament but as discipline: the same calibration of confidence we demand of every model, turned upon ourselves.
The question everyone is asking
In the autumn of 2012, three researchers in Toronto entered a neural network into an image-recognition contest and cut the error rate by more than a third overnight — the better part of five years’ progress at the contest’s prior pace, delivered in one entry.The 2012 result was “AlexNet” (Krizhevsky, Sutskever & Hinton, NIPS 2012), which cut the winning ImageNet ILSVRC top-5 error rate to 15.3%, against 26.2% for the next-best entry. The milestones that followed — AlphaGo’s 2016 defeat of Lee Sedol, AlphaFold2’s protein-structure predictions at CASP14 in 2020 — share one substrate — deep neural networks trained by stochastic gradient descent on very large corpora — though each added scaffolding of its own: tree search and self-play in the games, evolutionary profiles in the proteins; structure, that is, on top of scale. The point of this paper is that even the substrate travels far less well than the headlines suggest. In the years that followed, the same broad recipe (a flexible model, an ocean of data, and a great deal of compute) went on to translate between languages, to transcribe speech, to eclipse the world’s finest players of chess and Go, and, most improbably of all, to predict the folded shape of a protein from its amino acid sequence.AlphaFold is often hailed as the triumph of data-driven modeling: behold how the machine has solved the fifty-year-old holy grail of structural biology from experimental data alone. Nothing could be further from the truth. The labeled corpus it learned from is, by machine-learning standards, tiny — the Protein Data Bank as it stood before AlphaFold2’s April 2018 training cutoff (Jumper et al., Nature, 2021), on the order of a hundred thousand unique experimentally determined proteins, a corpus that has since grown substantially — yet it succeeds because it leans on a powerful prior as much as on scale — the coevolutionary signal in multiple-sequence alignments, whereby residues that mutate in a correlated way across evolution tend to sit close together in space: Göbel, Sander, Schneider & Valencia (Proteins, 1994) read spatial contact out of correlated mutation, and Marks, Colwell, Sheridan, Hopf, Pagnani, Zecchina & Sander (PLoS ONE, 2011) turned that reading into folded structures a decade before AlphaFold2. That human eureka moment and not brute-force supervised training was one decisive ingredient that made a scarce labeled corpus usable at all (without derogating from the importance of other ingredients such as the architecture itself, the iterative refinement and the self-distillation that all did real work and won the 2024 Nobel Prize). The multiple-sequence alignments themselves, it should be said, distill a gargantuan corpus of unlabeled sequences — billions of them — so the scarcity is one of labels rather than of data: the labeled examples were few, and the gap was closed by structure and by knowledge carried in through the inputs. It is a vivid illustration of this paper’s central engineering claim — that in a data-poor setting it is structure and prior knowledge, not the raw volume of labeled examples, that buy performance and generalization. Most telling of all, much of the field converged on a dominant architecture, the transformer, so indifferent to the kind of data it was shown (once each is minced into tokens) that sentences, images, audio, even the moves of a robot arm could be poured into the same machine and learned.The transformer’s indifference to modality is well documented. The Perceiver (Jaegle, Gimeno, Brock, Zisserman, Vinyals & Carreira, “Perceiver: General Perception with Iterative Attention,” ICML 2021) applies essentially the same attention-based network, with strikingly few modality-specific assumptions, to images, point clouds, audio, and video alike; and a single trained network, Gato (Reed et al., “A Generalist Agent,” Transactions on Machine Learning Research, 2022), uses one set of weights to caption images, hold a conversation, stack blocks with a real robot arm, and play dozens of Atari games, among hundreds of distinct tasks in all. The architecture itself is due to Vaswani et al. (“Attention Is All You Need,” NIPS 2017). The lesson this paper presses is that such apparent universality is a property of data-rich, largely stationary domains, and does not easily survive simple transplantation into the limited-history, regime-dependent world of financial data. It is only natural to ask the next question, and a great many people are now asking it with real money in hand: if a machine can fold a protein and outplay a grandmaster, can it pick stocks?
~Sometimes it can. Systematic strategies can deliver persistent alpha, but only when they are built under conditions these pages will name. Over the last four decades a broad cohort of systematic managers has shown that disciplined, data-driven processes can compound capital: long-only quantitative strategies at firms such as BlackRock, PGIM Quantitative Solutions, and DFA; statistical arbitrage and market-neutral books at firms such as Two Sigma and Man; the large multi-strategy and risk-premia platforms of Millennium, Citadel, D. E. Shaw, Bridgewater, and AQR.These are durable organizations housing substantial systematic businesses, named as anchors rather than as evidence — we claim no sight of their internal methods, nor that their survival validates the particular discipline this essay urges. Renaissance’s Medallion is perhaps the best case in point.Medallion has long been closed to outside money, and for nearly four decades has compounded at rates no efficient-market catechism can comfortably explain — and done it with machines. Investors should note where the fund operates and how it behaves: at short horizons, where the data is thick and the feedback quick; disciplined enough to cap its own capacity and hand capital back rather than let its edge decay, on the understanding this essay shares — that every edge is finite. We claim no knowledge of its methods, which are famously secret; the public facts of horizon, capacity, and closure to outside investors — chronicled in Zuckerman, The Man Who Solved the Market, 2019 — suffice to make the point.
Among the survivors an outsider can inspect, however, a pattern repeats that has little to do with the marketing language wrapped around the complexity of the models. The strategies that survived the test of time and of scale appear to have done so by matching their models to domains where the assumptions behind modern machine learning approximately hold: very short horizons, where the data isWe bow, under protest, to the convention of the Data Age and treat “data” as a singular mass noun throughout, its Latin plural notwithstanding. thick and the feedback fast; or medium- and longer-term signals built on economically motivated foundations, with real guardrails designed around them. Those diversified across assets, horizons, and styles, with robust portfolio construction and risk management embedded rather than bolted on, have enjoyed a far longer life expectancy — and have delivered outsized risk-adjusted returns across market regimes. The same decades filled the graveyard of quantitative finance with strategies that looked magnificent on paper and died the moment real capital touched them, the preconditions for success having been ignored. Long on flexibility, short on discipline, time and again.
?Our reading is that durable systematic managers, whether housed in hedge-fund partnerships or in large institutions, kept their track records by honoring the limits this essay insists on — horizon, capacity, the finitude of any edge — rather than by chasing ever more opaque sophistication. We should qualify that inference. We read the pattern in the survivors from outside, without sight of the research processes that produced it, and the graveyard kept no registries for us to audit either — a debt to our own standards that we settle, as far as it can be settled, at the close.
This is not evidence that the tools are weak, but that they are being pointed at an environment obeying rules unlike any they have conquered before. Financial markets violate, one by one, nearly every condition on which machine learning’s greatest triumphs tacitly rested. Portfolio managers who succeed in deploying machine learning are not those wielding the largest models, but those who understand why the standard toolchain breaks here — and who engineer around it from the first line of code rather than the last.
We write neither as evangelists nor as skeptics. One of us has spent a career on the machine-learning side of this divide and knows intimately how the celebrated systems are built. The other has spent one as a quant and an economist, and knows what markets do to a naive model the instant it meets them. We agree almost entirely, and set that agreement down because the public conversation about “AI funds” has collapsed into two equally useless camps: breathless enthusiasm on one side and reflexive dismissal on the other. The truth is likely more nuanced than either, and considerably more useful to anyone deciding where to put their capital. We take the hardest case: the forecasting of returns at medium horizons — holding periods, roughly, of days to a few months — where the contemporary “AI fund” conversation and nearly all of its marketing gather (the two are not always easy to tell apart).The horizon is not a free parameter. The belief that faster is better has long been advertised in the hedge-fund space; but the short-horizon traders who navigated the pre-COVID “quant winter” best would then have lagged the medium-term trend followers through 2022 — precisely when investors needed the diversification most. Rigid strategies built around a single horizon inherit a high sensitivity to the prevailing regime. Better to build appropriate models across domains and horizons — each ingesting the data, and employing the techniques, suited to it — and then blend them, so that the whole is hostage to no single regime. We run such a fund ourselves. The fact cuts both ways. It is why we know the machinery from the inside, but it is also why nothing we say should be taken on trust. We would rather arm the reader against us than be believed on charm. Every question with which this paper closes may — and should — be turned on its authors.
?And we should say plainly, and early, what we cannot prove. We cannot demonstrate mathematically that our way of building models is superior. That the governed survive where the merely sophisticated fail is not a theorem but a conviction (though, we would argue, the most reasonable belief available in a domain where neither backtests nor live records ever deliver quick, decisive answers). The reason we argue ex modo operandi, and not ex corpore delicti, is arithmetic. At the Sharpe ratios medium-horizon strategies actually earn, a live record cannot separate a sound process from a lucky one inside any investor’s patience, and where the edge is decaying it may never separate them at all. This inverts the importance everyone intuitively attributes to a track record: for an edge that is wearing away as it is traded, the evidence does not merely accumulate slowly. It accumulates, crests, and then thins — so that beyond a point reached surprisingly early, waiting makes the case weaker rather than stronger.The claim rests on arithmetic, and the arithmetic is brutal. To distinguish a track record’s Sharpe ratio from zero at the conventional two-standard-error bar takes on the order of years under the usual idealizations of independent, identical periods — autocorrelation, fat tails, and selection all lengthen the sentence: four years of live trading at a true Sharpe of 1.0, sixteen at 0.5 — and many institutional strategies live between the two. These figures, note, buy only even odds of clearing the bar — they mark where the expected test statistic first reaches two — and comfortable power costs a little over double. The live record, in other words, cannot certify a medium-Sharpe strategy within any investor’s patience — which is why the burden of proof must move earlier, onto the engineering done before the first trade. And the record races a second clock. Let the true Sharpe decay from an opening S₀ with half-life h — the crowd arriving, the capacity filling — and the t-statistic of the full-sample mean does not merely grow more slowly: the cumulative t-statistic runs as , and so it peaks — at about 1.8h years, at a height near , and falls thereafter, the accumulating evidence diluted by an ever-flatter edge. Such a strategy can never clear the two-standard-error bar at all unless its opening Sharpe exceeds roughly — 1.5 at a three-year half-life, 1.2 at five, 0.9 at eight. Even on one of the most generous published clocks — Chordia, Subrahmanyam & Tong (“Have Capital Market Anomalies Attenuated in the Recent Era of High Liquidity and Trading Activity?,” Journal of Accounting and Economics, 2014) fit exponential trends to prominent anomalies and put the half-life of their NYSE/AMEX equal-weighted hedge portfolio at 12.8 years — an edge of Sharpe 0.5 tops out below 1.4 and is never significant at any horizon. Two caveats on the claim: this is the naive full-sample statistic, the test to which records are actually put, and a procedure that models the decay explicitly does better; and the arithmetic assumes exponential decay with independent periods, which positive serial dependence and fat tails only worsen. The moral is not that such edges are absent but that waiting is no way to prove them — past roughly twice the half-life, patience makes the evidence worse. Our arithmetic also has a trader-side twin: Da, Nagel & Xiu (“The Statistical Limit of Arbitrage,” NBER Working Paper 33070, 2024) prove that where alphas are weak and rare, estimation noise caps the Sharpe attainable by even the statistically optimal learner, at a distance from the true-alpha Sharpe that no algorithm closes — mispricing surviving because noise obscures it. What the noise denies the trader, it denies the trader’s auditor; the two bounds are one fact read from two chairs.
Judging the process is not our preference over judging the record; it is the only alternative left when the record cannot answer. We return to this claim, and to its caveats, at the close.
Our aim, then, deserves stating outright. This paper tries to do three things: to show why the gifts that powered machine learning’s triumphs elsewhere are precisely the ones the markets withhold; to describe the engineering that takes those deprivations seriously as constraints; and to hand the reader the one operating principle we hold above every other — the controls must match the method’s flexibility — together with the questions that reveal whether a manager has honored it. Much of the diagnosis is shared ground, and we say so where it is; the treatment plan is what we are here to argue. The principle returns in full when we reach the spectrum of strategies; the questions close the paper. Governance, as we use the word, has two arms: the engineering discipline that governs the model (objective, priors, constraints, uncertainty, costs) and the institutional discipline that governs the people and the evidence (registry, separation of powers, precommitment, audit). The pages that follow take them in that order — though the argument starts further back, with the discipline the machines are being asked to replace, and the floor beneath both arms is laid first.
What the quants have always done
For half a century, systematic investing has been, at its heart (or at least in its own telling), a hypothesis-first science. A researcher begins with an economic idea:We use “economic idea” and “hypothesis” loosely here. In practice the seed of a signal need not be an economic story at all — it may be a statistical regularity, a microstructural quirk, or a structural intuition — and ML-based shops in particular tend to hunt for edge more than for narrative. What earns the discipline its trustworthiness is not the economic pedigree of the idea but the ordering that follows it: a claimed edge, held with conviction, is put to the data and kept only if it survives contact with history. that cheap companies tend to outperform expensive ones, that recent winners keep winning for a while, that investors overpay for lottery-like volatility and so leave a premium in dull, steady stocks.Each of those regularities has a canonical paper behind it. Value and size, Fama & French (“The Cross-Section of Expected Stock Returns,” 1992; “Common Risk Factors in the Returns on Stocks and Bonds,” 1993); momentum, Jegadeesh & Titman (1993) and Carhart (1997); low-beta / betting-against-beta, Frazzini & Pedersen (2014); quality, Asness, Frazzini & Pedersen (2019). The fact that Harvey, Liu & Zhu (2016) were able to catalog 316 factors drawn from the published and working-paper literature is itself a caution about multiple testing. Only then does the researcher turn to the data, to ask whether the idea survives contact with history. The idea comes first. The data sits in judgment over it. This ordering is the source of the discipline’s trustworthiness. An idea with an economic reason behind it can be interrogated, argued with, and expected to persist for reasons a human can articulate.The articulability cuts both ways, and honesty compels us to say so. When an economically motivated strategy fails, there is never a shortage of equally articulate voices ready to explain, persuasively and after the fact, why the theory it rested on was flawed and why some new theory is the correct one — an account that will itself hold only until the next crisis arrives to overturn it.
✓This program has been extraordinarily successful — and its success has been its downfall.Or Untergang, if one prefers the Nietzschean register. Many discovered anomalies, once published, are gradually competed away, and the most scalable are the soonest to go. What began as a proprietary edge becomes a crowded trade, then a cheap index product, and finally a line item in a fee war.The decay has been measured. McLean & Pontiff (2016) estimate that published anomalies’ returns fall by roughly a quarter out of sample and by more than half after publication, consistent with arbitrage capital competing the premium away once it is public knowledge. What survives publication is what could not be arbitraged, not what nobody noticed. What ETFs did to long-only active management, systematic strategies are now doing to hedge funds. Medium-term trend following, considered pure alpha in the 1990s and priced accordingly (a 2% management fee and 20% of the gains), can today be had by any institution for a flat fee. The structuring desks of investment banks, and an ever-lengthening list of asset managers, now offer flat-fee exposure to mean reversion, risk arbitrage, carry, value, volatility, and dispersion, packaged as so many risk-premia products; the largest hedge funds, for their part, take their lower-alpha, higher-capacity models and offer them at a discount. As the saying goes, alpha is just undiscovered beta.For the machine-learning reader: in finance parlance, alpha and beta are the intercept and the slopes of a regression of the strategy’s excess return on one or more factor returns, themselves excess or self-financing: . The betas measure exposure to what an investor can buy cheaply elsewhere (the market itself, value, momentum); alpha is what is left once those exposures are paid for. Which factors sit on the right-hand side is a choice, and it decides the answer — which is why the saying is a claim about the model quite as much as about the market: yesterday’s alpha becomes today’s beta the moment someone writes down the factor that explains it. The march from alpha to commoditized beta is relentless, and it has left the traditional quant standing in an ever-thinner slice of genuine, un-arbitraged return. It is this squeeze (the exhaustion of the simple, human-legible ideas) that drove the industry to reach for something with more degrees of freedom. That something is machine learning.
We must stress, however, that machine learning is more than the replacement of a simple multifactor linear model with a neural network, or of a part of the investment process with an agent. It is a conceptual inversion of the entire procedure. Where the traditional quant constrains the hypothesis class aggressively before estimation, specifying a handful of rules for the data to confirm, a machine-learning model is handed thousands of inputs and left, to a remarkable degree, to discover the rules on its own — including subtle, non-linear relationships no human would ever think to write down. The two traditions are less two species than the ends of one dial (how much of the specification the researcher writes, and how much the data is left to choose), and it is the position of the dial, not the label on the software, that decides how much validation the result is owed. A signal that holds only for certain firms, only in certain regimes, only in certain seasons (a conditional sentence with a dozen clauses), is beyond a human to hold in mind, let alone a thousand such signals at once, yet a flexible model finds them by the bushel. Value (to invent a plausible example) might pay better in small companies than in large — but mainly at the bottom of the cycle, once credit spreads have widened and begun to compress, and once an earnings beat has set short-term momentum going: one such conditional sentence among the thousand the market is presumably whispering at any moment. Here lie both the promise and the peril — the same property viewed from two sides. The freedom that lets the model see what the human cannot is the same freedom that lets it “learn” patterns that were never there — coincidences of the particular history it was shown, dressed up as persistent, exploitable regularities.
Every strategy lies on a spectrum. At one end are the simple, transparent, commoditized rules (a value tilt anyone can replicate for a few basis points); at the other, the complex, opaque, expensive models whose behavior even their creators struggle to narrate. Traditional factor investing sits near the transparent end; the systems this paper concerns live near the opaque one. The single most important principle for evaluating a manager on this spectrum is easily stated and almost universally ignored: the controls must match the method’s flexibility. A rigid, few-parameter strategy can be governed loosely, because it cannot wander far. A flexible, many-parameter model handed the freedom to find anything must be surrounded by correspondingly serious machinery to prevent it from finding nonsense. Most of the disasters we will describe are, at root, a violation of this one principle: enormous flexibility, guarded by controls built for something far more docile.
The four unmerited gifts the markets refuse to give
None of the advances of the last decade — in vision, language, speech, protein structure, or human gamesThe history of machines conquering human games is more tangled than the slogan. In chess, superhuman play predates modern machine learning, and even in Go the first superhuman system leaned on human wisdom rather than discovering the game from scratch: IBM’s Deep Blue defeated the reigning world chess champion, Garry Kasparov, in 1997 using massive alpha-beta search over custom hardware and an evaluation function hand-tuned from grandmaster knowledge, with no learning driving its play; and even AlphaGo, which beat Lee Sedol in 2016, was bootstrapped on a large corpus of expert human games. The genuine break came with AlphaGo Zero (Silver et al., Nature, 2017), which reached superhuman strength from self-play alone, with no human game data, and in the process produced strategies no human had ever played — the closest thing in this story to a machine being “creative.” Much of what is loosely filed under “AI conquered games” is thus as much about search and human-encoded wisdom as about learning; AlphaGo Zero is the instance that most deserves the name. — required controls of the severity just demanded. Why, then, does the same machinery so often drown in finance? The answer is unflattering. Those victories, for all their variety, shared four conditions — and medium-horizon return forecasting, the arena of this essay, receives far less of each.
✓The first is data, in almost unfathomable abundance. A model learning to recognize a cat is shown millions upon millions of labeled photographs, and can be shown millions more on demand. The markets, by contrast, are miserly. It is tempting to point at the tick-by-tick torrent of modern exchanges and declare data is not an issue, but this confuses volume with information. What a medium-horizon strategy actually learns from is something closer to the number of independent episodes the market has lived through: distinct regimes, distinct crises, distinct decades. By that measure the record is thin — tens of episodes, and, counted in the overlapping observations a medium-horizon label actually consumes, a few thousand correlated ones. A rounding error, by the standards of computer vision (not even mentioning the standards of large language models).What governs learning is the effective sample size, not the raw number of samples. Overlapping return windows and labels — and the persistence of volatility and regimes — mean that daily bars carry far fewer independent degrees of freedom than their count implies: the returns themselves are nearly serially uncorrelated, but the labels built on them are anything but. By that accounting, decades of history compress into an effective sample whose order is thousands, not millions — the exact figure depending on the question asked of the record: on the horizon, the overlap, the factor structure, and the estimator to which it is put. And the affliction is not confined to the calendar. One is tempted to recover the lost degrees of freedom by counting across the market rather than along it — thousands of names at every instant — but the cross-section is correlated exactly as time is: a handful of common factors move most names together, so the effective breadth of a day’s level returns is a small multiple of the number of its leading principal components, not the number of tickers — while the residual those components leave behind is another matter, and is where the cross-sectional bet lives. The traditional quant will recognize the quantity: it is the breadth of Grinold & Kahn’s fundamental law of active management — the information ratio growing as skill times the square root of the number of independent bets — whose perennial abuse is to count the tickers rather than the bets. The law’s later refinement supplies the name for a price these pages concede repeatedly and nowhere quantify: Clarke, de Silva & Thorley (“Portfolio Constraints and the Fundamental Law of Active Management,” Financial Analysts Journal, 2002) insert a transfer coefficient — the correlation between the positions one would hold unconstrained and those one actually holds — so that the information ratio becomes skill, times the square root of breadth, times the fraction of it the constraints let through. That fraction is measurable; it is what question 12 of our closing list asks a manager to produce, and it is the honest denomination of the price. Machine learning changes the arithmetic of skill; it leaves the law’s discipline intact, for no architecture manufactures independence the market has not supplied. ImageNet, by contrast, offers around 1.2 million labeled images — and note that this is a small dataset by present standards, already modest in 2012 beside the web-scale corpora then being assembled and utterly dwarfed by the billions of examples routine in training today’s foundation models. The point is revealing: markets fall short by several orders of magnitude not of the frontier, but of a benchmark the field already considers quaint. The gap is not a detail; it is the whole problem in miniature. We are, in every way that matters, data-starved. One qualification, developed below: the starvation is not uniform across the questions one asks of the record, and part of the discipline lies in asking only what the data can afford to answer. And the starvation bites hardest exactly where the appetite is greatest. Economists and quants have always loved to linearize (a line, after all, needs only two points plus an act of faith), but the world is stubbornly non-linear. It is also stubbornly conditional, and a model’s appetite for data grows with the flexibility it is granted. The subtle, regime-dependent relationships most worth finding are precisely the ones a few thousand observations cannot hope to feed. A strategy that blindly adopts the latest technique without first budgeting for this starvation is not being bold; it is arranging its own failure out of sample.
✓The second is a world that holds still — or at least one whose ground truth does. A cat photographed tomorrow or from a different angle is still a cat, and where the camera drifts, fresh labels can be gathered to prove it; a game’s rules do not move at all. A trading pattern enjoys no such stability. Regimes of policy and of appetite succeed one another akin to phase transitions in matter (the same molecules, abruptly different laws); the microstructure of the exchanges is rebuilt every decade; the population doing the trading is replaced wholesale. Nobody announces any of it. The rules drift even when no one is watching, and a relationship learned in one era arrives at the next already out of date. Investors will remember how, over the last decade, markets oscillated, at times violently, between macro regimes in which good news was good news and regimes in which good news was bad news, monetary policy being the usual culprit (and, on the worst days, the only culprit anyone could name). A model’s architects must therefore build in the safeguards that let it de-lever when the environment turns hostile — in realized returns and in expected ones — rather than blindly target one level of risk across all regimes.
?Non-stationarity also changes what we should mean by an alpha. The trade’s habitual ideal is the evergreen signal — one that works more or less continuously in the record and is expected to keep working until it decays — and we suspect that ideal is partly an artifact of how strategies are sold. A signal that declares in advance the conditions under which it should work, and is silent otherwise, is worth more to a governed process than one claimed to work always: it can be sized by whether those conditions hold, falsified when it fails inside them, and retired without argument.The claim that the market’s regularities are conditional is not only ours. Stambaugh, Yu & Yuan (“The Short of It: Investor Sentiment and Anomalies,” Journal of Financial Economics, 2012) find a broad set of anomaly returns to be state-dependent: they strengthen following periods of high investor sentiment, and the strengthening arrives almost entirely on the short leg, where the impediments to arbitrage are greatest, the long leg showing no such relation. The finding is descriptive where ours is normative, so it carries only half our weight — that the regularities are conditional, not that a signal declaring its conditions is worth more than one that does not. It carries the half that can be checked. The evergreen signal is the exception the record does not much support, and a search built to find one is looking for the rarer thing. We are convinced the market’s regularities are conditional far more often than they are permanent. The conditional is also, we think, where more of the remaining gold lies (the shallow seams having been worked out some decades ago, and by better-equipped diggers). However, a search that admits conditioning has a larger haystack to work in — and more ways to fool oneself while searching it, each declared regime being one more fitted choice.
✓The third is a favorable signal-to-noise ratio. When a model transcribes speech, the words tower over the background hiss. In markets the proportions are inverted and then some: the predictable component of tomorrow’s return is a whisper in a gale, a faint regularity almost entirely buried under the crash and roar of essentially random fluctuation.Concretely, the monthly out-of-sample R² for individual-stock return prediction is a small fraction of one percent: Gu, Kelly & Xiu (2020) report roughly 0.3–0.4% for tree- and neural-network models, against 0.16% for a linear benchmark restricted to size, book-to-market, and momentum (their Table 1, the monthly stock-level panel; portfolio-level exercises fare better, but the stock-level figure is the honest measure of the raw signal). Even excellent strategies win by a slender edge repeated over an enormous number of bets, not by being reliably “right.” The practitioner’s version of the same fact is blunter. Renaissance’s Robert Mercer, of the firm that has compounded at rates no one else has matched: “We’re right 50.75 percent of the time… but we’re 100 percent right 50.75 percent of the time. You can make billions that way,” as Zuckerman records. A fractional edge, repeated without end: the discipline these pages describe is about surviving repetition, not about being right. And (perhaps more importantly) the ratio itself refuses to hold still: the whisper waxes and wanes, and a serious strategy accounts explicitly for when the whisper can be heard at all. Most do not.
✓The fourth is a target that does not move because you predicted it. The benchmarks of vision and speech sat still while they were being conquered: a cat does not learn to evade the classifier that found it, and a Go board does not rearrange its rules upon losing.The conquest of games is the example most often invoked against this. The rules of chess and Go are perfectly stationary — fixed for centuries, identical tomorrow and today — so a strategy learned against them never goes stale. A market’s mechanics change slowly and in the open; its payoffs change fast and in secret — the game is repriced the instant you learn to play it. Superhuman play in a stationary game is therefore weak evidence of anything in a non-stationary market. Put most compactly, what every one of those conquered domains supplied was cheap, verifiable feedback — a simulator or a ground truth against which any candidate could be scored, at will and without consequence. Self-play manufactures unlimited data because the game can be replayed; the market grants replay of only one history — 2008 can be rerun forever, and it remains the same 2008; what is withheld is independent replay. The fashionable rejoinder — generative market simulators, agent-based markets, synthetic histories — concedes the point while seeming to refute it: a simulator rich enough to contain undiscovered alpha would have to be fit to the same scarce, over-mined history, importing the very overfitting it was meant to escape. Where the mechanics are known — an order book’s rules, the execution problem — simulation genuinely helps, which is where these pages endorse it; where the object is the undiscovered regularity itself, a simulator is a prior by another name. The econometrician’s version of the missing replay has a name and a distinguished literature — synthetic control, which manufactures the counterfactual for a policy question from a weighted chorus of donor units (Abadie & Gardeazabal, American Economic Review, 2003; Abadie, Diamond & Hainmueller, Journal of the American Statistical Association, 2010), kin to the natural-experiment program the 2021 Nobel honored — and it is instructive here because its precondition fails: the market offers no donor pool, no parallel unit that did not live through 2008. The simulator literature deserves its names too: agent-based markets from the Santa Fe tradition to modern high-fidelity engines (LeBaron, Handbook of Computational Economics, 2006; Byrd, Hybinette & Balch, “ABIDES: Toward High-Fidelity Multi-Agent Market Simulation,” ACM SIGSIM-PADS, 2020), generative models of price series (Wiese, Knobloch, Korn & Kretschmer, “Quant GANs: Deep Generation of Financial Time Series,” Quantitative Finance, 2020) — and its own practitioners are candid about the pitfalls (Assefa et al., “Generating Synthetic Data in Finance: Opportunities, Challenges and Pitfalls,” ICAIF, 2020). The rehearsal is real; the new information is not. A trading pattern enjoys no such courtesy. It begins to decay as capital arrives against it, and it decays because it was found: the market is not a passive dataset waiting to be characterized but a crowd of intelligent adversaries (many of them better capitalized, and some of them simply better) who will happily take the other side of your cleverness and, in doing so, erase the very edge you discovered. In few other applied fields does the act of learning something reliably destroy the thing learned (the closest cousin is antibiotic resistance, and we come to it at the close). We gave at the outset the arithmetic of that erosion and what it does to any hope of proving an edge from the record. The rule has exceptions, and they matter: a return that compensates risk deliberately borne, a premium to supplying liquidity or balance sheet, an edge protected by limits to arbitrage or by frictions the marginal investor cannot cross — these can survive being known, because what sustains them is not ignorance. The decay law governs the rest — the mispricings, the crowded signals, the exploitable microstructure — and the allocator’s question is therefore not only whether an edge exists but which kind it is.Goodhart’s law — in Strathern’s now-common paraphrase, when a measure becomes a target, it ceases to be a good measure — is its nearest kin among named social-science regularities. The economist’s canonical statement is the Lucas critique (Lucas, “Econometric Policy Evaluation: A Critique,” Carnegie-Rochester Conference Series on Public Policy, 1976): a relationship estimated under one regime does not survive a policy built to exploit it. The machine-learning literature has since named the mechanism performative prediction (Perdomo, Zrnic, Mendler-Dünner & Hardt, ICML 2020), in which deploying a model shifts the very distribution it was trained to predict; markets are its most unforgiving instance. Even in Goodhart’s case the measure is merely corrupted rather than consumed. Within finance, Lo’s Adaptive Markets Hypothesis (Journal of Portfolio Management, 2004) gives the ecology its proper name: strategies as species, alpha as a resource depleted by its own consumers, efficiency as the equilibrium of a well-hunted ground.
The adversary works on two clocks, and it helps to keep them apart. At the shortest horizons the response is personal and swift: a footprint large enough to notice is noticed, gamed, and traded against — the market quite literally learns you. At the horizons where most asset managers live, the mechanism is slower and nothing personal: your own footprint pushes prices against you; then come crowding and, eventually, publication (the scholarly literature being, from a trading desk, a very slow leak), eroding the anomaly for everyone at once, the way a fishing ground is exhausted by a fleet rather than by any rival’s malice. Both clocks are adversarial; only the first has your name on it. And the slower is the more treacherous precisely because it is slower: it pays out years of comfortable returns in which to forget that it is coming. The machine is measured against a price, and a price is not nothing: it is the pooled, battle-tested judgment of every well-paid participant who traded before you — and of everyone trading with or against you at this very moment — assembled by the very adversaries who will reprice the game the moment you learn it. To beat it you must be not merely good but reliably better than a consensus that has already priced in everything easy.
Abundant data, a stationary world, a favorable signal, and a target indifferent to being predicted: these were the four tacit gifts behind the headlines of AI’s triumphant conquests of one domain after another. The markets, at these horizons, grant each only grudgingly. The four are not independent. The adversary is one of the engines of the drift, repricing the rules as fast as they are learned, while policy, technology, and the slow replacement of the trading population supply the rest. But they fail separately often enough to deserve separate names. Everything that follows is a direct consequence of these four deprivations.The diagnosis has company from inside the profession. Israel, Kelly & Moskowitz (“Can Machines ‘Learn’ Finance?,” Journal of Investment Management, 2020) name much the same conditions — a dataset that grows only as time passes; a signal-to-noise ratio held low by the very trading that exploits any signal; a data-generating process that changes because agents act on it — and observe, as we will, that the data-rich problems of implementation and risk — reward flexible models long before return prediction does. A reader will recognize three of the gifts and the outline of the fourth. What these pages add is what follows from taking the fourth seriously: the adversary as a deprivation in its own right, the arithmetic it forces on the track record, and governance built into the model rather than counseled around it — the difference between a diagnosis and a treatment plan.
A shining lie
✓The gravest of those consequences is a single, characteristic failure, and it arrives almost inevitably: the backtest flatters by construction.In this case, all four deprivations are culpable of the flattery, and the division of labor is interesting to analyze. Let us write the flattery as the ratio F of the Sharpe ratio a selected backtest shows to the one the portfolio then earns. With N attempts against an effective history of T years, a true edge S of which a fraction ϱ survives from the studied regime into the traded one, and a half-life h once deployed, after t years of live use. We implore our readers to treat this formula as merely a stylized, qualitative mathematical illustration rather than a real quantitative estimator. Having said that, every term in it is real — the error of a measured Sharpe ratio is Lo’s arithmetic (“The Statistics of Sharpe Ratios,” Financial Analysts Journal, 2002), and the luck of the best of N, with its exact finite-N constant, is Bailey & López de Prado’s (“The Deflated Sharpe Ratio,” Journal of Portfolio Management, 2014), to which note 39 returns — and each deprivation owns a seat. The first factor couples scarce data and the faint signal — the luck against the prize — and dies whenever the prize outgrows the luck; is the world that moved; is the target that moved because you predicted it — the one factor that grows after the research ends, and on the same clock by which note 9 prices the evidence. Plugging in nothing exotic — ten thousand attempts, ten effective years, an edge of one half, and the decay literature’s own survival and half-life, four fifths and four years (McLean & Pontiff, “Does Academic Research Destroy Stock Return Predictability?,” Journal of Finance, 2016) — the flattery two years into live trading is better than sixfold, of which the luck alone is nearly four. And this is the charitable reading, for it assumes the right strategy won its own search — which, at these numbers, it does about once in seventy times; in the remaining sixty-nine the winner is a dud and the earned Sharpe is nothing at all. Halving the backtest Sharpe, the industry’s rule of thumb, is generosity rather than caution — the haircut done properly is Harvey & Liu’s (“Backtesting,” Journal of Portfolio Management, 2015), and it is rarely so kind. Grant instead an excellent edge of one and a half — large enough that the winner of the search is, four times in five, the right strategy rather than a lucky one — and the flattery is still better than threefold, its floor set by the two factors indifferent to the quality of the edge: the moved world and the reacting target. And the limit tells the other half of the story: let the data or the signal grow, ϱ approach one, and the half-life lengthen, and F falls to one factor by factor — the same procedure, run where the gifts are granted, is called model selection, and machine learning performs it daily without scandal. While the selection procedure is not pathologic per se, its failure is caused by the application in an environment deprived of the assumptions it tacitly relied on. Give a flexible model enough freedom and enough attempts against a fixed slice of history, and it will always find something that looks like skill. Try ten thousand strategies against the same past and the best of them will glitter — not because it understood anything, but for the same reason the luckiest of ten thousand coin-flippers looks like a prophet after an afternoon of tossing. Line up ten thousand people, have each flip a coin ten times, and someone will almost certainly flip ten heads.The ten thousand flippers are not a figure of speech. A single flipper’s chance of ten heads in a row is 1/1024, under a tenth of one percent. But the chance that at least one of ten thousand independent flippers manages it is — a near-certainty. The apparent prodigy is manufactured entirely by the number of attempts, not by any skill in the flipper. That flipper is no prophet. He is the tail of a distribution you were bound to draw once you sampled it ten thousand times.This is the multiple-comparisons problem. Under N independent trials, the expected maximum in-sample Sharpe ratio rises steadily with N even when every true Sharpe is exactly zero — the curve of expected best-Sharpe against number of trials is the plot David Bailey has called the most important in finance. A selected backtest is a draw from the distribution of the maximum, not the mean, which is why the raw number is so badly misleading.
The backtest that results from this process is not a forecast. It is a flattering portrait, painted entirely with hindsight, of a strategy that does not yet exist. In a sufficiently flexible search this is not a rare accident that befalls the careless but the default outcome — the thing that happens all by itself whenever nobody is actively working hard to prevent it. It should be presumed selection rather than skill, produced by luck until you have gone to considerable lengths to prove otherwise. The industry’s machine-learning disappointments are mostly not frauds and not failures of intelligence. A large share of them — larger, we suspect, than anyone cares to count — are this single statistical fact, misunderstood.In fairness, the machine-learning field is itself often cavalier about statistical significance: a decade of incremental fractions-of-a-percent improvements on benchmarks such as ImageNet may, in aggregate, partly reflect collective overfitting to a fixed test set — though the evidence cuts both ways: test sets built afresh for ImageNet show accuracies dropping but the ranking of models largely preserved (Recht, Roelofs, Schmidt & Shankar, “Do ImageNet Classifiers Generalize to ImageNet?,” ICML 2019). Vision has a saving grace, however — one can nearly always gather more images and test genuinely out of sample. In markets, that escape is closed: to test out of sample is to trade live and wait, often prohibitively long, before the result is statistically meaningful. The lived record of the early machine-learning funds tells the same story, from both of its endings. Sentient Technologies, launched on evolutionary algorithms, traded from late 2016, returned four percent in 2017 and nothing in 2018, and was liquidated that year; Aidyia, a Hong Kong–based machine-learning fund founded around 2011, began live trading only in January 2016 — the founding and start dates from contemporaneous reporting — and survived less than a year of it (Buczynski, Cuzzolin & Sahakian, “A Review of Machine Learning Experiments in Equity Investment Decision-Making: Why Most Published Research Findings Do Not Live Up to Their Promise in Real Life,” International Journal of Data Science and Analytics, 2021). And Voleon, among the first of them — founded in 2007, and grown to some $29 billion in regulatory assets by its 2026 Form ADV — lost money through its first two years of live trading before its edge showed itself (the firm’s later ascent is chronicled in “Why hedge fund managers are happy to let the machines take over,” Financial Times, October 17, 2019): even the survivor took years to be told apart from the graveyard’s tenants — the arithmetic of the live record, with a name attached.
When such a strategy reaches an allocator, the tell is almost always the same: a track record with no economic account of itself (an account owed, we will argue, at the level of the portfolio and its persistence, not of the weights), returns that cannot be attributed to any mechanism a person can state out loud. Regulators have begun to notice the gap between a manager who says the coveted words “we use AI” and one whose process earns them, and recent enforcement actions have put the point on record.The record now includes penalties. The SEC’s first “AI-washing” enforcement actions, of March 2024, were settled against the investment advisers Delphia and Global Predictions for false and misleading statements about their purported use of AI and machine learning (SEC Press Release 2024-36; civil penalties of $225,000 and $175,000 respectively). What was charged in both cases was the claim, not the technique. But enforcement polices only the claim. Policing the substance remains, and will remain, the allocator’s own work. Exposing that gap requires knowing what the substance is supposed to look like. What was tried, and how many times? What does the model see that the market has not already priced? How would its owners know the day it stopped being true? A manager who cannot answer has, in effect, answered. The rest of this paper is our attempt to describe the substance to probe.
Where machine learning genuinely earns its place
None of this argues against machine learning in finance. It argues for using it where its assumptions approximately hold, and treating those pockets with respect, rather than spraying the technique across a problem that punishes indiscretion. The discipline lies largely in matching the tool’s flexibility to what the available data can honestly support. In several places the match is good.
The first and least controversial is as an instrument of measurement rather than an oracle of prediction. Reading a sentiment from a news wire, extracting a number from a scanned filing, estimating footfall from a satellite image of a car park, inferring spending from a wash of anonymized transactions: here the model is not being asked to forecast a price at all, only to convert unstructured reality into a clean, structured measurement that a human-designed strategy can then reason about. The signal-to-noise problem is far gentler; the ground truth is often checkable (someone can, in principle, go and count the cars). This is machine learning as a superb pair of eyes, not as a prophet.
The second is the short horizon, where the data actually is abundant and the feedback is fast. In execution (the unglamorous business of buying and selling without unduly moving the price against yourself) and in the highest-frequency corners of trading, a model sees millions of genuine events and learns from its mistakes within seconds, minutes, or hours rather than decades. Here the markets come closest to granting the first of their withheld gifts. Even here, it should be said, the daily workhorse is often classical statistics rather than anything fashionably deep: stationarity and unit-root tests and their sequential variants, the cointegration machinery, the estimated half-life of a mean-reverting spread. The abundance of data is what finally lets those old instruments bite, and the learned models are layered upon them rather than replacing them.The instruments are the econometrician’s oldest: unit-root and stationarity tests — Dickey & Fuller (1979), Phillips & Perron (1988), KPSS (Kwiatkowski, Phillips, Schmidt & Shin, 1992) — the variance-ratio test of Lo & MacKinlay (1988), and the cointegration machinery of Engle & Granger (1987) and Johansen (1988), with the half-life of an Ornstein–Uhlenbeck spread as the practitioner’s summary statistic. At medium horizons these tests starve with everything else; at the shortest, the data is finally thick enough to feed them — and thick enough to reveal how quickly their verdicts drift. Some techniques (reinforcement learning chief among them) have shown themselves able to navigate highly non-linear feedback loops within realistic simulated market microstructure.One demonstration is clean enough to state in full. Cheridito & Weiss (“Reinforcement Learning for Trade Execution with Market and Limit Orders,” Quantitative Finance, 2026) train an agent against a simulated limit-order book, and it learns when to cross the spread and when to rest a limit order. It can learn this because execution grants what forecasting withholds: millions of simulated episodes in a structured environment, and feedback within minutes. The sandbox is not inert, either: its counterparties do react to one’s own flow, the tactical traders responding to the imbalance one’s orders create. But they react by fixed rules rather than by learning to hunt one as a living adversary would. The fourth deprivation is softened inside the sandbox, not abolished.
The third is the combination of many individually feeble signals. No single weak predictor is worth trading. But how a hundred of them should be weighed and blended (with all their non-linear interactions and mutual redundancies) is a question a flexible model answers better than a human with a spreadsheet or a linear specification — provided the raw signals were themselves independently motivated: economically, behaviorally, structurally, institutionally, or microstructurally.What does a flexible model add when every ingredient is weak? Gu, Kelly & Xiu (“Empirical Asset Pricing via Machine Learning,” Review of Financial Studies, 2020) document that gradient-boosted trees and shallow neural networks, handed hundreds of individually feeble predictors, outperform the linear benchmarks. The gain traces to the non-linear interactions and redundancies among signals that no researcher could enumerate by hand for a spreadsheet or an ordinary regression. The machine’s contribution is the combination, not the ingredients.
And the fourth is the machinery of risk and portfolio construction itself: estimating how assets will move together, how a shock will propagate, how a portfolio should be shaped to survive it. These are measurement and modeling problems more than they are forecasting problems (nobody here is being asked what the market will do tomorrow), and they reward the technique fairly.The same point holds for the risk machinery, and there statistical learning wears its plainest clothes — regularized estimation rather than deep networks. Shrinkage estimation of the covariance matrix in the tradition of Ledoit & Wolf (“Honey, I Shrunk the Sample Covariance Matrix,” Journal of Portfolio Management, 2004), and allocation schemes such as López de Prado’s hierarchical risk parity (“Building Diversified Portfolios that Outperform Out of Sample,” Journal of Portfolio Management, 2016). Both stabilize the portfolio machinery by asking less of the data than a full inverse-covariance estimate would demand. Asking less of the data is the whole of the technique here.
~Machine learning earns its place wherever the problem is rich in data, checkable against reality, or modest in what it asks the model to predict. It forfeits that place the moment it is asked to conjure a confident forecast of tomorrow’s return from a few decades of noise. To predict an asset’s future return as a naked point estimate is, at these horizons, the hardest and least stable version of the problemWe mean here the medium-to-lower frequencies — daily horizons and beyond — at which most asset managers operate. The highest frequencies are a different regime: there, as noted above, automated statistical techniques dominate and machine learning genuinely earns its place. The two facts are connected at one remove: latency competition manufactures efficiency around the moments of price formation, while a slow anomaly is eroded by the slower machinery — capital arriving, the result published, the trade commoditized. The record concurs: as markets have grown more efficient, alpha at the longer horizons has become harder to extract. Post-earnings-announcement drift is the canonical example — a stock’s price once kept drifting in the direction of an earnings surprise for up to sixty days, a window modern pricing efficiency has compressed to nearly nothing — and Martineau (“Rest in Peace Post-Earnings Announcement Drift,” Critical Finance Review, 2022) finds the phenomenon has faded so far that it may have disappeared outright in recent years. The horizon at which an anomaly could once be harvested has itself been closing, which is why the technique must be matched to the clock rather than to the fashion. — and, less obviously, it is often the wrong one to ask. The escape is not from prediction altogether. A rank, a conviction score, a volatility forecast, and a tail estimate are all predictions, only coarser and sturdier ones,The estimator that makes the point is Han’s maximum rank correlation, which maximizes agreement in sign over pairs and is consistent under conditions far weaker than the parametric model would need (Han, “Non-Parametric Analysis of a Generalized Regression Model,” Journal of Econometrics, 1987), with its asymptotic normality established through the theory of U-processes (Sherman, “The Limiting Distribution of the Maximum Rank Correlation Estimator,” Econometrica, 1993). All of this lives inside the monotone single-index setting stated above, and claims nothing about ranking against regression in general. Placing it beside Manski’s maximum score estimator — every bit as ordinal, and cube-root convergent at — locates the cost precisely: what is expensive is not ordinality but the sharp edge of an objective built from indicators of a single observation. Smooth that edge with a kernel and the rate improves to on the very same ordinal target — and, under stronger smoothness, arbitrarily close to the parametric rate (Horowitz, “A Smoothed Maximum Score Estimator for the Binary Response Model,” Econometrica, 1992); average over pairs, which smooths it implicitly by replacing each indicator with its conditional expectation, and the parametric rate, , returns in full. In this comparison, ordinality costs nothing; the sharp edge of the criterion costs a good deal. coupled directly to sizing and abstention and stripped where possible of the common noise the record is least able to resolve.
A diligently engineered pipeline can win on markedly better-posed problems: ranking assets against one another,That the relative question is the easier one is not merely our intuition; under conditions one can state, it is a theorem. Where the outcome is a latent score seen through an unknown increasing distortion of the scale and the decision reads only the ordering, the rank vector is the maximal invariant of that family: every scale-blind procedure is a function of the ranks alone, and nothing the decision could have used is discarded (Lehmann & Romano, Testing Statistical Hypotheses, 3rd ed., 2005, ch. 6). Survival analysis has lived off this for fifty years — Cox’s partial likelihood is, in the uncensored case, the marginal likelihood of the ranks, recovering the coefficients at the parametric rate while the baseline hazard, an infinite-dimensional monotone nuisance, is never estimated at all (Kalbfleisch & Prentice, “Marginal Likelihoods Based on Cox’s Regression and Life Model,” Biometrika, 1973). And what follows is a rate, not a preference. Where the ordering is generated by a one-dimensional score, ranking spares one the monotone link entirely, whatever the complexity of the score itself; where that score is a linear index — the regime in which a cross-sectional alpha model is conventionally written down, and an approximation whose bite fades as the true structure departs from it — the saving can be named exactly. In that monotone single-index model the direction that fixes the ordering is estimable at the parametric rate , the error falling like one over the square root of the sample, while the conditional mean is estimable no faster than , the gap between them the price of the nuisance (Balabdaoui, Groeneboom & Hendrickx, “Score Estimation in the Monotone Single-Index Model,” Scandinavian Journal of Statistics, 2019). What buys this is invariance together with the pairwise form of the criterion, not the coarseness of the target: Manski’s maximum score estimator, as the previous note records, is every bit as ordinal and converges only at , with a limit that is not even Gaussian (Kim & Pollard, “Cube Root Asymptotics,” Annals of Statistics, 1990) — which is also the better explanation of why, in the ranking literature, the pairwise and listwise formulations so often outperform the pointwise one, which regresses the values and ignores both relative order and a document’s position in the list (Burges et al., “Learning to Rank using Gradient Descent,” ICML 2005; Cao et al., “Learning to Rank: From Pairwise Approach to Listwise Approach,” ICML 2007; Liu, “Learning to Rank for Information Retrieval,” Foundations and Trends in Information Retrieval, 2009). The robustness is of a piece: the sign-based rank statistics read only the order of a pair and so need no moments at all, and the normal-scores procedures — which read rather more than signs — never lose efficiency to the Gaussian-optimal one and gain against it almost everywhere, in the location and linear-model settings where that has been proved (Chernoff & Savage, Annals of Mathematical Statistics, 1958, for two-sample location; the dominance has since been carried to linear models and to multivariate and serial settings — Paindaveine, “A Unified and Elementary Proof of Serial and Nonserial, Univariate and Multivariate, Chernoff–Savage Results,” Statistical Methodology, 2004), while least squares degrades once the errors run short of moments (Han & Wellner, “Convergence Rates of Least Squares Regression Estimators with Heavy-Tailed Errors,” Annals of Statistics, 2019) — and returns have a finite variance while their fourth moments are estimated so unstably that no finite sample settles whether they exist at all. Empirical asset pricing has sorted on ranks since Fama and MacBeth, though its machine-learning descendants rank the predictors and then regress the level of the return — the invariance kept where it costs little and surrendered where it pays. sizing conviction, forecasting a return’s volatility or its tail rather than the number itself.
The cross-section, as the first deprivation’s note conceded, is no more independent than time. A few common factors move most names together, and counting tickers escapes the correlation no better than counting days. The virtue of the relative question is not that it draws on more data but that it draws on better-posed data. Ask which asset will beat which, and the market level cancels (the largest, most violently non-stationary and least predictable component of the answer, and the one most correlated across names). The styles do not cancel. Their loadings differ across names by design, which is why they need the exposure budget of a later section rather than the ordering. Even for the market the cancellation is exact only where the loadings match, weakening as they spread. That makes the remedy a matter of degree rather than of principle: orthogonalize far enough to narrow that spread, and no further. What remains, once matching or explicit neutralization has done its work, is a cross-sectional residual at once higher-dimensional than the common level and steadier than it. The same correlation that made the naive headcount a lie is what makes the relative question affordable. The absolute question (in or out of the market, now or later) keeps the worst of the data and asks the impossible of it; the relative question discards the worst and asks what the record can answer. The residual is not costless. Netting the common factors out can amplify idiosyncratic noise, raise turnover, and lean on a factor model that is itself unstable. Choosing between them is where the principle of matching controls to flexibility does its most concrete work, in the choice of question rather than of controls.The advantage has three bounds, the last of them against ourselves. The decision must read only the ordering: the moment sizing reads magnitude the invariance is gone and the cube-root object returns — which is why we claim the advantage for selection and not for sizing, the uncertainty apparatus of the pages that follow governing the second question, and nothing here proposing that a book be built from ordering alone. Rankings do grow unstable near ties, and quantifiably so: the comparisons needed multiply as the inverse square of the separating margin, so the advantage collapses rather than fades. At the signal the cross-section actually offers — the monthly stock-level fit reported earlier — most of the middle of the book sits in that near-tie regime, which narrows the claim rather than defeating it: it is about aggregate agreement among the extremes a book actually trades, never about recovering an ordering exactly (Mao, Weed & Rigollet, “Minimax Rates and Efficient Algorithms for Noisy Sorting,” Proceedings of Algorithmic Learning Theory, 2018). And regressing the values and sorting them afterward is not wrong, merely expensive: the sharpest transfer known reaches a given ranking accuracy only by paying its square in regression accuracy (Cossock & Zhang, “Statistical Analysis of Bayes Optimal Subset Ranking,” IEEE Transactions on Information Theory, 2008; Agarwal, “Surrogate Regret Bounds for Bipartite Ranking via Strongly Proper Losses,” Journal of Machine Learning Research, 2014). A detour with a toll.,One of us has met the pattern at first hand outside finance: finger flexion from forearm ultrasound — the control problem of a prosthetic hand. Learned as value regression, on continuous kinematic labels, it worked and would not travel: the map from signal to angle had to be recalibrated for every wearer (Zadok, Salzman, Wolf & Bronstein, “Toward Predicting Fine Finger Motions from Ultrasound Images via Kinematic Representation,” ICRA, 2023). Learned instead as a ranking — which of two sequences carries the greater motion — it traveled, and the successor system reads a forearm it has never seen with no calibration at all (Zadok, Wolf, Bronstein & Salzman, “SonoRank: Toward Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences,” IROS, 2026). The calibration-free of that title is the theorem in miniature: the per-wearer map from signal to angle is the unknown distortion, and the rank criterion deletes it rather than estimating it. Worth noting how the ordering earns its keep there — as the pretraining objective, with the flexion decision read off the representation afterward. The ordinal target need not be the final output to be the thing that makes the problem learnable. Importantly, sorting itself is not differentiable — the ranks are integer-valued — and was long approached through surrogates rather than optimized outright — pointwise first, then the better pairwise and listwise objectives described in the previous note. Recent advances made (smoothed) sorting and ranking into a trainable layer, sorting cast as an entropically regularized optimal-transport assignment (Cuturi, Teboul & Vert, “Differentiable Ranking and Sorting Using Optimal Transport,” NeurIPS 2019), with a top-k variant apt for selecting the leaders of a cross-section (Xie et al., NeurIPS 2020) and a faster projection-based cousin (Blondel et al., ICML 2020). Where the ordering is the whole of the target, that machinery lets it be optimized directly rather than approached through a proxy — the sturdier target not merely chosen but built into the objective itself.
?And it is here, we should say plainly, that a fifth place is earned: the medium horizon itself — uninhabitable by nature, made habitable by engineering — the very pocket in which we ourselves operate, and the reason the rest of this paper is about the engineering. Of the five, this last alone is contestable. Here alone all four deprivations bite at once: the data is sparse, the world drifts, the signal is faint, and the target answers back. The other four a skeptic grants outright. This one we can only argue for, and (as the closing pages concede) hardly prove. The argument needs the engineering in view, so we make it once the engineering has been set out.
The floor
If a shining backtest is the null result, the first duty of a serious process is to subject it to relentless suspicion. The problem, it should be said, is older than the machines: financial econometricians have warned for decades that repeated use of the same historical data, expansive specification searches, and selective reporting can make chance discoveries look statistically — and economically — persuasive. They also built much of the machinery for catching the trick.The correction has a literature of its own. Sullivan, Timmermann & White (“Data-Snooping, Technical Trading Rule Performance, and the Bootstrap,” Journal of Finance, 1999); White (“A Reality Check for Data Snooping,” Econometrica, 2000); Hansen (“A Test for Superior Predictive Ability,” Journal of Business & Economic Statistics, 2005). Each supplies a way of asking whether the best of many rules could have looked this good by chance. More recently, David Bailey, Marcos López de Prado, and their co-authors have translated those warnings into a practical program of quality control for modern quantitative researchNumbers have been put on the damage. Bailey, Borwein, López de Prado & Zhu (“Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance,” Notices of the American Mathematical Society, 2014); Bailey & López de Prado (“The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality,” Journal of Portfolio Management, 2014); Bailey, Borwein, López de Prado & Zhu (“The Probability of Backtest Overfitting,” Journal of Computational Finance, 2017). It is the arithmetic that turned a warning into a procedure. López de Prado’s own field guide — “The 10 Reasons Most Machine Learning Funds Fail” (Journal of Portfolio Management, 2018) — is the nearest prior essay on this paper’s question; its ten reasons are the research-process half of the diagnosis, the four deprivations and the institutional argument being the other half. And the flattery does not even need the search. Martin & Nagel (“Market Efficiency in the Age of Big Data,” Journal of Financial Economics, 2022) model a market whose investors must estimate many parameters from a short history and price rationally on their estimates; studied afterward, that record still shows in-sample return predictability, because the researcher sees the realized outcomes the investors were still learning about. Predictability of this kind was never tradable — it is the residue of the market’s own learning, not a missed inefficiency — and it would decorate the record even if no researcher had ever mined it. A quality-control program can count and correct a search; this flattery survives the correction, because no search produced it. It is one more reason only performance out of sample carries economic meaning. — a program never more relevant than today, when a researcher can evaluate millions of signals and configurations by machine, retain only the most attractive result, and call the survivor a discovery. We endorse the program wholeheartedly. The instruments demand careful handling (as we shall see), but the attitude admits no compromise: a backtest is not evidence to admire but a claim to stress, to deflate, and to attempt to falsify. The unit under test deserves a word as well. It need not be the lone signal — a book of individually feeble, partially independent signals is a legitimate object of validation, judged as the ensemble it is. Whatever the unit, it passes through the same gates.
~The deflated Sharpe ratio puts the first question to a glittering track record: does it still glitter once we account for how many strategies were tried before this one was selected? A Sharpe ratio of two means one thing if it was the first idea tested and quite another if it was the best of ten thousand. The deflation makes that difference explicit instead of leaving it to flatter the unwary. It is, in statistical dress, a multiple-testing correction on the Sharpe ratio, kin to the Bonferroni adjustment in spirit if not in mechanism. Like Bonferroni it is a deliberately severe judge, pessimistic by design and hard to pass — which is what one wants of a gatekeeper in a field so practiced at self-deception.The deflated Sharpe ratio (Bailey & López de Prado, 2014) discounts an observed Sharpe by the expected maximum Sharpe attainable from N effectively independent trials under the null of zero skill, while also correcting for the track record’s length and for the non-normality (skewness and excess kurtosis) of returns. Conceptually it is a selection-adjusted null rather than family-wise-error control proper: the benchmark moves from zero to the expected maximum Sharpe of N trials under the null while the sampling distribution around the estimate stays the single-Sharpe one — the right object, and a sterner one. It asks whether a result survives once we admit how many results were sought. Like all such corrections it trades power for protection against false discoveries. In a domain where the base rate of genuine edges is low, that asymmetry is a feature, not a flaw: a false positive puts capital behind noise, while a false negative merely discards one candidate among many. Three cautions temper the instrument without dulling it. First, its severity is only as honest as the count it deflates against, and in strategy search the trials are seldom independent: a single hyperparameter sweep spins one idea into dozens of near-identical variants, and distinct signals bleed into one another through shared factor exposure. Treating such correlated trials as independent inflates the effective count and over-deflates (Harvey & Liu, 2015, 2021; and the wider multiple-testing-under-dependence literature — Romano & Wolf, 2005; Benjamini & Yekutieli, 2001). Principled alternatives exist — corrections built on the return correlation structure, resampling that respects the dependence, empirical-Bayes pooling across trials — but none is standard, and the effective count remains a free parameter that can be tuned, wittingly or not, in either direction. Second, and this is the corollary the asymmetry tempts us to forget: passing a severe threshold is strong evidence, but failing one is not proof of no edge — a real strategy can be buried by an over-honest correction. This is why the gate cannot stand alone; it tells us when to distrust, never how to build, and the engineering that precedes it does the heavier work. A third caution belongs beside them. The deflation takes the sample skewness and excess kurtosis of a short and deliberately selected series as inputs — the very moments this paper elsewhere calls unreliable in returns. Where fourth moments are absent they cannot be estimated at all; where they exist, the estimate wants eighth moments to hold still. Under heavy tails we would rather compute the null by a block bootstrap that respects the dependence than let the sternest gate in these pages rest on the shakiest moment. Harvey & Liu (“False (and Missed) Discoveries in Financial Economics,” Journal of Finance, 2020) put both error rates into a single calibrated frame — the hurdle chosen from the prior odds of skill and the relative price of each mistake — which is the machinery for deciding how stern the gate should be, in place of a sternness defended once and forever. A severity worth keeping in proportion: the raw count of experiments is not, by itself, a defensible statistical correction, and the honest count (the effective number of independent trials) has no canonical estimator. Each available route rests on assumptions that must be checked, and the number they return can be moved, wittingly or not, in either direction. The registry’s count remains a governance fact of the first order even where it fails as a statistical input. We therefore treat the deflated Sharpe less as the whole floor than as its sternest single gate: a strategy that clears it has passed the correction aimed most directly at selection, and gentler ones make a more practical daily threshold. Another gauge, the probability of backtest overfitting, asks, in effect, how likely it is that the strategy which looked best in-sample will fall below the median of the trials out of sample, which is why one half is the no-information baseline rather than a passing grade. And purged, embargoed cross-validation attends to the subtle ways that information from the future leaks into the evaluation of the past, closing the seams through which a model is accidentally allowed to see the answers before the exam.The seams have names. Ordinary k-fold cross-validation leaks information in time-series settings because label windows straddle the train/test boundary. “Purging” removes training observations whose label windows overlap the test set; “embargoing” additionally drops a short buffer immediately following it (López de Prado, Advances in Financial Machine Learning, 2018). Both repairs are cheap, and neither is what a standard cross-validation routine does by default. The same cross-validation machinery reappears later in a different office (coverage rather than hygiene), and the two should not be confused.
We regard this machinery as the floor. Its absence is disqualifying. A portfolio manager who cannot tell you how many strategies were tried, or who reports a raw Sharpe ratio with no accounting for the search that produced it, has not cleared the lowest bar of the discipline (a low bar, and it is remarkable how often it is walked under), and whatever else such a manager shows you should be read in that light. The instrument may be any defensible one: deflation, White’s reality check, Hansen’s SPA, resampling, shrinkage, a holdout under lock. It is the documented accounting for the search, not the particular statistic, that admits no substitute. One further demand. The deflated Sharpe is only as honest as the count it is fed. The number of strategies tried must be a fact of record, not of recollection. Every configuration ever run, logged automatically and immutably in a registry the researcher cannot edit after the fact — so that whatever correction is computed rests on an artifact that exists and can be attested to rather than on a number the manager remembers.The count is slipperier than it looks, even for the honest: Gelman & Loken (“The garden of forking paths,” 2013) observe that a single reported analysis silently embodies the many analyses that would have been run had the data come out otherwise, so the effective number of trials exceeds the number of backtests actually executed. A registry can count what was run; it cannot count what was contemplated. Memory, in this business, is an instrument of flattery. But a floor is not an edge. These are instruments of inspection. They examine a finished strategy and tell you whether to distrust it. A validation gate, however severe, can reject a thousand overfit strategies; it cannot produce a single good one. It is the quality-control station at the end of the line, and no amount of inspection there can substitute for having engineered the product correctly at the start.
Before leaving the floor: there is a second overfitting, subtler than the backtest’s, that no registry of runs can capture. The registry records what was executed; it cannot record what the researcher would have executed had the results come out otherwise — the feature quietly dropped after a disappointing glance, the architecture adjusted, the idea abandoned half-tested in favor of its more promising cousin. This is the overfitting of the researcher, and it requires no dishonesty: it is what any diligent, hopeful person does, one reasonable decision at a time (and every design choice we are about to commend is itself such a decision). The backtest overfits in the open; the researcher overfits in private. The second is worse. We know of no complete cure, but the mitigations are concrete: a registry extended from runs to decisions, recording the roads not taken beside the ones that were; a final tranche of history locked away under someone else’s key, the number of looks at it budgeted in advance and each look spent like capital; a separation of powers, so that the researchers who build a strategy are never the judges who certify it; and replication cold, on markets and asset classes the idea has never touched, where no memory of the development data can flatter it. We must not oversell the registry of decisions: its completeness is exactly what no outsider can verify, for the choices that bias a search most are often those a researcher never consciously made. The allocator’s firmer ground is the handful of tests that need no such trust at all, and the closing list comes to them.
✓A last word on the ground beneath the floor. Before any statistical subtlety bites, a strategy can be killed — or, worse, falsely raised from the dead — by the plumbing of its data: prices not taken as they stood, a universe quietly purged of its failures, the costs of borrowing and financing waved away. None of this is specific to machine learning; it is the old hygiene of empirical finance, and we glance past it only because it predates our subject (which is not at all the same as saying it is behind us).The subtlest leak hides in data that everyone trusts. Historical price series are almost invariably back-adjusted by the vendors — rescaled so that yesterday’s prices are stated in today’s terms, splits and dividends smoothed away. Convenient, and more treacherous than it first appears — though the hazard is narrower and more specific than the folklore holds. The adjustment is built so that ratios of adjusted prices recover total returns: a later split or dividend rescales the entire preceding series by one common factor, and that factor cancels in every return computed wholly before the event, so a model that consumes returns reads nothing of the future in them. The leak opens only when the adjusted level is fed as a feature — for a stock that split many times is thereby marked, in the absolute price of its early history, as one that would go on to prosper; the future has been painted, faintly, onto the past, and a model shown price levels will read the brushstrokes. The moral is not to distrust adjusted data but to distrust adjusted levels: adjusted returns are legitimate when corporate actions and information timing are handled consistently, while absolute adjusted prices are not point-in-time observables and have no business as raw inputs. The coarser sins are better known and more easily purged — survivorship (a universe stripped of its delisted dead), look-ahead in point-in-time fundamentals (restated numbers standing in for those actually available then), the omission of borrow costs, financing rates, and short-sale constraints — but back-adjustment is the one that slips past careful hands. We note only that flexibility sharpens the blade rather than dulling it: a flexible model, or a modest one inside a vast search, will find and feed on the faintest leak of the future into the past far more eagerly than any linear rule.
The agent that knew too much
~The newest instruments sharpen both edges of this blade at once, and they arrive quickly. Systems in the family of AlphaEvolve (a language model proposing candidate programs, an evolutionary loop selecting among them, running unattended) are, at bottom, the genetic algorithms of an earlier era, reborn with a sampler good enough to make search affordable in spaces that used to defeat random mutation.The lineage is easy to check. AlphaEvolve (Novikov et al., “AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery,” Google DeepMind technical report, 2025) couples a language-model proposer with evolutionary selection over programs; among its results is a reduction in the number of scalar multiplications for 4×4 matrix products over the complex numbers — the first advance in that bilinear, recursively applicable setting since Strassen’s 1969 construction. Its sibling AlphaTensor (Fawzi et al., Nature, 2022), a reinforcement-learning search, had already bettered Strassen in arithmetic modulo 2, and its predecessor FunSearch (Romera-Paredes et al., Nature, 2024) found new constructions in extremal combinatorics the same way. The reading we press is deliberately unglamorous: AlphaEvolve and FunSearch are, at bottom, genetic algorithms whose proposal distribution is a language model — vastly better, but a proposal distribution still. Note, too, what both systems were given: a perfect, instantaneous, adversary-free evaluator — does the program run faster, is the construction larger — which is the gift the markets withhold. A subtlety the specialist will press: a proposer that has digested the anomaly literature is often merely a better prior, and a good prior narrows the effective search rather than inflating it. The hazard is sharper than that — that literature is itself the surviving residue of a vast, largely unlogged search over the very same historical record the backtest will reuse (Harvey, Liu & Zhu counted 316 factors across published and working papers), so the agent does not simply know where to look but silently re-runs the field’s hidden trials on the data it will be scored against. Prior knowledge distilled from genuinely independent evidence would be a gift; prior knowledge distilled from the same over-mined history is leakage wearing the mask of a prior, and no per-shop registry can price it. It can be narrowed, at a cost: an agent trained only on corpora ending where the evaluation begins, its proposals scored strictly past that cutoff, buys back the arithmetic by paying with the one thing the agent was hired for — the accumulated hindsight itself. The step from mathematics to markets has since been taken by name: AlphaAgent (Tang et al., KDD 2025) wraps a language-model proposer in exactly the originality and complexity regularizers this concern predicts it needs — its authors reporting that unconstrained miners “predominantly replicate existing market inefficiencies already exploited by participants,” the monoculture conceded from inside — and R&D-Agent-Quant (Li et al., NeurIPS 2025) carries the full propose-implement-backtest loop to industrial maturity. What none of them yet prices is the charge of these paragraphs: what a proposer that has read the field’s every published trial does to the effective count of its own search. Set loose on strategy discovery, such a system runs our ten thousand coin-flippers at machine speed, and the search space explodes from thousands of candidates into millions. The effective number of independent trials any deflation actually needs becomes harder to estimate rather than merely larger, the proposals being correlated with one another and conditioned on a record the machine has already read. Agentic search changes the arithmetic of multiple testing, never its logic. Yet the same machinery, honestly deployed, keeps the research honest. Unlike the human researcher, the machine forgets nothing and hides nothing — every candidate it generates can be logged by construction, and the garden of forking paths becomes, for once, a garden with a survey. The gift is smaller than it appears. Logging every candidate tames the search only if the candidates are drawn independently of the answer. Here they are not. A large model that proposes strategies has read the public record of what has already worked: the anomaly literature, the folklore, a thousand published backtests (and, in time, these pages). Hindsight enters not through the data, where purging and embargo could reach it, but through the weights of the agent’s language model, where they cannot. There is no embargo on a memory. Call it pretrained hindsight. It is contamination that arrived before the experiment began, and that no purge of the data can reach. The registry records what was tried; it cannot record what the agent already knew, and that corrupts the one number the floor depends on.
The deflated Sharpe ratio wants a number the registry cannot supply on its own: not how many trials were run, which the ledger records exactly, but how many independent ones they amount to. Agentic search answers “millions.” The honest answer is worse than any number: those millions of guesses were never independent draws from a space of ideas. The machine was guessing with the answers half-remembered: its proposals are educated recollections of what has already worked. The machine researcher forgets nothing, which we praised; it also knew something before it began, which no amount of logging can undo. The correction thus depends on the model that did the proposing, not merely on the count of its proposals. The ledger remains an auditable fact about the count of the search. What it will no longer support is an inference about the independence of what was searched. The registry alone can no longer measure how wide the search truly was.
~Nor is the hazard confined to the machine that proposes whole strategies; it travels with every feature a language model touches. The sentiment score, the summarized filing, the embedded headline — each is produced by a model whose training corpus contains the very years the backtest will replay, and each carries hindsight into the pipeline through the same door as the agent’s — not the data, but the weights.The effect has been measured, not merely feared — Glasserman & Lin (“Assessing Look-Ahead Bias in Stock Return Predictions Generated by GPT Sentiment Analysis,” Journal of Financial Data Science, 2024) show that an LLM scoring headlines can carry two distinct biases: a look-ahead bias, in which the model already knows the returns that followed an article, and a distraction effect, in which its general knowledge of the named firms colors the sentiment it reports. Their measurement found the second effect the larger, and they conclude that, out of sample, look-ahead is not the operative worry. What they establish for us is the channel, and a way of closing it: anonymize the headline and the firm’s identity goes with it. That look-ahead proved the smaller hazard in one setting bounds nothing about a signal a shop has tuned for years against the same corpus. The remedies the body lists have since acquired builders and measurements. He, Lv, Manela & Wu (“Chronologically Consistent Large Language Models,” arXiv:2502.21206, 2025) train the point-in-time reader outright — vintage models fed only text available at each date — and find the leakage premium in their news application modest: a real-time Sharpe statistically indistinguishable from the contaminated reader’s. Engelberg, Manela, Mullins & Vulicevic (“Entity Neutering,” SSRN 5182756, 2025) automate the anonymization, rewriting until an independent model can no longer name the firm, and bound the residual look-ahead by the sentiment the neutering changes — a few percent in their corpus, wider on volatile days and in longer filings. The caution that makes their verification loop necessary is Sarkar & Vafa (“Lookahead Bias in Pretrained Language Models,” SSRN 4754678, 2024): instructing a model to forget the future does nothing, and naive masking little more, the model reconstructing the censored name most of the time. And the propensity can be audited from outside: Gao, Jiang & Yan (“Detecting Lookahead Bias in LLM Forecasts,” arXiv:2512.23847, 2025) read the memorization probability off the model’s own recall — near saturation in sample, vanishing past the training cutoff. The pattern across all of them repeats the one above: measured in a clean setting, the bias is often small; no measurement yet binds the tuned, proprietary case, which is why the discipline stays. A backtest of a machine-read signal is honest only if the reader itself is point-in-time: evaluated from its training cutoff forward (a short and shrinking window), on a checkpoint frozen at the historical date, on a model trained solely on corpora available then, or on a reader demonstrably decontaminated of the period it is asked to relive. Rewriting the history past recognition is a stress test rather than a proof. A shop that quotes decades of simulated performance for a signal read by a model that has, in the relevant sense, already read the newspapers is reporting memory as foresight. And a second hazard follows the first, familiar from every crowded trade: when a hundred shops distill their signals from the same two or three foundation models fed the same news (one supplier in a hundred coats), the signals arrive correlated at birth — the crowding that once took years of imitation to assemble now ships pre-assembled, in the weights everyone rents. The deprivations compound. The adversary, this time, is the monoculture itself.
This, finally, is why we care so little about the form of the model and the particular way it was found (hand-built network or evolved program, gradient descent or agentic search). What matters is the discipline that surrounds the model and the structure built into it. That structure is a form too — the only one that decides anything. The four deprivations do not ask how the box was made, and neither should the discipline that answers them.
This is where our two areas of expertise meet, and where we believe the genuinely important work now lies. The statistician’s achievement is to have made much of overfitting detectable. The engineer’s task — ours — is to make it improbable. Inspection catches bad models; engineering makes them scarce. Generalization is designed in from the start, not tested for at the end.
Constrain and conquer
To design for generalization rather than test for it means accepting the four deprivations as engineering constraints and building a system whose every part answers one of them.
The word “governance” carries the two burdens named at the outset, and they part company here. The engineering arm is the constraints, priors, and calibrations built into the learning problem itself (what the model may express, how far it is believed, when it must abstain). The other is institutional: the registries, the separation of the builders from the judges, the pre-committed definitions of failure — what the organization does around the model, whatever the model is. Medicine, to which we return at the close, keeps the same two ledgers: the dose and its contraindications on one side; the registered trial, the review board, and the vigilance after approval on the other. A serious fund needs both arms, and much of the confusion in the public conversation comes from praising one while neglecting the other.
Four responses follow, one to each deprivation. Between them they carry most of the difference between a strategy that survives and one that merely backtests well. They answer, though not in the order the deprivations were introduced, four questions: what is the model allowed to do; how much should we believe it; what does its confidence actually promise, and does it know when it has wandered into the unknown; and what will the market do about it once it acts?
~Consider first what the model is allowed to do. The industry default is to train a model in complete freedom and bolt a risk overlay onto its output afterward (position limits, a volatility cap, a kill switch), like fitting a speed limiter after teaching someone to drive on an empty racetrack. The two then spend their lives at war. The model strains toward positions the overlay exists to forbid, and the overlay intervenes only once the model has resolved to do something dangerous. We advocate the opposite approach. The risk controls should live inside the learning problem itself (in the objective, the constraints, the space of actions the model may express), and learning happens through them, so that the model has no occasion to learn to want what it is not permitted to have.Learning through hard guardrails is useful well beyond finance. Safe reinforcement learning and constrained policy optimization hold robots and other safety-critical controllers near a safety envelope during training rather than only after it (Achiam, Held, Tamar & Abbeel, ICML 2017) — near-satisfaction under assumptions, not an unconditional guarantee. Physics-informed neural networks fold conservation laws into the objective as penalties, which encourage consistency rather than guarantee it; hard-constraint variants exist to close that gap. Making a learned model provably uphold its physics — rather than merely incline toward it — remains an actively researched, largely unsolved problem across physical AI (Krishnapriyan, Gholami, Zhe, Kirby & Mahoney, NeurIPS 2021, on the failure modes of the soft penalties; Lu, Pestourie, Yao, Wang, Verdugo & Johnson, SIAM Journal on Scientific Computing, 2021, on hard-constrained inverse design; Hansen, Maddix, Alizadeh, Gupta & Mahoney, ICML 2023, on enforcing conservation laws). One of us has worked this seam from the physics side: a generative, normalizing-flow framework for quantum fermionic systems in which the physical symmetries — fermionic antisymmetry and permutation structure among them — are built into the network itself rather than hoped for from training (Freedman, Rozenberg & Bronstein, “A Theoretical Framework for an Efficient Normalizing Flow-Based Solution to the Electronic Schrödinger Equation,” AAAI, 2025). The same hands have worked it from the engineering side: in the PILOT line of work, MRI scan trajectories are learned with the scanner’s hardware limits — peak gradient amplitude and slew rate — carried inside the training loop as differentiable hinge penalties on the trajectory’s velocity and acceleration, so that infeasibility is priced at every step rather than forbidden (Weiss, Senouf, Vedula, Michailovich, Zibulevsky & Bronstein, “PILOT: Physics-Informed Learned Optimized Trajectories for Accelerated MRI,” Machine Learning for Biomedical Imaging, 2021). Control-theoretic constructions do guarantee by construction, under their models’ assumptions, what learned penalties only encourage — Lyapunov functions stability, control-barrier functions the invariance of a declared safe set. The fine print about what is promised, and under what conditions, travels with each method. In each case the constraint delivers safety and data-efficiency at once — when the structure imposed is the right one. That proviso is the whole of it: a wrong constraint buys the safety and pays for it in edge, which is why these pages argue for walls only where what is enforced is exactly promised or exactly known, and for budgets everywhere it is merely estimated. A catastrophic position is not vetoed after the fact; it is simply absent from the model’s vocabulary, unthinkable in the most literal sense. The mechanism is a projection onto the feasible set, which is exact, rather than a penalty that merely discourages, which is not.That sentence has a research program behind it, and it deserves its name. Training a model through the optimization that will consume its outputs — the learner scored by the decision it induces rather than the prediction it makes — is decision-focused, end-to-end learning: Elmachtoub & Grigas (“Smart ‘Predict, then Optimize’,” Management Science, 2022) supplied the loss and its theory, portfolio experiments included; Donti, Amos & Kolter (“Task-based End-to-End Model Learning in Stochastic Optimization,” NIPS 2017) trained forecasters through constrained stochastic programs; and the exact projection of our sentence exists as a trainable layer, the optimization differentiated through its own optimality conditions (Amos & Kolter, “OptNet: Differentiable Optimization as a Layer in Neural Networks,” ICML 2017; generalized in Agrawal et al., “Differentiable Convex Optimization Layers,” NeurIPS 2019). Butler & Kwon (“Integrating Prediction in Mean-Variance Portfolio Optimization,” Quantitative Finance, 2023) carried the machinery to the mean-variance problem itself, and where their result is the sharper one for our purposes: with the covariance known, the decision loss is a covariance-weighted quadratic in the prediction error, so integration buys nothing a correctly weighted regression would not; the gain in their experiments reappears precisely because the covariance must itself be estimated. Integration earns its keep, there as here, from what the pipeline does not know. One of us has spent the better part of fifteen years building exactly this machinery in imaging and inverse problems, where learning through the physics of the instrument is the difference between a reconstruction that looks right and one that can be acted on. What that literature buys with the machinery is decision quality; what the envelope buys with it is governability — walls kept distinct from budgets, costs conditioned on rather than hard-wired, and every constraint’s contribution auditable by rerunning the book without it. The machinery is shared. The office is new. Two kinds of restriction do different work here and should not be confused: the hard constraint that renders a position infeasible (a leverage ceiling, an instrument the mandate forbids) and the soft penalty that merely prices a tendency, leaning the model away from concentration or turnover without forbidding either. The first is a wall, the second a slope. A serious system carries both, and knows at every point which of the two it is relying on.
The outer overlay does not disappear. A limit that can be audited, and altered without retraining, is a virtue of governance no serious fund should surrender. But its office changes from combatant to tripwire: if the envelope within has done its work, the wall without (the emergency stop, as distinct from the workaday constraints that may bind by design) should almost never be touched. How often it is touched becomes one of the most eloquent diagnostics the system produces. A model repeatedly rescued by its outer limits (not by the occasional sanity check against a corrupt feed or a halted exchange, which is exactly what such limits are also for, but rescued, as a matter of course, from positions of its own choosing) is a model whose training has failed. It is telling you so. A learner given a hard wall will learn to lean against it. The envelope must therefore penalize the approach and not merely forbid the crossing. How often the model loiters at its own boundaries is itself worth watching.
Three further admissions, each with its remedy. First, to bake a friction or a limit into the objective is to entangle what the model has learned with a number that will not sit still. Costs change, mandates change, and a behavior trained against yesterday’s spread is stale until retrained. The remedy is to condition rather than to hard-wire. Let the costs and the limits enter the model as inputs rather than as constants of its training, so that what is learned is a family of behaviors indexed by the constraint, and a change in the world becomes a change of argument rather than a change of weights. Second, an objective with many terms invites a question the fund must be able to answer: which term produced which behavior? Keep the objective decomposable and log its parts separately; better still, rerun the system counterfactually with a constraint relaxed, and let the difference between the two books attribute the behavior to alpha or to envelope. A learned envelope that cannot be attributed is not a governed model; it is a second black box grown quietly inside the first. Third, the envelope is a doctrine, not a monopoly. Many serious shops keep the classical separation (forecast, then optimizer, then an independent risk layer), and it is a defensible architecture, with virtues of modularity and audit, provided the learner was trained and evaluated under the constraints and costs it will actually face. What is indefensible is not the separation but the model raised in a frictionless fiction and corrected only at the door.
This does not create new independent data, and should not be sold as though it did; but it is easily misread. The constraint is a fixed, non-learned layer; the model’s learnable body and the history that trains it are untouched. Sweeping the constraint during training need not enlarge the estimation problem where the structure cooperates: it is the same problem seen under augmentation, every episode reused at each constraint level. And where the constraint acts monotonically on what the model has learned to rank, one learned ordering already serves the whole family, re-projected rather than re-estimated. That holds for the single-period sizing of a cross-section; it does not hold in the path-dependent world of multi-period execution, where a change of cost changes which trades exist at all. The consequence is worth drawing outright. In exactly that case (single period, monotone constraint) the governed architecture and the classical sequence of forecast, optimizer, and risk layer produce the same book. There the classical separation is not merely defensible but equivalent. Our claim bites where paths and multi-period objectives do, which is most of the working world and not all of it. The conditioning therefore does not sit in tension with our claim that structure substitutes for scarce data; it is that claim. What buys the family cheaply is precisely that the constraint is imposed rather than inferred — a low-dimensional, smooth layer the model reads as an argument, not a high-dimensional map it must learn from samples it does not have.
~Within its engineered envelope the model stays free — free, in particular, to vary its own volatility and leverage according to how strongly it believes what it is seeing, running hot when it has earned the right to and cool when it has not. The contrast with the industry’s standard volatility-targeting overlay is instructive. A constant volatility target is no folly. Where volatility is persistent and the premium it is paid for is not, scaling exposure against it can demonstrably lift the realized Sharpe ratio, and the published result requires neither diversification nor any rotation of risk. Our quarrel is narrower than the result. It lives in two places its admirers tend to hurry past: the scalar in aggregate, when every book obeys it at once; and, at the level of a single fund, the implementation the result is usually invoked to defend, which is not the one it tested: a risk-budgeted optimizer under an equality constraint on volatility, where the published rule levered a book up and down whole. The first is systemic. When dozens of funds de-lever by the same rule in the same hour, the diversification that made the target attractive evaporates exactly as it is drawn upon, and each fund’s prudent selling becomes the next one’s falling market. The objection reaches our own proposal, and the monoculture returns with it. Conviction is a deterministic function of public data, and where the field distills its signals from the same two or three foundation models, conviction too arrives correlated at birth, so correlated de-risking follows by a longer inferential chain and without the common trigger a volatility rule at least makes visible. What buys a shop its idiosyncrasy is the part of the pipeline nobody else holds (its own labeling, its own constraints, its own retirement rules), and a shop that cannot name that part should assume it has none. The second belongs to the single-model book one actually runs. To insist on a fixed risk budget when the model finds little worth backing (the target met not by scaling the book down but by reallocating it) is to pour weight into one’s few highest-conviction names — to concentrate in the very act of controlling risk, and so to shed diversification at the moment one meant to hoard it.
The market-neutral book shows why no trailing number can stand in for judgment, though it refuses a single moral. When realized volatility erupts, the naive target counsels retreat. What the eruption means depends on the book: in some convergence trades the rush of correlations toward one leaves the long side and its hedge more offsetting than the statistic suggests, while in others crowding, basis risk, and the withdrawal of liquidity make historical neutrality a poor guide at precisely that moment. No single trailing number can tell the two apart — it will overstate one book’s exposure while understating the storm breaking over another’s. Even where the correlation-to-one does flatter your hedge, it is the very channel by which everyone else’s forced unwinding arrives at your door. A model that modulates its own conviction (leaning in when the opportunity is real, standing aside when it is not, but always and only within limits it can never breach) answers both, because it varies risk by the strength of the signal rather than by a single trailing number. That number is blind to liquidity, to crowding, to basis, to the quality of the opportunity itself — and it misreads the world precisely in a crisis.That volatility timing — scaling exposure inversely to recent realized variance — lifts single-fund Sharpe ratios is well documented — Moreira & Muir (“Volatility-Managed Portfolios,” Journal of Finance, 2017) report substantial improvements in the portfolios they study, and our argument concedes the point — though Cederburg, O’Doherty, Wang & Yan (Journal of Financial Economics, 2020) find the advantage fragile once the rule must be run in real time. Our concern is what the rule does in the aggregate and at the extremes — and, beneath both, whether a single trailing number should be setting the size of a book at all. The record supplies instances — three distinct mechanisms, not one: the quant deleveraging of August 2007, when crowded market-neutral books unwound into one another (Khandani & Lo, “What Happened to the Quants in August 2007? Evidence from Factors and Transactions Data,” Journal of Financial Markets, 2011); the implosion of the short-volatility complex in February 2018 (its anatomy in “Volatility Is Back,” BIS Quarterly Review, March 2018); and the deleveraging cascade of March 2020, in which volatility-targeted vehicles, bound by mandate to cut risk as volatility spiked, sold into a falling market in unison — one channel among several the postmortems document (Schrimpf, Shin & Sushko, “Leverage and Margin Spirals in Fixed Income Markets during the Covid-19 Crisis,” BIS Bulletin, 2020). What unites them is not a shared cause but a shared reflex: rules reading a single trailing risk number, and portfolios obliged to shed risk on its say-so, selling in concert. The mechanism, stated with care, is not that such rules control risk but that a single realized-volatility number can misread the world at the only moments that matter — sometimes overstating a convergence book’s exposure as correlations rush toward one, sometimes understating a crowded book’s fragility as liquidity withdraws, and always understating the systemic storm the shared reflex unleashes on everyone at once. None of this argues for an unconstrained portfolio. Prudence insists that models be treated as models, and a cap on a strategy’s expected volatility belongs in any sober mandate. But the concentration just described has a second edge. Diversification’s first rule is many signals, across many horizons, across many assets. When that breadth collapses (and in periods of heightened uncertainty it so often does), the diversification stands revealed as false, realized volatility arrives far above anything anticipated, and the strategy is forced to de-lever after the event. And our own alternative enjoys no exemption. A model that earns the right to run hot is making a learned claim about its own conviction, and that claim owes the floor the same deflated, suspicious evaluation as any other product of the research process.
In a data-starved world, a constraint is not a tax on the model but a gift of information: every sound restriction (every economically motivated prior, every symmetry the model is told to respect) supplies knowledge from outside to compensate for the data we lack. The gift, though, is only ever as good as the prior behind it. A restriction sound in one regime can bind wrongly in the next, and a stale economic assumption must be retired with the same vigilance we demand of the model itself. Constraining what the model can express is not the price of generalization; it is one of its chief engines.In the language of statistical learning, economically motivated constraints are inductive biases, and imposing them is a form of regularization: they raise bias in order to cut variance — a favorable trade precisely when data is scarce and an unconstrained estimator would overfit. The Bayesian reading is equivalent: a prior that concentrates probability mass on economically plausible hypotheses. The machine-learning reader’s reflexive objection here carries a name — the bitter lesson (Sutton, 2019): hand-built priors, the lesson runs, always lose eventually to scale and compute. The rebuttal is the first deprivation itself: the bitter lesson presupposes the one resource markets refuse to supply — more data on demand. Scale manufactures parameters, not independent episodes; the history is finite, non-replayable, and non-stationary, and “eventually” never arrives. Compute, meanwhile, widens the search without enlarging the evaluator — more candidates asked of the same finite history is more multiplicity, not more knowledge. AlphaFold is again a deceptive case. AlphaFold2 used an SE(3)-equivariant architecture, baking the symmetries of Euclidean space into the model itself; its successor discarded that geometric construct for a generic diffusion model (Abramson et al., “Accurate Structure Prediction of Biomolecular Interactions with AlphaFold3,” Nature, 2024) and taught the same invariance — equivariance, to be exact — by augmentation instead, every structure recentered and randomly rotated at each step. On the surface the message is clear: the bitter lesson arriving for geometric deep learning. It is nothing of the kind. The prior did not depart; it changed vehicle. An inductive bias can ride in the architecture, in the objective, or in the data, and moving it between them is not the same as doing without it. The trade is real, and it has been measured. A taught symmetry is approximate where a built one is exact — the same gap the physics-informed literature meets — and Brehmer, Behrends, de Haan & Cohen (“Does equivariance matter at scale?,” Transactions on Machine Learning Research, 2025) find the built-in version the more data-efficient, the gap narrowing once an augmented plain transformer is given enough data and epochs, but the equivariant model still ahead at every compute budget they test. The appeal of the generic architecture is thus engineering before it is statistical: simpler code, better hardware utilization, no bespoke layers to maintain. What decides the vehicle is whether the symmetry can be sampled at all. SO(3) is compact and three-dimensional, so a random rotation per step costs nothing. Contrast this with the symmetry of a network’s own weights: it is not cheap to sample. Permuting the neurons of a hidden layer leaves the function untouched, and that group is a product of symmetric groups whose size is the product of the factorials of the layer widths — so vast that augmentation can visit only a vanishing fraction of it, which is why Navon, Shamsian, Achituve, Fetaya, Chechik & Maron (“Equivariant Architectures for Learning in Deep Weight Spaces,” ICML, 2023) build the invariance into the architecture and beat permutation augmentation comfortably. Markets are the third case, and the worst of the three. Here the symmetry is not merely expensive to sample but largely unknown, and no transformation of the record manufactures another 2008. The prior must therefore ride in the model or the objective, which is where the economic constraints described here sit.
~This is not an argument that models should be small — that is not the lesson the deep-learning decade teaches. What that decade found, to nearly everyone’s surprise, is that scale is not the enemy of generalization: large, deep, richly overparameterized neural networks inhabit loss landscapes that are easier to descend, and the humble workhorse of stochastic gradient descent carries, on one influential account, an implicit regularization of its own, seeming to prefer, among the many solutions that fit the data, the flatter and the simpler. The mechanism remains unsettled. We are arguing for governance, not for parsimony: a large model, diligently controlled along the dimensions that matter, is entirely welcome in these pages. The controls of the preceding paragraphs are best understood exactly so — as inductive biases, the knowledge that would be prohibitively hard to learn from data this scarce, supplied instead by construction. We are aware that a respected line of work presses the opposite corner harder still, finding a “virtue of complexity” in return prediction (models expanded far beyond the size of the sample improving out of sample, even on training windows of months), and that a subsequent analysis attributes much of the measured benefit to the mechanics of volatility timing rather than to genuine foresight. Our own sympathy is a qualified lean toward complexity — but only within a training regime built to supply the regularization that scarce data cannot, where the constraints and priors of these pages already do the work the data will not. The clinical record we will invoke at the close (flexible models failing, again and again, to beat a humble regression on scarce data) is no embarrassment to this lean but its very condition. Those studies divide cleanly, by the reviews’ own accounting: where the comparison was done properly the flexible model reached parity, and where it was not, the flexible model’s apparent advantage was manufactured by exactly the leakage, the small biased samples and the unvalidated flexibility our own construction does not permit. What no party to the debate disputes is that the implicit regularization of the deep-learning recipe was certified in worlds rich in data and stable in their rules, and that in a world which is neither, the explicit biases must carry the heavier share of the load.The claim that the implicit biases carry the load has a contested literature behind it. On generalization in overparameterized networks, Zhang, Bengio, Hardt, Recht & Vinyals (“Understanding Deep Learning Requires Rethinking Generalization,” ICLR 2017) posed the puzzle; Belkin, Hsu, Ma & Mandal (PNAS, 2019) described the “double descent” by which test error falls again beyond the interpolation threshold; Bartlett, Long, Lugosi & Tsigler (“Benign Overfitting in Linear Regression,” PNAS, 2020) gave conditions under which interpolating noise is harmless (the flat-minima account of the implicit regularization is itself contested — sharp minima can generalize: Dinh, Pascanu, Bengio & Bengio, ICML 2017). In finance proper, Kelly, Malamud & Zhou (“The Virtue of Complexity in Return Prediction,” Journal of Finance, 2024) argue that the benefit of expansion survives even when observations are few; Nagel (“Seemingly Virtuous Complexity in Return Prediction,” NBER Working Paper 34104, 2025) replies that, in that construction, the apparent gains are largely mechanical — in effect a volatility-timed momentum strategy in disguise rather than extracted foresight. Kelly & Malamud’s rejoinder (“Understanding the Virtue of Complexity,” Swiss Finance Institute, 2025) disputes the mechanics and presses the distinction between nominal and effective complexity — which, if anything, restates our reading: the virtue, where it is found, arrives already regularized. The debate is live, and our claim does not depend on its outcome — for the virtue on offer is of complexity under heavy shrinkage: flexibility governed is the principle, not an exception to it. The book-length statement of that principle is Nagel’s (Machine Learning in Asset Pricing, Princeton University Press, 2021): machine learning earns its place in asset pricing once economic considerations — portfolio problems, absence of near-arbitrage, investor learning — choose the regularization the data cannot.
One failure mode is especially seductive. Train a flexible network on financial data and it will, with very little coaxing, rediscover the classic factors entirely on its own — momentum in one corner of its weights, mean reversion in another, a whiff of value somewhere else. The spectacle of a neural network rediscovering decades of financial economics from raw prices or returns has a genuine magic to it, and it is precisely the wrong thing to be impressed by. A model that rediscovers momentum has demonstrated only that momentum was in the data — which we knew — and has said nothing whatever about whether it can find something we did not already know. The capacity to recover known factors is no evidence at all of the capacity to find idiosyncratic alpha, those independent, un-owned sources of return that are the only thing an investor should be willing to pay active fees for. The rediscovery itself deserves no condemnation. A model must carry the known factors within it if it is to govern and time them, as we are about to argue. What deserves condemnation is the temptation to hail the rediscovery as evidence. A model that brings back only the familiar, with nothing residual beyond it, has failed at the one task that justifies its fees. A model that holds the familiar in order to control it, while finding what is not yet in the textbook, is doing exactly its job.
Controlled substance
~The envelope’s hardest case deserves a separate discussion. It sharpens a question every serious shop must answer: what to do about the known factors? Here we part company with a common academic habit and a tempting implementation shortcut. A common temptation is to orthogonalize: to strip the known factor exposures out of the data wholesale, forcing the model to operate only in the residual left behind. Anything it finds is then guaranteed independent of value, momentum, and the rest (though the guarantee runs only to the factors as specified, and only over the window used). It is a tidy idea, and it is too tidy. Orthogonalization is several operations wearing one name (residualize the target, residualize the predictors, constrain the portfolio’s exposures, hedge the realized returns), and they are not equally destructive. A hard neutrality imposed only on the final portfolio can coexist with a model that still sees the factors as inputs. Our objection is to the wholesale version. Hard orthogonalization discards real information. The known factors interact with the unknown in ways that themselves carry signal, and amputating them entirely throws those interactions out along with the redundancy. It also imposes exactly the kind of rigid constraint we have been warning against: a straitjacket where an envelope would be better suited. We prefer to soft-control exposure. Rather than forbidding the model all contact with the known factors, we let it draw on them where they genuinely help, while penalizing and bounding the portfolio’s net exposure to them,Two clarifications, since the word covers more than one operation. First, neutrality can be imposed at the level of positions or at the level of returns, and the investor-facing object is the latter: what an allocator ultimately experiences is a return stream decorrelated from the known factors, not a portfolio mechanically orthogonal to them. Positions-level neutrality is the stricter discipline, but it pays for its strictness by narrowing the envelope the model may explore; returns-level neutrality permits a wider envelope while still delivering the decorrelation that matters — though realized decorrelation is an ex-post verdict: it is monitored and attributed, never promised in advance, and it complements rather than replaces the control of exposures. Second, the edge in any such scheme lies not in the relativization itself — subtracting a benchmark, residualizing against factors, and their kin are commoditized operations — but in the judgment of which quantities one chooses to relativize against which. The operation is generic; the choice is where the alpha hides. so that the factor footprint stays deliberately small over time and the information survives. The known factors become a controlled ingredient, not a banned substance.Orthogonalization, in the sense the previous note separates out, here means residualizing returns (or signals) against a known factor model — the academic Fama–French-style factors, or a commercial risk model such as Barra, Axioma, or Bloomberg — and trading only in the residual. It is the standard route to “market-neutral” or “factor-neutral” exposure. Our objection is not to neutrality as a goal but to achieving it by amputation: a soft penalty on net factor exposure, tuned rather than absolute, preserves the factor-times-idiosyncratic interactions that hard orthogonalization destroys, while still holding the realized factor footprint to a chosen budget.
A nuance, before the principle is applied indiscriminately: not every neutrality deserves softening. Exposures that are contractual, linear, and cheap to enforce exactly (the dollar neutrality of the book, its neutrality to the market itself) should be enforced without a budget. They are promises made to investors. The first is an identity of arithmetic, audited exactly; the second rests on estimated betas, a promise about process rather than arithmetic, enforced to the estimate and monitored ex post. Nothing in the data argues for letting either float. It is the style factors (value, momentum, and their many cousins) whose interactions carry information, and whose exposure deserves a budget rather than a wall. The criterion beneath both cases is the same: enforce exactly what has been promised exactly, and budget what is merely estimated and unpromised. The contractual neutralities earn their wall from the promise, not from the precision of the estimate. What one actually wants is that tomorrow’s portfolio be orthogonal to tomorrow’s factor exposure (a quantity nobody observes and everybody estimates). Exact enforcement against an estimate does not deliver neutrality; it delivers neutrality to the estimate, and pays for it in real information destroyed.A rough accounting of what the cross-section has already done, with the arithmetic on its face. Write a name’s return as its loading on a common shock plus the cross-sectional signal and its noise. Ranking within the date annihilates the part common to that date — the level, not the differences between names — and leaves the differential exposure behind, which is second order in the dispersion of the loadings rather than in their level: of the market’s contribution to the pooled target, the fraction removed is , with σ the cross-sectional standard deviation of the loadings about a mean of unity. At a dispersion of σ ≈ 0.3 — an ordinary equity cross-section — the ordering has thus already removed some nine-tenths of it, and removed it for nothing: no factor model estimated, no neutralization step to get wrong. What exact neutralization contends for is the last tenth, at the costs named above.
And this buys something hard orthogonalization simply cannot. We permit controlled exposure at any given instant while holding the systematic, through-the-cycle exposure to near zero. That leaves the model free to time the factors (leaning toward momentum, value, profitability, or leverage in precisely those windows when a factor is likely to pay, and stepping away when it is not), earning a return from the timing itself while carrying no permanent factor bet at all. Hard orthogonalization, by forbidding the exposure at every instant, forecloses that entire avenue. A portfolio pinned to zero factor loading at all times cannot profit from knowing when a factor is worth holding. Whether the factors can be timed is itself contested. Respected voices counsel sinning only a little, while a growing literature finds that the factor premia themselves trendThe question is genuinely contested, and the evidence runs both ways. Asness, Chandra, Ilmanen & Israel (“Contrarian Factor Timing Is Deceptively Difficult,” Journal of Portfolio Management, 2017) find contrarian factor timing a weak addition for the diversified investor, while the same authors’ counsel to “sin a little” concerns timing the market itself on valuation rather than timing the styles (Asness, Ilmanen & Maloney, “Market Timing: Sin a Little,” Journal of Investment Management, 2017) — a temperament we borrow, not a finding about factors; on the other side, Gupta & Kelly (“Factor Momentum Everywhere,” Journal of Portfolio Management, 2019) and Ehsani & Linnainmaa (“Factor Momentum and the Momentum Factor,” Journal of Finance, 2022) document that factor returns themselves trend — a modest, disciplined scope for factor timing. One of us has been on the empirical side of this question: Brooks, Katsaris & Persand (“Timing Is Everything: A Comparison and Evaluation of Market Timing Strategies,” SSRN 834485, 2005) put a range of timing rules on common footing on a long span of S&P 500 data, testing each for sensitivity to its user-adjustable parameters — the arbitrary-choice objection that had beset the earlier literature — and find all but one beating buy-and-hold in risk-adjusted terms, the strongest built on the spread between the earnings-price ratio and short-term Treasury yields. The margins are thin and struck before frictions, though at a handful of round trips per decade the frictions are unlikely to reverse the ranking. We read it then as we read it now — the option is real and worth keeping open, and it is not a foundation. — and our own view is measured. Some factors can be timed some of the time (particularly when their behavioral, structural, or risk-sharing drivers line up in recognizable regimes), enough to keep the option open but not reliably enough to build the entire book upon. Either way, we do not rest our case on the answer. The case for soft control rests on the information the wholesale version destroys, and it is conditional: it stands only where the preserved interactions earn incremental value after costs and multiplicity. Hard orthogonalization forecloses the timing avenue forever and calls the amputation tidiness. Soft control keeps the avenue open at a price that can at least be named — an avenue whose payoff, as the last decade’s dispersion across risk-premia strategies shows, will be uneven across regimes.
And the price must indeed be named rather than waved away. The penalty’s weight is one more tuned number (one more fork in the researcher’s path, to be logged in the registry like every other choice). Whatever return arrives through the permitted factor timing must be reported to the allocator as exactly what it is, priced at the fee such exposure commands, rather than left for them to discover in a decomposition they had to run themselves. It is, in the end, the same philosophy as the risk controls with which we began: govern the exposure, do not disable the capability.
A constraint that binds forever does not always mean a training that has failed. Which constraint binds tells a different story each time.Or, in the terminology of numerical optimization, which constraint is active. The reader who would rather meet the whole portfolio allocation problem in that language — objectives, constraints, costs, and holdings as convex optimization, with no more finance than the problem strictly requires — will find it self-contained in Boyd, Busseti, Diamond, Kahn, Koh, Nystrup & Speth (“Multi-Period Trading via Convex Optimization,” Foundations and Trends in Optimization, 2017) — where the envelope of these pages is simply the feasible set. A concentration limit that binds every day may mean the model holds genuine, well-earned conviction — or that the limit has quietly assumed the sizing a conviction signal should have supplied. The two are told apart not by inspection but by counterfactual. Rerun the book with the limit relaxed and ask whether the returns change their shape, in which case the limit was binding on real signal, or merely their scale, in which case it was standing in for signal the model never found. A factor-exposure budget that binds persistently says something sharper and less comfortable: that the raw signal, left to itself, wants the factor — that the “alpha” is in part the very beta the budget exists to withhold. That is not overfitting. It is contamination, and it is exactly the quantity the allocator will later demand to see stripped away. The contractual neutralities (dollar and market) bind always and mean nothing by it; they are promises kept, not diagnostics. The lesson is not that binding is failure, but that persistent binding is a question, and the constraint that binds names the question being asked.
Uncertainty, incorporated
~Consider next how much we should believe the model on any given day — for the deadliest failure of a trading system is not the one that announces itself. A model does not only crash when it is wrong. More often than not, it goes on emitting confident, precise, mistaken numbers while the losses accumulate, and by the time the returns reveal the error the loss is irreversible. The defense is to treat the model’s own uncertainty as a first-class output, measured as carefully as the prediction itself, and in several independent ways, since no single measure deserves blind faith. Machine learning practice suggests training not one model but a committee of them, separately, and watching where they disagree. Where they diverge, that divergence is itself the plainest warning we have that the answer may be “we do not know.” In a similar spirit, a single model can be interrogated many times over with parts of it randomly silenced to see whether its conviction survives the perturbation. A sturdy opinion holds; a fragile one dissolves. But the instrument is one-sided. Disagreement is news, though never a verdict; agreement does not imply calibration: models raised on the same history inherit the same blind spots and can concur, fluently, on a shared delusion. We therefore treat divergence as a graduated brake rather than a binary veto: conviction and the capital behind it scale down as disagreement widens, all the way to zero at the extreme.A worked example — its numbers illustrative, not prescriptive — in the spirit of a two-member committee: give the senior model 75% of the weight and the junior 25%. When both like a name, deploy the full risk budget allotted to it. When the junior merely abstains, size down — to 60% of the budget, say, rather than the 75% the weights alone would allow. When the junior actively disagrees, cut the position to a third or so, deliberately steeper than the weights alone would imply, because disagreement is evidence about the weights themselves — but do not mechanically flatten it: disagreement is information about confidence, and it should scale the position rather than merely switch it off. The outright veto is reserved for the widest divergence, when the members of the committee are no longer describing the same world. Concurrence is never a proof. And the diversity on which the brake depends is not left to random initialization alone (which buys more independence than one might expect, and still not enough). It must be engineered. Different features, different windows of history, different architectures, different objectives. One confident voice is an opinion; a dozen voices in disagreement are a warning.Two standard ways to manufacture the dozen voices. Deep ensembles (Lakshminarayanan, Pritzel & Blundell, 2017) and Monte-Carlo dropout (Gal & Ghahramani, 2016) both furnish tractable estimates of predictive uncertainty. Disagreement across the ensemble approximates epistemic — model — uncertainty, which is the component that should rise when the input drifts away from the training distribution — and often does, though not reliably enough to be trusted alone: deep generative models have been shown to assign higher likelihood to data they have never seen than to the distribution they were trained on (Nalisnick, Matsukawa, Teh, Görür & Lakshminarayanan, “Do Deep Generative Models Know What They Don’t Know?,” ICLR 2019), which disposes of the comfortable intuition that the unfamiliar looks unfamiliar. That said, the theory these techniques possess must be read for what it promises. Monte-Carlo dropout is not theory-free — Gal & Ghahramani derive it as approximate variational inference in a deep Gaussian process, a genuine Bayesian pedigree — but the pedigree is not a coverage guarantee: their theorem says the procedure approximates a posterior, not how well, and not that its intervals will contain the truth at any stated rate. The approximation has itself been contested (Osband, 2016; Hron et al., 2018; Le Folgoc et al., 2021); its uncertainties need not concentrate with more data unless the dropout rates are themselves tuned, and ensembles and dropout alike are documented to grow overconfident precisely under distribution shift (Ovadia et al., “Can You Trust Your Model’s Uncertainty?,” NeurIPS 2019 — in whose tests, the deep ensemble degraded the most gracefully of the methods tried of the methods tried, one reason it is the alarm we lean on first). We therefore treat them as alarms rather than as calibrated instruments.
~Consider then what the model’s stated confidence actually promises — and, a question the fine print will force apart from it, whether anything in the system notices when the world has wandered somewhere the model has never been. The promise is the hallmark of a family of modern techniques (conformal methods chief among them) that supply statistically calibrated bands of uncertainty around a prediction without leaning on the comfortable assumption, so thoroughly false in markets, that returns are well-behaved and thin-tailed. Their first virtue is generality: in basic form they wrap around any model whatsoever and convert its bare point prediction into a calibrated set with a guaranteed rate of coverage. That universality is useful, and it is where most practitioners stop. But we believe that it is not where the real gain lies. A calibration bolted on after the fact can only describe the uncertainty of a model that was never built to reason about it. More is to be had from building and training the model from the outset to produce a prediction set rather than a lonely point estimate, so that awareness of its own uncertainty is baked into what it learns rather than measured once the learning is done. Conformalized quantile regression is the clean example of this marriage: a model trained to predict a range, then conformally calibrated so the range carries a rigorous statistical guarantee.The marriage is worth taking apart, because the two halves guarantee different things. Split (inductive) conformal prediction (Vovk, Gammerman & Shafer, Algorithmic Learning in a Random World, 2005) can wrap essentially any point predictor and, using only a held-out calibration set, return a prediction set with finite-sample coverage under exchangeability alone — no parametric assumption about the return distribution, which is attractive given fat tails. The jackknife+ of Barber, Candès, Ramdas & Tibshirani (Annals of Statistics, 2021) reaches a coverage guarantee of 1 − 2α through leave-one-out refits — the worst-case miscoverage doubled, from α to 2α, absent stability assumptions, and that doubling is the theorem rather than its fine print — while split conformal keeps 1 − α at the price of a held-out calibration set; variants for dependent, non-exchangeable sequences follow from the weighted schemes below. Conformalized quantile regression (Romano, Patterson & Candès, NeurIPS 2019) is the train-aware variant referenced in the text: a model is trained to output an interval, which is then conformally calibrated, so the interval adapts its width to local uncertainty (heteroscedasticity) while retaining the coverage guarantee. Every guarantee here is about how often, and none is about which day. A caution against hearing too much in these bands. At the signal-to-noise of return forecasting they are often so wide as to promise almost nothing about tomorrow’s number. Their use is not the point estimate they bracket but the counsel to abstain, to size small, or to sound the alarm on the day they yawn suddenly open.The same move goes further than one name at a time. For a vector response — the joint behavior of a book — the quantile function generalizes through optimal transport (Carlier, Chernozhukov & Galichon, “Vector Quantile Regression: An Optimal Transport Approach,” Annals of Statistics, 2016), and one of us has helped make it practical: nonlinear and fast at scale (Rosenberg, Vedula, Romano & Bronstein, “Fast Nonlinear Vector Quantile Regression,” ICLR, 2023), then continuous, a conditional optimal-transport map usable as a layer (Vedula, Tallini, Rosenberg, Pegoraro, Rodolà, Romano & Bronstein, “Continuous Vector Quantile Regression,” ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023). Two things should not be run together here: this models the conditional law, and a finite-sample coverage statement about it would still want a conformal calibration of its own. Our experience counsels modesty besides — mathematically beautiful as it is, vector quantile regression is useful in low dimension, and beyond a handful of coordinates the very notion of a prediction set begins to suffer the curse of dimensionality.
The fine print matters here, because the second deprivation waits inside it. Three offices must be kept apart. Calibration asks whether the bands cover at the promised rate while the world resembles the record they were tuned on; detection asks whether today still resembles it; adaptation decides what to do — widen, forget, de-risk — once it does not. Conformal machinery, in its predictive form, holds the first office and only the first — though the same p-values, read as a sequence rather than one day at a time, can be made to hold the second. The classical conformal guarantee rests on exchangeability (the premise that the days on which the machinery was calibrated and the day being predicted are, in distribution, interchangeable). Non-stationarity breaks that premise outright, as, more quietly, does serial dependence. Left unattended, the objection would hollow out the guarantee precisely where it is needed most. It has not been left unattended. The literature has met it head-on, first with weighted variants that tilt the calibration set toward days that resemble the present, and then with adaptive, online variants that track the coverage actually achieved and tune the bands as the world drifts, holding a weaker, long-run form of the promise without assuming the world sits still.The repairs are specific rather than general. Tibshirani, Barber, Candès & Ramdas (“Conformal Prediction Under Covariate Shift,” NeurIPS 2019) reweight the calibration set by the likelihood ratio between present and past — exactly valid when the inputs have moved and the law linking inputs to returns has not, and provably silent when that law is what moved; in markets the ratio is never known and must itself be estimated, which is the fine print that matters. The scheme that merely tilts toward recent, resembling days is the fixed-weight variant of Barber, Candès, Ramdas & Tibshirani (“Conformal Prediction Beyond Exchangeability,” Annals of Statistics, 2023), which asks less and pays for it with a bound on the coverage lost when exchangeability fails outright. Gibbs & Candès (“Adaptive Conformal Inference Under Distribution Shift,” NeurIPS 2021 — and, for arbitrary rather than gradual drift, “Conformal Inference for Online Prediction with Arbitrary Distribution Shifts,” Journal of Machine Learning Research, 2024) maintain long-run coverage by adjusting the working quantile online — and it is worth knowing how that promise is kept, since under hard drift the quantile is driven to its limit and the method emits intervals of unbounded width. That is the mechanism rather than a defect: the band widens until it counsels abstention, which is what this section says width is for. These are the instruments built for calibration under drift — non-stationary series their motivating case, though the covariate-shift machinery is far more general — and a fund that leans on conformal machinery should be expected to name which of them it runs. But the repair must be reported with the same candor as the flaw. What these methods restore is a weaker currency (coverage on average over the long run, adaptively maintained) and not a warranty for any particular regime. A fund that quotes the stronger promise while running the weaker machinery has misread its own instruments, a hazard to which we return in a moment.
A coverage guarantee, however, promises how often, not when. It states the rate at which the truth will fall inside the band, not the day the band has quietly ceased to mean anything. A calibrated set can keep its width, unmoved, on a world it has never seen.The two instruments just kept apart are perpetually conflated in practice, and the distinction is sharper than it looks. A conformal procedure delivers a statement about coverage — the marginal rate at which the realized value lands inside the predicted set — under exchangeability; in its predictive form it is not, in and of itself, a detector of distribution shift. Read sequentially it becomes one, and this is worth knowing because it answers in advance the deficiency this note closes on. The stream of conformal p-values is exchangeable while the world holds still, so a bet placed against that exchangeability compounds into a martingale whose growth is itself the alarm — anytime-valid by Ville’s inequality, and requiring no threshold fitted to the past, since the wealth is the evidence (Vovk, Nouretdinov & Gammerman, “Testing Exchangeability On-Line,” ICML 2003; Fedorova, Gammerman, Nouretdinov & Vovk, ICML 2012; and, for the betting formalism the argument now runs in, Ramdas, Grünwald, Vovk & Shafer, Statistical Science, 2023). One statistic, both bells: it monitors the joint law, and so hears a shift in the inputs and a shift in the relation alike, at the price of not telling you which it heard. Three questions must be held apart: whether the sets cover at the stated rate (predictive calibration); whether today’s inputs differ from those the model was trained and calibrated on (covariate shift); and whether the conditional law linking inputs to returns has moved beneath a stable input distribution (concept shift). A split-conformal interval can retain its width on a wholly unfamiliar input, and conformalized quantile regression leans on the underlying quantile model to widen — a model that may be confidently wrong precisely under shift. Width is not vigilance, and a band that holds its shape is no evidence that the world has held its own. The alarm for that day is a separate office, never a corollary of coverage — and whether it is discharged by a watch on the inputs or by those same p-values read as a sequence, it must be added deliberately rather than assumed to arrive with the calibration. A naive system, shown a world unlike any it trained on, extrapolates into it with the same serene confidence it brought to familiar ground — precisely how such systems come to grief in a crisis. A system equipped to detect that it has left its own map does the wise thing instead: it recognizes the unfamiliar, widens its bands, and withdraws risk until the ground is known again. We should not oversell the instrument. It has little power against a small shift, and in a world that never stops drifting, the unfamiliar is a matter of degree, and the threshold that tells ordinary evolution from true terra incognita is itself a number fitted to the past. The alarm is a disciplined prior, then, not an oracle — it earns its keep by being wired to shrink risk when it fires, never by promising foresight.
The watch must be kept on the inputs, not merely on the errors.The detection literature is older and better furnished than its absence from most prospectuses suggests, and, as with the conformal machinery of the note before last, a fund that claims a watch should be expected to name which one it runs. On the inputs, the standard instruments are distributional two-sample tests — the kernel maximum mean discrepancy (Gretton, Borgwardt, Rasch, Schölkopf & Smola, “A Kernel Two-Sample Test,” Journal of Machine Learning Research, 2012) and its classifier-based cousin, which trains a discriminator to tell yesterday’s data from today’s and reads the shift off its accuracy (Lopez-Paz & Oquab, “Revisiting Classifier Two-Sample Tests,” ICLR 2017) — with the practitioner’s comparison of what actually works in the wild given by Rabanser, Günnemann & Lipton (“Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift,” NeurIPS 2019). On the errors, the machinery is the change-point literature: the cumulative-sum chart (Page, “Continuous Inspection Schemes,” Biometrika, 1954), asymptotically optimal under a worst-case delay criterion (Lorden, Annals of Mathematical Statistics, 1971) and exactly optimal under the same criterion (Moustakides, Annals of Statistics, 1986). The survey that maps the whole field is Gama, Žliobaitė, Bifet, Pechenizkiy & Bouchachia (ACM Computing Surveys, 2014) — and it is worth reporting that the distinction it draws in setting up the problem, virtual drift against real drift, is the distinction we drew above between a shift in the inputs and a shift in the relation, arrived at independently. Their name has priority; ours, we would argue, says what it means. An alarm that waits for losses to accumulate rings only after the model has begun to be wrong, whereas the world’s unfamiliarity is visible in the data itself, before a single position has been staked on the resemblance. Sooner is not instantly. Nor should we exempt our own alarm from the arithmetic we have turned on everything else. A detector is a test like any other, so a shift gross enough to be obvious is seen at once, while one small enough to be merely lethal must accumulate its evidence like any other estimate, and how long that takes is set by its size and not by our impatience. Yet the earlier bell hears only one kind of trouble — a shift that announces itself in the inputs. The quieter failure, the law linking inputs to returns moving beneath a surface that still looks familiar, is invisible to the input watch and surfaces only in the errors, once capital has already leaned on the stale relation. A serious system therefore listens for both bells, knows which one it is hearing, and sizes its trust accordingly.
Retreat, however, is only half an answer to a world that drifts, and it is the passive half. The active half is adaptation. Models retrained on a rolling window at a declared cadence, conditioned on the regime, updated online within the envelope’s walls — the map redrawn as the territory changes, not merely abandoned at its edges. We commend it and in the same breath fence it, because adaptation is a temptation as much as a tool. The cadence of retraining and the rate of forgetting are themselves hyperparameters, fitted (where else?) to the very history whose treachery they are meant to answer. A model updated too eagerly does not adapt; it chases, promoting the last few months’ noise to the rank of law. The tradeoff is the oldest one in learning (plasticity purchased with stability), and no cleverness abolishes it.The tension is continual learning’s oldest, known there as the stability–plasticity dilemma (a term owed to Grossberg); its pathological pole is catastrophic forgetting (McCloskey & Cohen, 1989). The practical forms in a quantitative shop are walk-forward retraining on rolling windows and online updates with explicit forgetting factors; the cadence and the forgetting rate are hyperparameters like any others, and belong in the registry with the rest. The tradeoff has now been priced in this setting: Capponi, Huang, Sidaoui, Wang & Zou (“The Nonstationarity-Complexity Tradeoff in Return Prediction,” arXiv:2512.23596, 2025) decompose the error into misspecification, estimation, and drift, the last growing with the very window a complex model needs — under fast drift the simpler model on the shorter window wins, and the dilemma not repealed but priced. In plain terms: retrain and forget quickly enough to track a changing world, and you erode the stability of what you knew; hold the model still, and you surrender its ability to adapt. Between those two objectives there is no free lunch. So the discipline is the familiar one, applied once more. Declare the cadence in advance and log it in the registry like any other choice. Let the declared alarms (on the inputs, and on the errors themselves, read as drifting statistics rather than as grievances) decide when the map must be redrawn — never last month’s regret. And treat the retraining rule itself as part of the strategy’s specification: routine refits under a pre-validated policy are executions of the same strategy, while a change of method — new features, a new label, a new objective, a new architecture family, a new cadence — is a new strategy, owing the floor a fresh passage through its gates.
Of every tool of uncertainty quantification, ask one blunt question: what exactly does it promise? Five things are forever run together here, and it is worth keeping them apart: a model of the conditional distribution, a mechanism that calibrates it, a detector that notices when the world has moved, an adaptation that decides what to do once it has, and a rule that turns the width into a position size or an abstention. The three offices named above are the middle of this list; the model and the sizing rule stand at either end of it. None of the five implies another, and a system that has four of them has a hole where the fifth should be. The answers differ not in degree but in kind. A committee of models in agreement, a neural network that holds its opinion while parts of it are silenced: these guarantee nothing. They are the model’s opinion of its own opinion, an alarm and never a warranty — and such opinions are known to fail where they are needed most, far from familiar ground. A calibrated band promises something real, but only on average — and an insurer who is right on average can still be bankrupted by a single storm.Or a pithier metaphor: the poor statistician who drowned in a pool whose average depth was 25 centimeters. One can, and should, do better: the promise can be made to hold separately within each kind of day one thinks to name in advance — the violent days, the illiquid days, the days after a shockThere is a practical route to that conditional guarantee. One curates a library of named regimes — the crises, the liquidity droughts, the days after a shock — for which the ground truth is known, a collection most shops already maintain for stress testing, and calibrates the model’s stated confidence within those regimes rather than settling for coverage that holds only on average across all of them. Conditional calibration bought this way does not manufacture crisis data — in the rarest regimes the empirical guarantee is thinnest exactly because the category is thin — and its virtue is different: it makes the failure visible regime by regime, and forces the stress test the marginal promise lets one skip. Two conditions bound it, and both deserve stating plainly. Inside a bucket of n calibration days the attainable non-trivial coverage levels are the grid k/(n+1); asking for more than n/(n+1) — about ninety-five percent for a month of twenty-one trading days — returns the trivial, unbounded interval rather than a certificate, at any sample size the market will ever supply. And the label must be computable at the moment of prediction: “the violent days,” assigned from the realized volatility of the very window being predicted, is a look-ahead label, and it voids the guarantee it appears to strengthen — our own leakage, committed in the name of uncertainty quantification. — so that coverage is assessed separately in precisely the regimes where failure matters most. But no mathematics can extend the promise to the day one did not think to name; a guarantee covering even the unimagined kind of day is provably impossible to give.The impossibility is a theorem, not a lament: distribution-free conditional coverage — a guarantee holding on each kind of day rather than merely on average — is provably unattainable in any nontrivial sense (Vovk, 2012; Lei & Wasserman, 2014; Barber, Candès, Ramdas & Tibshirani, “The Limits of Distribution-Free Conditional Predictive Inference,” Information and Inference, 2021). What can be had is the intermediate: Mondrian and group-conditional conformal methods deliver the promise separately within each category of day named in advance — the qualification we insist on. The detection arm has its exact twin, and the symmetry is worth having: out-of-distribution detection is not learnable in general, and the conditions under which it becomes learnable have been characterized (Fang, Li, Lu, Dong, Han & Liu, “Is Out-of-Distribution Detection Learnable?,” NeurIPS 2022). One impossibility on each arm. Neither is a reason to forgo the instrument; both are reasons to say what it promises.
This is why calibration, however refined, must work in concert with the alarm we have just described. Calibration governs the days we knew to prepare for, while the watch on the inputs and the errors stands guard against the failure of imagination — the day no one thought to name.The phrase “a failure of imagination” attaches to astronaut Frank Borman’s testimony before Congress after the Apollo 1 fire of 1967 — not, as sometimes misremembered, Challenger. What Borman actually told the Senate hearings (Apollo Accident: Hearings before the Committee on Aeronautical and Space Sciences, 1967) was plainer. Pressed on whether the fire was negligence, he answered that he would prefer to call it an oversight, “since I feel that I share my full share of the blame for overlooking this problem.” The hazard had sat in plain sight: everyone had feared a fire in space, and no one had imagined one on the launch pad, during a routine test. The crisp three-word phrasing appears to be the gift of its dramatization in From the Earth to the Moon. The engineering lesson is ours: the fatal regime is the one left out of consideration. The hazard, in the end, is not the absence of a guarantee but the misreading of one — treating an opinion, or an on-average promise, as though it held on the one day that matters.
Governance and transparency are separable virtues, and it is worth being clear which one we are claiming. The uncertainty apparatus just described teaches a model the edges of its own competence; it does not pretend to lay its mind open to us, and the two should never be confused. There is a fashionable pose (that if one could explain what a model was doing it would not, by definition, be intelligence) and it is exactly backward. Opacity is a property to be managed, never a badge to be worn. We believe one can comfortably run a model whose weights resist narration, provided its inputs were chosen because they are relevant and not merely abundant, and provided its behavior can be attributed after the fact.In fact, the settlement we urge is already the tacit norm of daily life: people now delegate serious work to LLM-based systems that no one — least of all their makers — can fully explain, and the trust they extend rests, quite reasonably, on evidence of usefulness and the ability to check the output, not on an account of the mechanism. Society, like medicine, trusts the checkable black box; it merely declines to say so. A shop that feeds a model millions of signals it never troubled to interrogate, and then declines to ask why any of them fired, has mistaken abdication for sophistication. Attribution does not require narration. Run the model with a feature and without it, with a constraint relaxed and with it enforced, and the difference between the two books answers the question that actually mattersThese perturbation tests — ablation studies — are common in deep-learning practice. With correlated inputs they measure functional dependence rather than clean causal credit — a distinction that matters when two features proxy one another. The test says what the model leans on, not what the market rewards. — not the unanswerable “why does the model believe this?” but the tractable “which input, which component, drove this trade?” A momentum tilt, a value lean, a leverage that rises with conviction: once structure is imposed, these become legible mechanisms whose contribution can be measured even where the arithmetic beneath them cannot be recited. That is the interpretability an allocator should demand — and the only kind a data-starved, adversarial market permits us to honestly supply.
The market strikes back
~Consider, last, what the market will do about the model — for of the four deprivations, the adversary is the one that no cleverness inside the model can repeal. The discipline here is to assume, from the first line of code, that the market will do to this strategy what it has done to every strategy before it. One builds for the decay rather than being surprised by it. That begins with frictions. A backtest that trades free of cost portrays a generous world that does not exist (the one place where liquidity is unlimited and the spread is somebody else’s problem). The remedy is the same envelope philosophy with which we began: the costs of trading (the spread crossed, the price moved by one’s own footprint) belong inside the objective the model learns through, so that a strategy profitable only in a frictionless world is never learned at all, absent from the vocabulary in the same way the catastrophic position is. It continues with capacity. Every genuine edge is finite. A serious shop estimates in advance (and in writing) how much capital a signal can carry before it erases itselfThe first-order instrument is the square-root impact law — the empirical regularity that the price moved by a metaorder grows roughly as the square root of its size relative to daily volume (Tóth et al., “Anomalous Price Impact and the Critical Nature of Liquidity in Financial Markets,” Physical Review X, 2011); a direct estimation on equity data finds a concave temporary impact with an exponent near 0.6 (Almgren, Thum, Hauptmann & Li, “Equity Market Impact,” Risk, 2005) — related but not the same measurement, and the difference decides the answer: Tóth’s exponent is in the size of the metaorder relative to daily volume, Almgren’s in the rate of participation, and Almgren’s permanent component is modeled as linear in size, a form the data does not reject. A square-root temporary cost grows as the three-halves power of size and implies a soft ceiling; a linear permanent cost grows as the square and implies a much harder one. Worth adding that the square-root law is calibrated on metaorders running from a few ten-thousandths to a few percent of daily volume and is then extrapolated to the size at which the edge is exhausted, which is why the number is a bound rather than a measurement. From it, a capacity estimate follows as arithmetic: the signal’s gross edge, the decay of its forecast, and the cost of realizing it meet at a size beyond which the edge is consumed by its own footprint. The estimate is crude, which is no objection; it is written down before the fact, which is the point. And it must price the stressed market, not the average one: liquidity and crowding are endogenous, and the capacity that matters is the capacity of the market in which everyone wants out at once. — then checks that estimate against the slippage the live book actually pays, the one out-of-sample test nobody can rig. Significance, note, is economic before it is statistical. A robust effect can be uninvestable after costs, and a statistically faint signal can still earn its place in a diversified book. The arithmetic that decides is net of friction, at scale.The crossing is easy to state and hard to make, and one of us has made it twice. Brooks & Katsaris (“Trading Rules from Forecasting the Collapse of Speculative Bubbles for the S&P 500 Composite Index,” The Journal of Business, 2005) take two-regime models of periodically partially collapsing bubbles and ask not whether the effect is there but what an investor acting on their collapse probabilities would have earned, judged against buy-and-hold; a companion paper adds a third regime and repeats the exercise (Brooks & Katsaris, “A Three-Regime Model of Speculative Behavior: Modeling the Evolution of the S&P 500 Composite Index,” The Economic Journal, 2005). Both are candid about what the crossing costs. The first beats buy-and-hold on the Sharpe ratio even after transaction costs, and still leaves the investor with less terminal wealth than simply holding the index; the second reports that some of its superiority fades once costs are charged, and that the model “is not a ‘money machine.’” What survives the twenty years is the distinction rather than the trade: the model that detects something and the rule that trades it are different objects, and only the second has a profit and loss. The rule was also the more demanding client of the model — it wanted a probability calibrated well enough to size on, which no fit statistic ever asked for. It ends with a definition of failure. Before a strategy trades its first dollar, one writes down what evidence would mean it is dead (how much underperformance, over what horizon, would overwhelm the prior that it works). Beneath that final threshold sit the gentler ones: what summons review, what forces de-risking, what suspends. Each decision is made once, in cold blood, rather than renegotiated nightly with hope.Pre-registration is the operative analogy: clinical trials declare their endpoints and stopping rules before enrollment to forbid the after-the-fact reinterpretation that hope makes irresistible. The same logic governs the strategy graveyard — a kill criterion adopted after the losses have begun is not a criterion but a negotiation. The clinician will recognize the moves: the dose bounded before it is prescribed, the trial registered before it is run, the drug withdrawn when the field turns against it.
That completes the set, and the four are better seen together than apart. They are not a menagerie of clever tricks. They are one attitude, applied four times. Constrain what the model may do; calibrate how far it should be believed; detect when it has left the world it understands; and expect the world to strike back at whatever it does. Each answers one of the four deprivations, and the pieces lock together. The constraints decide what the model is permitted to express; the uncertainty machinery decides how loudly, on any given day, it is permitted to speak. Belief and capital are coupled by design, so that the system stakes the most when it knows the most, and steps aside precisely when it knows least. That coupling, more than any single algorithm, is what we mean by engineering generalization into a strategy.The objective is where much of the craft lives. A shop optimizes for explicit and idiosyncratic targets — some function of return, Sharpe, drawdown, turnover, and capacity, weighted to its own mandate — and the formulation of that objective routinely matters as much as the choice of architecture. The same holds for the training pipeline behind it: pre-training on broad history, fine-tuning to the traded regime, multi-stage objectives, and constraints that tighten or relax across stages. ML-based quant shops commonly treat this pipeline as their central intellectual property. Because the right choices are so tightly bound to a given universe and set of financial goals, we do not dwell on ours: a general account could only mislead, by pretending either that one objective is canonical or that a recipe transfers.
Sweetness and sting
Our doctrine ought not to be spared the suspicion it preaches. Every control in these pages has a price, and the prices are not all of one kind. The envelope forbids trades that would have paid, the abstention sits out days that would have rewarded boldness, the severe gate buries (along with a thousand illusions) the occasional genuine edge. Three claims of ours must be squared here. They look like contradictions and are not, and the squaring is a matter of naming which register each belongs to. A constraint built into the model is inductive bias: it trades bias for variance and so lowers the in-sample book while raising what survives out of sample — the gift. A limit imposed on the live system is insurance: it costs expectation and buys a bound on the loss — the tax. And the research discipline itself (registry, separation, retirement) manufactures nothing at all: it can reject a bad strategy and never produce a good one — the null. Gift, tax, and null, in three different registers. A fund can die of indiscipline; it can also die of discipline — more slowly and more respectably, its returns governed down to where the fees consume what the caution has spared. The remedy is not to loosen but to account. And the accounting has one more line than is comfortable: the control apparatus is itself a fitted object — thresholds, budgets, cadences, every one a number chosen against the same scarce history — and a shop should be able to say how many such numbers it carries, each owed to the registry like any other choice. The cost of every constraint is estimated in the same experiments that justify it, watched thereafter in the live book, and reported to the allocator as a price deliberately paid rather than left to be discovered as a disappointment. A manager who cannot name the cost of their controls has not measured it; and a control whose price nobody watches is usually either loose or performative.
?A harder objection remains, and it deserves a plain answer. Each of these instruments has been hedged here almost to vanishing. The floor rejects and never creates, the envelope is a doctrine and not a monopoly, the uncertainty apparatus offers alarms rather than warranties, the detector is a disciplined prior and not an oracle, and the capacity arithmetic is crude. Why should these insufficient instruments amount to a sufficient one? Because they are insufficient in different directions, and a strategy must pass all of them and go on passing.The argument is known outside finance under the yummy name of “Swiss cheese,” which is the only frivolous thing about it. Reason’s defense in depth holds that a system survives not because any single barrier is impenetrable but because several penetrable barriers have their holes in different places, so that a hazard must find a path through all of them at once (James Reason, Human Error, Cambridge University Press, 1990, where the model first appears in book form; the layered-slices picture arrives in Managing the Risks of Organizational Accidents, Ashgate, 1997, and the compact statement most often cited in “Human Error: Models and Management,” BMJ, 2000; the “Swiss cheese” label is generally credited not to Reason but to Rob Lee of Australia’s Bureau of Air Safety Investigation). Its documented failure mode is the one this paragraph goes on to name: the holes line up. A common cause collapses five layers into one labeled with five names. We borrow the form of the argument, and we inherit its caveat with it. The registry and the deflation read the search; the envelope reads the positions; the calibration reads the realized coverage; the detector reads the inputs and the errors; the capacity arithmetic reads the fills. That makes five evidentiary channels. A strategy wrong for one of the ordinary reasons (an overfitted search, an accidental exposure, a regime it never saw, a size the book cannot carry) is wrong in a way that at least one of them was built to see. What would sink that argument is a common cause: one model that proposes the strategy and calibrates its uncertainty and scores its drift, one vendor feeding all the channels, one researcher who tunes each threshold in turn until the apparatus falls quiet. Wherever the instruments share a cause they are worth no more than the cause — which is why the independence this architecture leans on is an institutional achievement before it is a technical one, and not the machine’s achievement at all.
The human factor
That brings us to the last element of this architecture, and the only one that is not engineering at all — its fifth essence — the word the ancients reached for when the four they had would not account for what they saw. We have not argued the human out of the room. We have argued quite the opposite. The machine is not the strategy. Behind it stands a person — in the regulated corners of finance answerable to supervisory expectations,The regulated corner of finance has, in fact, already articulated much of what this essay urges. The interagency guidance on model risk management issued by the Federal Reserve, the OCC, and the FDIC — SR 11-7 (2011; adopted by the FDIC in 2017), revised and superseded in April 2026 by SR 26-2 — expects of bank models, in proportion to their risk, an inventory, independent validation, documented limitations, and ongoing monitoring: governance of the opaque model as supervisory expectation, now in its second generation. The revision adds a wrinkle this essay’s readers should savor: a footnote places generative and agentic AI outside its formal scope, the agencies judging such technologies novel and rapidly evolving, while traditional statistical and machine-learning models remain squarely within it — a line drawn by model class and maturity rather than by what the four deprivations ask, and a gap this essay would close rather than endorse. A narrower European cousin, the ECB’s guide to internal models, runs only to the models banks use for regulatory capital, not to models at large; its own 2025 revision moves the other way, adding a section that brings machine-learning techniques explicitly within scope, explainability demanded and complexity made to justify itself. The securities regulators’ counterpart reaches the asset managers directly: IOSCO’s final report on the use of artificial intelligence and machine learning by market intermediaries and asset managers (FR06/2021) asks firms for named senior accountability, testing in segregation before deployment, managed model dependencies, and disclosure — governance again, and again stopping short of registries, effective counts, and a pre-declared definition of death. That the hedge-fund world, freer of such supervision, should reimpose the discipline on itself voluntarily is the settlement this paper argues investors ought to demand. and everywhere disciplined, imperfectly, by mandate, by fiduciary duty, and by whatever of their own capital rides alongside — whose judgment enters at every stage and answers for all of them. It enters at the beginning, in the design of the model and the data it is fed; in the middle, in the thousand reasoned choices no objective function will make for you (the horizon, the universe, what counts as an event at all); and at the end, where every failure the team can imagine is written into the envelope the model learns through, so that fear becomes structure rather than an afterthought bolted on. The guardrail is not a hand wrestling the wheel from the machine after the fact. It is the shape of the road we built for it to drive.
?And because rigor is enforced by people, governance is, in the end, a question of incentives before it is one of software. A registry no one is paid to respect will be respected accordingly. The questions that reveal the institution behind the machinery are blunt: who is rewarded for discovering a strategy, and who for killing one; who owns the model once it trades, and who can veto its deployment; whether the researcher whose experiment failed can rename it and resubmit; whether the validator answers to the portfolio manager whose bonus rides on the verdict. A shop that has never paid anyone for a funeral has told you what its registry is worth. Governance is not software. It is institutional design made compatible with the incentives of the people inside it.
This is not a contest between the quantitative and the human. It is a symbiosis: everything we do remains quantitative, data-driven, disciplined as a science; the human’s office is to guide what the model learns and to sanction what it is permitted to implement, never to overrule its arithmetic with a hunch. Overrides exist (for the corrupt feed, the broken exchange, the world visibly outside the model’s charter), but they come under explicit authority, logged, attributed, and reviewed after the fact: an act of governance, never of taste. Two ancient maxims govern the temperament this demands, and we have found no better. The first is μηδὲν ἄγαν — nothing in excess — inscribed at Delphi in the temple of the oracle — and it is our whole doctrine of controls in four words: match freedom to evidence, and distrust any conviction, the model’s or your own, that asks to be indulged without limit. The second is festina lente, make haste slowlyThe maxim is originally Greek — σπεῦδε βραδέως — a favorite, Suetonius reports (Divus Augustus, 25), of the emperor Augustus; festina lente is its Latin rendering. — the counsel that the surest way to arrive is to refuse to hurry, letting each strategy earn its capital at the pace this sparse and adversarial evidence can honestly afford. A fund built on those two attitudes will look, from the outside, unfashionably patient (slow, in the word an allocator is likely to use). That patience is its edge. A discipline once named is a discipline soon copied, and one would expect its edge to compete away like any other. We believe it will not, or at least not soon (otherwise we would not be writing this essay publicly). Unlike a shared signal, a soft-controlled envelope can be filled by countless distinct behaviors, so shops that adopt the same principles still diverge in what they become. Rigor is a common grammar, not a common trade — it leaves more room for competition than it closes.
Rigor before sophistication
Return, for a moment, to Toronto in 2012. The lesson usually drawn from that autumn and the decade it propelled is that scale conquers everything — that with enough data and enough computation, any problem yields. The lesson we draw is nearly the opposite, and we think it is the more useful one. Those triumphs were not won by scale alone. They were won by scale operating in worlds that happened to be generous (rich in data, stable in their rules, kind in their signal, forgiving in their standards). Bring the same machinery into a world that is none of these things and refuse to acknowledge the change, and you will produce exactly what the graveyard is full of: a magnificent backtest and a ruinous live record.
But the difficulties are not mysteries, and that fuels our optimism. Each of the markets’ four withheld gifts has a concrete governance answer, as these pages have argued: structure against the shortage of data; detection and controlled adaptation against the shifting ground; a candid accounting of uncertainty against the noise; and priced frictions, a sober arithmetic of capacity, and a pre-written definition of failure against the adversary. All of it rests, as a floor, on the statistician’s suspicion of the backtest. The medium horizon is uninhabitable by nature because all four deprivations bite there at once; it is habitable in practice because each of the four has an answer, and the answers hold together. We cannot prove the pocket is worth occupying — that is why we called it the contestable one — but the case for it is the engineering just set out, and nothing besides.
✓Finance is not alone in this predicament. Older, more sober disciplines learned its central lesson long ago. In much of clinical medicine a study of a few dozen patients can be a serious one; there, where lives ride on the answer and the data is scant, the record of fashionable deep-learning methods is poor — a parade of models that dazzled on the page and proved useless at the bedside, repeatedly failing to beat the humble logistic regression a statistician would have reached for first.The medical literature is unusually candid on this point. A systematic review by Christodoulou et al. (2019, Journal of Clinical Epidemiology) found no performance benefit of machine learning over logistic regression for clinical prediction; van der Ploeg, Austin & Steyerberg (2014, BMC Medical Research Methodology) show that flexible modern methods are markedly “data-hungry,” needing far more events per variable than classical models to reach stable performance (see also the events-per-variable rule of Peduzzi et al., 1996). For the deep-learning era specifically, Varoquaux & Cheplygina (2022, npj Digital Medicine) catalog methodological failures in medical-imaging ML, and Roberts et al. (2021, Nature Machine Intelligence) screened 2,212 papers on detecting or prognosticating COVID-19 from chest scans and examined the 62 retained for detailed analysis (37 of them deep-learning), finding not one model fit for clinical use — most undone by small, biased datasets and internal-only validation, with train-test contamination from recycled public sets a documented case rather than the rule, The failure modes are exactly the ones these pages are about. That field’s mature response has not been ever-larger networks but precisely the disciplines we have described: an honest accounting of how many models were tried, a ruthless suspicion of in-sample performance, and a preference for methods whose workings can be inspected. That last preference is the instructive one. The clinician’s logistic regression holds its own against the deep network not because it is transparent but because, on data that is scarce, it was the better-governed model (governed in the statistical sense, its capacity matched to its evidence). The lesson is not that simplicity wins but that governance does, in both of its arms, whatever the model’s size. Medicine prefers the inspectable model because its errors are unhedgeable (a single patient, a single irreversible outcome) and because its verdicts arrive in trial-sized doses, years apart. A fund’s errors are diversified across thousands of bets, and its profit and loss arrives daily, though its verdict does not arrive for years and sometimes never. The price of opacity is set very differently where mistakes can be survived, measured, and reversed — though only in ordinary weather: in a crisis the thousand bets collapse into one, liquidity departs exactly when diversification is wanted, and the daily verdict that soothed for years turns ruinous in a week.
The deeper habit of medicine points, in the end, our way: the same discipline that distrusts the opaque model prescribes the opaque molecule every morning, and has for generations. Lithium has steadied moods since the 1940s and no one can quite say how; general anesthesia has been practiced for nearly two centuries with its mechanism still a live question; aspirin sat in every household for some seventy years before anyone could explain what it did.Aspirin’s inhibition of prostaglandin synthesis was elucidated by John Vane only in 1971 (Nobel Prize, 1982), some seven decades into the drug’s mass use; the mechanism of paracetamol remains contested to this day; lithium’s, in clinical use against mania since Cade’s report of 1949, is unresolved; and the molecular basis of general anesthesia, administered since 1846, is still an active field of research. Complete mechanistic understanding has generally not been a condition of regulatory approval — evidence of safety and efficacy has, and modern practice can even fold mechanistic evidence into the confirmatory mix. Nor are the four exceptional: many an approved drug still keeps its precise molecular target to itself. The argument has been made for the clinic in nearly these words: London (“Artificial Intelligence and Black-Box Medical Decisions: Accuracy versus Explainability,” Hastings Center Report, 2019) holds that much of medicine is exactly this kind of atheoretic knowledge, aspirin his standing example, and that preferring an explainable model to an accurate one, absent evidence the trade pays, is a prejudice the patient pays for. We arrive by the same door; what changes in finance is the patient — a market that adapts to its medicine, which is why our settlement must borrow from the antibiotic as much as from the aspirin. Medicine has been governing black boxes for a century and a half without ever calling them that. Even the adversarial gift has a medical cousin: the antibiotic, whose efficacy is consumed by its own use, and which medicine answers not with a cleverer molecule but with stewardship (restraint in deployment, reserves of last resort, surveillance for resistance, retirement of the compromised), the very program of this essay.The stewardship carries the load because the laboratory, for once, cannot: nearly every class of antibiotic in clinical use was discovered in or before 1987, and the decades since are known in the field as the discovery void (Silver, “Challenges of Antibacterial Discovery,” Clinical Microbiology Reviews, 2011); the WHO’s pipeline reviews find, year after year, too few candidates and too little novelty among them (Analysis of Antibacterial Agents in Clinical and Preclinical Development: Overview and Analysis 2025, World Health Organization, 2025). The new molecule being scarce, restraint is not one instrument among several; it is most of the arsenal. Medicine, it turns out, has never demanded complete mechanism as the price of trust. It has demanded evidence — randomized, controlled, ruthlessly reviewed, and watched long after approval, with withdrawal always on the table for the drug that fails in the field — and the trust attaches to the protocol, not to the molecule. That is the settlement we are urging for the model.
The analogy breaks at randomization. The bedrock of medicine’s evidence is the randomized trial (the experimenter’s power to assign the drug by coin flip and so balance, in expectation, every confounder known and unknown), and that power is the one finance can never have. Nobody assigns bull markets at random. A live track record is an observational study, uncontrolled and unreplicated, conducted on a single patient who changes beneath the treatment. The arithmetic of our first deprivation applies to it with full force. At typical Sharpe ratios,Set out in full at the outset, where we said what we cannot prove: four years of live trading, at even odds, to distinguish a true Sharpe of 1.0 from zero, sixteen at 0.5 — and for an edge that decays while the record accumulates, possibly no number of years at all. And if a long record does eventually break the surface, read it before crediting a strategy with it. What clears the bar over a decade is rarely one edge that lasted. It is a parade of them, each found, promoted, traded, and retired once it had decayed below the friction of trading it, the next arriving before the last had finished dying. The record then measures the machinery that kept producing them, not any alpha within it. Even the allocator patient enough to wait out a track record ends up trusting a process after all. the years of track record required to distinguish a dead strategy from a merely unlucky one can exceed the patience of any investor and, where the edge is itself decaying, the life of the alpha the record was meant to certify. That is why even the kill criterion written in the coldest blood is right as governance while resting on modest statistical power. The conclusion, however, runs in our favor rather than against us. Precisely because the live evidence must arrive slowly and can never carry a trial’s authority, the burden of proof shifts earlier — onto the work done before the first trade, the constraints and the registries and the pre-committed definitions of failure. The settlement we urge is medicine’s in spirit, with the weights rearranged for a discipline that cannot randomize.
Our claim is deliberately limited. We do not claim to have set the black box ajar and rendered its every decision legible; a sufficiently flexible model on a sufficiently hard problem always retains a core that resists plain narration.AlphaFold is again the illuminating case. It is, strictly, a black box: it predicts folded structures without narrating the mechanism of folding, and it does not solve the Schrödinger equation that ultimately governs the underlying chemistry, whose exact solution is computationally intractable at this scale. Yet it has transformed structural biology and has helped legitimize a wider generative ecosystem that now designs proteins with functions found nowhere in nature (Huang, Boyken & Baker, “The Coming of Age of De Novo Protein Design,” Nature, 2016), and it is not wholly opaque: it exposes per-residue confidence scores and predicted-error maps that can be interrogated. Usefulness and mechanistic transparency are not the same virtue; a governed black box can forgo the second and still deliver the first in abundance. What we claim is narrower and, we think, far more useful: that such a model can be made to work — not by prying it open, but by surrounding it with discipline. The box need not become transparent. It needs to become governed: constrained in what it may do, honest about what it does not know, quick to shrink when the world turns unfamiliar, built for the decay of its own edge, and, beneath all four, never for a moment spared the statistician’s suspicion of its own track record. Trusted, in short, the way medicine trusts a drug: on the strength of its evidence and the vigilance of its watch, not on an account of its inner workings.
?This leads us to the conviction beneath that principle, and the surest mark of quality we know of in the field: rigor matters more than sophistication.The conviction has not been measured, but its shape has. Cassar & Gerakos (“Do Risk Management Practices Work? Evidence from Hedge Funds,” Review of Accounting Studies, 2017) put the question to hedge funds directly and found that those running formal quantitative risk models fared significantly better through the extreme down months of 2008, while dedicated risk officers and stated position limits showed no such effect. The null is the more useful half: what paid was the machinery inside the process, and what did not was the apparatus arranged around it — the distinction between the envelope and the overlay, a pattern that rhymes with the distinction between the envelope and the overlay, though the study tested neither. Dimmock & Gerken (“Predicting Fraud by Investment Managers,” Journal of Financial Economics, 2012) supply the other half of the shape, finding that disclosure items readable on a manager’s Form ADV before any money moves predict subsequent fraud. Neither study tested research governance, which is our subject and remains untested; both tested something beside it, observed in advance and scored against a later verdict, and both found signal where the return series alone would have found none. That is the most we claim: the conviction stays a conviction, and we have marked it as one. And rigor is, in the end, a human discipline before it is a technical one — only ever as real as the people who impose it, the very people we have just argued no fund can do without. The temptation of the Data Age is to believe the next disappointment will be cured by a larger model, a richer dataset, a cleverer architecture — that the answer to a failed black box is a more powerful one. It almost never is. A better model fails more expensively; a better process fails less often. The research discipline manufactures no alpha; a perfectly governed random-number generator is still a random-number generator. Its office is to keep noise from being mistaken for alpha, and to bound the price of the mistake when it is made. The funds that endure are not those with the most sophisticated models but those with the most disciplined process around whatever model they use: the ones that deploy the technique only where its assumptions hold, that trust their own backtests least, that spend their ingenuity on the humble machinery that keeps the machine honest rather than on enlarging it. We cannot prove this, and we should not pretend the argument compels it. Two things can be said, and they are not the same.
The first is well supported by adjacent evidence. On data this scarce, sophistication alone does not pay. The clinical record of the preceding pages — a useful warning from the nearest neighboring discipline with scarce, consequential data — finds flexible models failing to beat the humble ones,The analogy has a tension inside it, since our own discipline’s best-known machine-learning result appears to point the other way: gradient-boosted trees and shallow networks, handed hundreds of feeble predictors, beat the linear benchmarks on exactly the problem these pages are about (the result of note 30). The two records are reconciled by their conditions rather than by their methods. The clinical failures ran on small, biased samples and internal-only validation, with leakage — of the kind we spend a section on — flattering the flexible model rather than sinking it; the asset-pricing win is small, cross-sectional, and drawn from the pocket we endorse — the combination of very many feeble signals, none of them trusted alone. And where clinical data is abundant the flexible model does earn its place: one of us has built such a system for electrocardiography, where what made it usable at the bedside was not the accuracy alone but the uncertainty and the explanation delivered beside the diagnosis (Elul, Rosenberg, Schuster, Bronstein & Yaniv, “Meeting the Unmet Needs of Clinicians from AI Systems Showcased for Cardiology with Deep-Learning-Based ECG Analysis,” Proceedings of the National Academy of Sciences, 2021). Neither record licenses sophistication where the data is thin. and no reason offers itself why markets should be kinder. The second is a conviction, not a result. Among strategies of comparable ambition, the survivors are divided from the dead by the rigor of the process around them rather than the power of the model within. We hold it because the whole argument points that way — if the default of flexibility is a shining lie, survival must be decided by the machinery that catches the lie — but we cannot marshal a census to prove it, and we are bound by our own thesis to say so.
Nor does the discipline offer an easy escape from this modesty: the backtest cannot validate a strategy before it trades, and the live record, at the Sharpes such strategies earn, cannot validate it for years after. Between those two, some part of the trust we place in process rests on faith — earned, disciplined, watched, but faith. That admission is not a retreat from our conclusion; it is the form the conclusion must take in a discipline that cannot unambiguously attribute cause-effect relationships. A modest model built with and wrapped in serious rigor will, over a full cycle, quietly outlast a brilliant one that is merely admired. Skepticism toward “AI funds” is rational as a prior — the four deprivations promise a crowded field of failed claims — and it is owed to any particular fund until its claims survive tests commensurate with the flexibility of the process that produced them. But skepticism is not dismissal, and the practices we believe separate the survivors are observable and testable — registries, gates, kill criteria, the governed envelope; whether they in fact cause survival is the hypothesis this essay stakes, names, and offers up for refutation.

?A theory that forbids nothing is not a theory but a creed — Karl Popper’s test for a science, which we have applied implicitly to every strategy in these pages and must now, in honesty, turn upon ourselves.Popper supplied the principle without quite supplying the instrument: he could say that a theory must forbid something, not how much a theory had earned by surviving. Mayo’s severity supplies the missing quantity — data warrants a claim only to the degree that it has passed a test that would very probably have found the claim false had it been false (Deborah Mayo, Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars, Cambridge University Press, 2018). The demand has an older and shorter statement. Wittgenstein once greeted Elizabeth Anscombe by asking why people say it was natural to think the sun went round the earth rather than that the earth turned on its axis; told that “it looked that way,” he asked what it “would have looked like if it had looked as if the earth turned on its axis” (Anscombe, An Introduction to Wittgenstein’s Tractatus, Hutchinson, 1959). A backtest is that sky. It looks like skill, and it would have looked exactly the same had it been selection. On that reading the floor of these pages is one long severity argument, and its corollary is the sentence we have been writing in different words throughout: a search with enough freedom to flatter some strategy confers nothing on the strategy it flattered, whatever the t-statistic at the end. It is the criterion we borrow, not the quarrel. We have asked each strategy to declare, before it trades, the evidence that would prove it dead; it would be a shabby exemption to spare our own thesis the same test. So let us name what would sink us. We would abandon it if, over a full cycle, the governed funds proved no likelier to survive than the ungoverned — if discipline and blind luck sorted the living from the dead in equal measure. We would abandon it if the strategies that cleared the deflated Sharpe and the purged validation went on to die at the same rate as those that never faced them, for then our floor would be guarding an empty gate. And we would abandon it if the survivors proved to be marked not by the rigor of their process but by the raw wattage of their models — the very sophistication we have spent these pages demoting. And the test is runnable. The data exist, scattered but real: the governance an allocator can observe at the moment of investment — the registries shown or credibly attested, the kill criteria produced or merely promised, the builders separated from the judges or seated at one desk — matched, five and ten years on, against the roster of the living and the dead. The nearest such census already conducted comes from operational due diligence, and it found what we would expect it to find.The census has a partial precedent, nearer to it in design than the two studies of note 81, and a step closer to our subject. Brown, Goetzmann, Liang & Schwarz (“Trust and Delegation,” Journal of Financial Economics, 2012; and, earlier, “Estimating Operational Risk for Hedge Funds: The ω-Score,” Financial Analysts Journal, 2009) find that operational failures visible to a diligent examiner before investment — inadequate controls, misrepresentation, conflicts left unmanaged — predict subsequent fund death and underperformance. Process, observed ex ante, forecasts survival. The study speaks of operations rather than of research governance, and we do not claim more for it than it shows; but it is evidence, gathered by the profession’s own examiners, that discipline observable today separates tomorrow’s living from the dead — and it is the direction in which our conviction leans. The caution the body raises against our own census applies here with equal force: what such a study marks may be institutional quality at large rather than research discipline in particular. And we would wager the two disciplines keep company: the shop that maintains an honest registry of its research is, more often than not, the one that keeps honest books — rigor is a temperament before it is a technique, and it seldom confines itself to a single room. The wager runs both ways, which is the honest end of the matter: the very companionship that lets operations stand in for research is what stops the study from telling the two apart. We would welcome a fuller one, and we accept in advance the verdict of any honestly run.
One honesty the census itself demands: it would test a marker, not a mechanism. Governance travels with capital, talent, and institutional quality, and no observational design separates them. It is a retrospective study, as a clinician would say — the defect we charged to the live track record, now raised against our own evidence — and one that promptly demands a prospective one — and randomized, since only the coin flip tells correlation from causation and controls the confounders. That trial has no viable form here: it would assign governance at random and withhold it from a control arm that is somebody’s money, managed worse on purpose, for a decade. No synthetic demonstration — a governed and an ungoverned model raced on simulated, regime-switching histories — could substitute for it. This is the precise sense in which we said at the outset that we could argue this claim and hardly prove it. A conviction that could name no evidence against itself would be our own shining lie, and no better for being ours.
Having said that, even without ever being able to unambiguously demonstrate governance as the cause of success, it remains an observable that can serve as a marker, separating likely success from likely failure. Outlining such a diagnostic is what we turn to in closing.
A coda for the allocator: what to ask
The substance behind the coveted words “AI fund” has a describable shape, and it ends in questions one can put across a table. We have sat on both sides of this table. No serious shop will hand a prospect its registry, its code, or its models; the crown jewels stay in the vault, and rightly so. What these questions test is not whether the vault opens but whether it exists. A manager who cannot show the registry can still describe its schema and its size, name who holds its keys, state the count it feeds and the correction computed from it, and offer attestation where inspection is impossible. Listen to the shape of the refusal. The precise refusal is worth more than a thousand pages of marketing; the vague one answers the question by failing to understand it. “We cannot show you the registry; here is its schema, its size, the count it feeds, and the auditor who has seen it” is an answer. “We take data science very seriously” is another way of saying there is no registry. None of what follows is a private catechism. A version of this exercise has been attempted collectively, and it is worth knowing how far that consensus reached and where it stopped.The collective attempt has a name and a date. Standards Board for Alternative Investments, “Backtesting: Key Questions for Investors to Ask,” SBAI Toolbox, July 2020 — a memo written by a working group of some twenty managers and allocators, one of us among them, then at CdR Capital. Its ground is the hypothetical track record and the discipline of separating it from the live one, and it goes further than most: it asks how many trials were run, sets a minimum backtest length against that count, and names statistical overfitting outright. Where it stops is instructive. It does not reach the registry as an artifact whose existence can be attested, nor the effective number of independent trials as distinct from the raw one; it names an independent validation committee in a single clause without developing it; it touches learned models in one sentence; and it does not reach the pre-declared kill criterion at all. That is the ground the questions below set out to cover. We cite it less as authority than as a floor already conceded: an allocator who asks these things is not asking for anything the industry has not, collectively, agreed is fair. The academy had arrived a year earlier: Arnott, Harvey & Markowitz (“A Backtesting Protocol in the Era of Machine Learning,” Journal of Financial Data Science, 2019) set out a researcher’s protocol running from the economic motivation of an idea through multiple testing, data hygiene, and cross-validation to model dynamics, complexity, and the culture of the research organization. Protocol for the builder, memo for the buy side, and the questions below for the reader outside the vault: the same discipline, asked from three distances.
~It is worth saying what these questions replace. Too often, though not always, the evaluation runs like this, and it passes for rigor: take the backtest, halve its Sharpe ratio because backtests are known to flatter, require three to five years of live record before writing a check, and then — the record being ambiguous, as it always is — back the person. The pedigree, the prior fund, the team that made money somewhere else. It is the venture capitalist’s method moved into a domain where it does not hold: back the founder, and treat the record as a formality.
Each step fails in a way these pages have already named. Halving the Sharpe ratio is not a correction for a search nobody counted; it is a superstition with a number attached, and it cannot tell the first idea tried from the best of ten thousand.Note 23’s stylized accounting, at parameters no one would call exotic, puts the honest divisor nearer six than two — and that under the charitable reading in which the right strategy won its own search. Halving is not wrong in principle, it is only wrong about the divisor, and wrong again in applying the same divisor to every record regardless of the search that produced it. The remedy is the count that question 3 below asks to see. Three to five years is, at the Sharpe ratios medium-horizon strategies actually earn, too short to separate skill from luck. Backing the manager who succeeded before is the selection trap in its purest form, since prior success is exactly the variable on which the candidates were sorted. None of this is foolish. It is a reasonable answer to an impossible evidentiary position, and in the absence of anything better it is what a careful allocator is left with. Our claim is that something better is available, and that it can be read on the day the money moves rather than a decade after. Here is what to ask instead.
Ask what the strategy’s objectives are, and what, precisely, its model is trained to optimize. The two are not always the same thing.
Ask at what horizon the strategy trades. Then ask whether the techniques employed are the ones that horizon can honestly support.
Ask to see the experiment registry, the count of everything ever tried, and ask how the manager corrects the record for the count it contains: by deflation, by resampling, by an effective-N estimate, by whatever instrument they trust. Have the correction tied to the registry itself — or, where the registry must stay veiled, to a summary attested by someone with standing to look (the administrator, the auditor, an operational-diligence firm) rather than to memory. Ask what the attestation asserts. No standard yet covers this ground, so at a minimum: when the registry was opened, whether that date precedes the first live trade, that entries cannot be altered after the fact, and who holds the keys. A manager who will not have those four things attested has answered the question.
Ask what independent challenge stands between the builder of a strategy and the capital it is given. In a large shop, a separate office of certification; in a small one, a locked holdout under another’s key, an external attestation, a committee with the standing to refuse. The forms should scale with the firm. Having none of them is not a size. Ask, while there, about the incentives beneath the org chart: who is rewarded for killing a strategy, and whether a failed experiment can be renamed and resubmitted — a shop that pays only for births will bury nothing.
Ask for the live record set beside the backtest that preceded it, with the difference attributed: costs, spreads, slippage and impact, financing and borrow, universe and data differences, model revisions — down to an unexplained residual with the uncertainty around it stated. Luck is not a line item, and a bridge that hides one is not an attribution.
Ask how the data is kept honest: whether the universe contains its dead, whether the fundamentals are point-in-time or quietly restated, whether borrow and financing costs are charged, and whether adjusted price levels are ever fed to the model as features. The plumbing is where the cheapest lies live, and the question costs the manager nothing but candor.
Ask to be shown the environments in which the strategy is expected to underperform. A manager who cannot name them is claiming a strategy without weather, and there is no such thing.
Ask what the strategy’s capacity is believed to be, and how that belief has fared against the slippage actually paid.
Ask what evidence would trigger review, de-risking, suspension, or retirement — and which of those thresholds were written before the first trade rather than after the first loss.
Ask for the days on which uncertainty materially changed the capital deployed (including any spells of near-abstention). A fund that cannot name one has no working measure of its own ignorance.
Ask how often the outer risk limits have been touched, and why. The walls exist to stop corrupt feeds and dislocations, and an occasional strike is the system working. It is the model routinely rescued from positions of its own choosing whose training has failed.
Ask what the discipline costs: which trades the envelope forbids, what return it is estimated to forgo, how closely the constrained book still tracks the unconstrained one (the transfer coefficient, in the language of the fundamental law), and how that estimate has fared in the live book. A price no one has measured is a price no one is managing, and “we are conservative” is not a number. The evidence has known shapes: a shadow portfolio run without the constraint, an ablation, a constrained-against-unconstrained comparison — with error bars on the difference.
Ask what fraction of the return is factor exposure that could be bought for basis points. Ask for it with error bars: the betas are estimated precisely enough on a short record to answer it. What remains when that share is stripped away is a different matter. The residual carries exactly the standard error these pages call fatal, so the test bounds what is cheap without certifying what is left.What to do with the part that will not be certified is an old problem of the allocator’s, and one of us proposed a discipline for it before this paper’s subject had a name: Katsaris, Hegazi & Goulet (“A Total Risk Measurement Framework for Hedge Funds and Funds of Funds,” SSRN 1546522, 2010) build a fund’s return distribution from an estimated factor model, then add back the two components that model cannot see — a residual sized from the model’s own out-of-sample errors, and a tail sized from qualitative judgment where no data will ever supply one — before pricing what leverage, liquidity, and concentration do to the probability of an excess loss. The machinery is unremarkable and the discipline is not: the unexplained portion is measured, sized, and carried as a known unknown rather than promoted to alpha by default. Ask, too, for the simplest portfolio that nearly reproduces the record (the momentum tilt, the value-and-quality blend, the market beta, a volatility-managed rule, a plain cross-sectional ranker), and watch how much of the celebrated return still stands once the cheap and the ordinary have been paid their due. What five lines of code can already account for is not what active fees are for.
Ask what evidence is genuinely independent of the sample in which the idea was born (a different universe of stocks, a different geography, another asset class, another era where the claimed mechanism should travelThe demand is not rhetorical, and one of us has been on the answering end of it: the regime-switching bubble model built on the S&P 500 composite was later taken to a sample it was not built on — the ten sector indices beneath the same index, over a different span (Anderson, Brooks & Katsaris, “Speculative Bubbles in the S&P 500: Was the Tech Bubble Confined to the Tech Sector?,” Journal of Empirical Finance, 2010). What earns the citation is not that it worked but that the paper says where it did not: the bubble model was the best specification in four sectors of ten, and simpler specifications with no bubble in them won elsewhere — including in basic materials, where a run-up the industry had long filed as a bubble turned out not to behave like one. A manager whose out-of-sample evidence contains no sentence of that kind has not looked hard enough. ) and why that evidence is relevant to the mechanism. An edge may be honestly domain-specific, but then the burden shifts to the account of why.
Ask for the deployment cohort: how many strategies have gone live since inception, how many have been killed, and the distribution of backtest Sharpe minus live Sharpe across all of them, survivors and corpses alike. It is our own selection critique turned upon the fund itself; it opens no vault; and the answer is a handful of integers and one histogram.
And ask, last and always, for the persistence account: what kind of edge is this (behavioral, structural, institutional, microstructural, risk-bearing, statistical), what scarce resource does the fund supply, and why should the return survive capital, competition, and costs?
Obviously, this is not the whole story. These questions examine an investment process, not the operating business behind it. A separate examination is owed, and not attempted here: custody and valuation, counterparty and key-person risk, data licensing and vendor dependence, release and change management, incident response, and the authority to halt an automated system — a boundary of scope rather than of principle, for the two disciplines are one institutional temperament observed from different rooms.
✓When every question has been asked of the manager, one remains that the allocator must put to themselves. It is the oldest trap in this paper, pointed the other way. The manager across the table is the most impressive of many examined, and the best track record among a hundred candidates is a draw from the distribution of the maximum — our luckiest coin-flipper again, in a better suit, standing by chance alone some two and a half standard errors above the mean. The allocator who deflates the manager’s Sharpe ratio for the searches the manager ran, and leaves their own selection uncorrected for the hundred managers they screened, has policed everyone’s multiple testing but their own. The graveyard’s funds all had investors; the discipline this paper urges is, in the end, as much the reader’s as the manager’s.
We owe the allocator one thing more. We have argued that process evidence should get a manager in, and shown that the record cannot get one out inside any tolerable horizon. An allocator who accepts both has surrendered the only exit they had. The replacement is this paper’s own discipline turned outward: kill criteria for the relationship, declared before the money moves and observable in months and quarters rather than in a track record (no waiting on a significant return series). The registry’s count stops growing, or its effective-N is revised downward with no reason given. A retrain occurs that meets the manager’s own definition of a new strategy and does not go back through the floor. The cheap replicating portfolio’s fit against the returns climbs past an agreed line. Realized slippage breaches the written capacity estimate. The outer limits are touched more than a set number of times in a quarter, from positions the model chose. Nothing has been killed over a stretch in which the shop’s own deployment cadence says something should have been. Any of these is answerable from what a manager should already be producing, and any of them is a reason to ask the harder question long before the track record could ever be made to speak. And where a manager will open none of it, firmer ground remains. The handful of tests that need no trust at all (cheap-portfolio replication and factor stripping) recompute what is cheap from the returns themselves and bound the claim from below, and they are where an allocator should stand when the rest is refused — with one warning the reader will by now supply unprompted. A replica chosen as the best fit from a large enough library of factor sets, lags, and windows is a backtest like any other, and an allocator who searches for it has rebuilt the manager’s problem one floor up. Fix the family before fitting it.
These questions must themselves expect the fate of every instrument in these pages. Once allocators score registries, registries will be groomed for the scoring; the precision of a refusal can be rehearsed like anything else. What cannot be rehearsed is already in hand — the returns-only tests just named — and the rest of this list should be expected to decay like any signal, and to be revised the way one revises a model.
None of these questions requires the box to be opened; every one tests whether it is governed — the only question worth asking from the first page. A machine that can fold a protein and outplay a grandmaster arrives in the markets with no letters of introduction: its victories elsewhere neither qualify nor disqualify it, and here it must earn its evidence again — not by becoming more brilliant, but by becoming harder to fool; not because the box was finally opened, but because it was finally governed.
About the authors
Alex Bronstein is an electrical engineer and a computer scientist. He is a professor at the Institute of Science and Technology Austria and in the Department of Computer Science at the Technion — Israel Institute of Technology. Bronstein’s work spans inverse problems, computational imaging, machine learning, and data-driven modeling for science and engineering. He is a Fellow of the IEEE and of ELLIS. Alongside academia, he is a technological entrepreneur, inventor, and investor, having co-founded and served in senior technical roles across ventures ranging from depth cameras and large-scale video search to medical devices, food tech, and quantitative finance. Alex is a co-founder and Chief Scientist of the machine-learning-based hedge fund Sibylla.
Apostolos Katsaris is a quantitative finance researcher and systematic investor whose work spans asset allocation, hedge funds, alternative risk premia, market regimes, and the design and evaluation of quantitative strategies for institutional investors. Over a career of more than 25 years across academia and institutional investment management, he has held senior roles including Acting Co-CIO at PGIM Wadhwani, CIO and Partner at CdR Capital, and Partner at Albourne Partners. Alongside his investment career, Apostolos has published academic research in empirical finance and financial econometrics. He is the Chief Investment Officer of Sibylla.
Disclaimer
This paper is published for information and discussion only and it is our hope it will generate some. It is not investment advice. It is not a recommendation to buy or sell any security. It is not an offer or solicitation with respect to any fund or investment vehicle; any such offer would be made only by definitive documents, to eligible investors, and would be much more boring to read. The views expressed (and vigorously defended) are the authors’ own and not necessarily those of any firm or institution with which they are affiliated. The dashboard in the opening illustration is an invention: its figures were made up entirely for the drama, and no security appearing there or anywhere in these pages is recommended, forecast, or held out as a position. We feel compelled to repeat that past performance does not indicate future results, a proposition to which these pages have devoted an unusual quantity of arithmetic. Prediction is, in general, a very difficult business, especially about the future, so statements about it are subject to risks that may cause outcomes to differ materially. The authors and their affiliates may hold positions in the instruments discussed, and would think less of themselves if they did not.
References
Machine-learning work is cited by conference: NeurIPS (formerly NIPS), the Conference on Neural Information Processing Systems; ICML, the International Conference on Machine Learning; ICLR, the International Conference on Learning Representations; AAAI, the AAAI Conference on Artificial Intelligence; ICRA and IROS, the IEEE conferences on Robotics and Automation and on Intelligent Robots and Systems; ACML, the Asian Conference on Machine Learning; ICAIF, the ACM International Conference on AI in Finance; KDD, the ACM SIGKDD Conference on Knowledge Discovery and Data Mining; and ACM SIGSIM-PADS, the Conference on Principles of Advanced Discrete Simulation.
Abadie & Gardeazabal, “The Economic Costs of Conflict: A Case Study of the Basque Country,” American Economic Review, 2003.
Abadie, Diamond & Hainmueller, “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program,” Journal of the American Statistical Association, 2010.
Abramson et al., “Accurate Structure Prediction of Biomolecular Interactions with AlphaFold3,” Nature, 2024.
Achiam, Held, Tamar & Abbeel, “Constrained Policy Optimization,” ICML, 2017.
Agarwal, “Surrogate Regret Bounds for Bipartite Ranking via Strongly Proper Losses,” Journal of Machine Learning Research, 2014.
Agrawal, Amos, Barratt, Boyd, Diamond & Kolter, “Differentiable Convex Optimization Layers,” NeurIPS, 2019.
Almgren, Thum, Hauptmann & Li, “Equity Market Impact,” Risk, 2005.
Amos & Kolter, “OptNet: Differentiable Optimization as a Layer in Neural Networks,” ICML, 2017.
Anderson, Brooks & Katsaris, “Speculative Bubbles in the S&P 500: Was the Tech Bubble Confined to the Tech Sector?,” Journal of Empirical Finance, 2010.
Anscombe, An Introduction to Wittgenstein’s Tractatus, Hutchinson, 1959.
Arnott, Harvey & Markowitz, “A Backtesting Protocol in the Era of Machine Learning,” Journal of Financial Data Science, 2019.
Asness, Frazzini & Pedersen, “Quality Minus Junk,” Review of Accounting Studies, 2019.
Asness, Ilmanen & Maloney, “Market Timing: Sin a Little — Resolving the Valuation Timing Puzzle,” Journal of Investment Management, 2017.
Asness, Chandra, Ilmanen & Israel, “Contrarian Factor Timing Is Deceptively Difficult,” Journal of Portfolio Management, 2017.
Assefa et al., “Generating Synthetic Data in Finance: Opportunities, Challenges and Pitfalls,” ICAIF, 2020.
Bailey & López de Prado, “The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality,” Journal of Portfolio Management, 2014.
Bailey, Borwein, López de Prado & Zhu, “The Probability of Backtest Overfitting,” Journal of Computational Finance, 2017.
Bailey, Borwein, López de Prado & Zhu, “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance,” Notices of the American Mathematical Society, 2014.
Balabdaoui, Groeneboom & Hendrickx, “Score Estimation in the Monotone Single-Index Model,” Scandinavian Journal of Statistics, 2019.
Bank for International Settlements, “Volatility Is Back,” BIS Quarterly Review, March 2018.
Barber, Candès, Ramdas & Tibshirani, “Conformal Prediction Beyond Exchangeability,” Annals of Statistics, 2023.
Barber, Candès, Ramdas & Tibshirani, “The Limits of Distribution-Free Conditional Predictive Inference,” Information and Inference, 2021.
Barber, Candès, Ramdas & Tibshirani, “Predictive Inference with the Jackknife+,” Annals of Statistics, 2021.
Bartlett, Long, Lugosi & Tsigler, “Benign Overfitting in Linear Regression,” Proceedings of the National Academy of Sciences, 2020.
Belkin, Hsu, Ma & Mandal, “Reconciling Modern Machine-Learning Practice and the Classical Bias–Variance Trade-Off,” Proceedings of the National Academy of Sciences, 2019.
Benjamini & Yekutieli, “The Control of the False Discovery Rate in Multiple Testing under Dependency,” Annals of Statistics, 2001.
Blondel, Teboul, Berthet & Djolonga, “Fast Differentiable Sorting and Ranking,” ICML, 2020.
Board of Governors of the Federal Reserve System & OCC, “Supervisory Guidance on Model Risk Management,” SR 11-7, 2011 (adopted by the FDIC, 2017); superseded by the interagency SR 26-2, April 2026.
Boyd, Busseti, Diamond, Kahn, Koh, Nystrup & Speth, “Multi-Period Trading via Convex Optimization,” Foundations and Trends in Optimization, 2017.
Brehmer, Behrends, de Haan & Cohen, “Does Equivariance Matter at Scale?,” Transactions on Machine Learning Research, 2025.
Brooks & Katsaris, “A Three-Regime Model of Speculative Behavior: Modeling the Evolution of the S&P 500 Composite Index,” The Economic Journal, 2005.
Brooks & Katsaris, “Trading Rules from Forecasting the Collapse of Speculative Bubbles for the S&P 500 Composite Index,” The Journal of Business, 2005.
Brooks, Katsaris & Persand, “Timing Is Everything: A Comparison and Evaluation of Market Timing Strategies,” SSRN 834485, 2005.
Brown, Goetzmann, Liang & Schwarz, “Estimating Operational Risk for Hedge Funds: The ω-Score,” Financial Analysts Journal, 2009.
Brown, Goetzmann, Liang & Schwarz, “Trust and Delegation,” Journal of Financial Economics, 2012.
Buczynski, Cuzzolin & Sahakian, “A Review of Machine Learning Experiments in Equity Investment Decision-Making: Why Most Published Research Findings Do Not Live Up to Their Promise in Real Life,” International Journal of Data Science and Analytics, 2021.
Burges, Shaked, Renshaw, Lazier, Deeds, Hamilton & Hullender, “Learning to Rank Using Gradient Descent,” ICML, 2005.
Butler & Kwon, “Integrating Prediction in Mean-Variance Portfolio Optimization,” Quantitative Finance, 2023.
Byrd, Hybinette & Balch, “ABIDES: Toward High-Fidelity Multi-Agent Market Simulation,” ACM SIGSIM-PADS, 2020.
Cade, “Lithium Salts in the Treatment of Psychotic Excitement,” Medical Journal of Australia, 1949.
Cao, Qin, Liu, Tsai & Li, “Learning to Rank: From Pairwise Approach to Listwise Approach,” ICML, 2007.
Capponi, Huang, Sidaoui, Wang & Zou, “The Nonstationarity-Complexity Tradeoff in Return Prediction,” arXiv:2512.23596, 2025.
Carhart, “On Persistence in Mutual Fund Performance,” Journal of Finance, 1997.
Carlier, Chernozhukov & Galichon, “Vector Quantile Regression: An Optimal Transport Approach,” Annals of Statistics, 2016.
Cassar & Gerakos, “Do Risk Management Practices Work? Evidence from Hedge Funds,” Review of Accounting Studies, 2017.
Cederburg, O’Doherty, Wang & Yan, “On the Performance of Volatility-Managed Portfolios,” Journal of Financial Economics, 2020.
Cheridito & Weiss, “Reinforcement Learning for Trade Execution with Market and Limit Orders,” Quantitative Finance, 2026.
Chernoff & Savage, “Asymptotic Normality and Efficiency of Certain Nonparametric Test Statistics,” Annals of Mathematical Statistics, 1958.
Chordia, Subrahmanyam & Tong, “Have Capital Market Anomalies Attenuated in the Recent Era of High Liquidity and Trading Activity?,” Journal of Accounting and Economics, 2014.
Christodoulou, Ma, Collins, Steyerberg, Verbakel & Van Calster, “A Systematic Review Shows No Performance Benefit of Machine Learning over Logistic Regression for Clinical Prediction Models,” Journal of Clinical Epidemiology, 2019.
Clarke, de Silva & Thorley, “Portfolio Constraints and the Fundamental Law of Active Management,” Financial Analysts Journal, 2002.
Cossock & Zhang, “Statistical Analysis of Bayes Optimal Subset Ranking,” IEEE Transactions on Information Theory, 2008.
Cuturi, Teboul & Vert, “Differentiable Ranking and Sorting Using Optimal Transport,” NeurIPS, 2019.
Da, Nagel & Xiu, “The Statistical Limit of Arbitrage,” NBER Working Paper 33070, 2024.
Dickey & Fuller, “Distribution of the Estimators for Autoregressive Time Series with a Unit Root,” Journal of the American Statistical Association, 1979.
Dimmock & Gerken, “Predicting Fraud by Investment Managers,” Journal of Financial Economics, 2012.
Dinh, Pascanu, Bengio & Bengio, “Sharp Minima Can Generalize for Deep Nets,” ICML, 2017.
Donti, Amos & Kolter, “Task-based End-to-End Model Learning in Stochastic Optimization,” NIPS, 2017.
Ehsani & Linnainmaa, “Factor Momentum and the Momentum Factor,” Journal of Finance, 2022.
Elmachtoub & Grigas, “Smart ‘Predict, then Optimize’,” Management Science, 2022.
Elul, Rosenberg, Schuster, Bronstein & Yaniv, “Meeting the Unmet Needs of Clinicians from AI Systems Showcased for Cardiology with Deep-Learning-Based ECG Analysis,” Proceedings of the National Academy of Sciences, 2021.
Engelberg, Manela, Mullins & Vulicevic, “Entity Neutering,” SSRN 5182756, 2025.
Engle & Granger, “Co-Integration and Error Correction: Representation, Estimation, and Testing,” Econometrica, 1987.
European Central Bank, ECB Guide to Internal Models, version 4.0 (July 2025), amended June 2026 — successor to the Targeted Review of Internal Models (TRIM).
Fama & French, “Common Risk Factors in the Returns on Stocks and Bonds,” Journal of Financial Economics, 1993.
Fama & French, “The Cross-Section of Expected Stock Returns,” Journal of Finance, 1992.
Fama & MacBeth, “Risk, Return, and Equilibrium: Empirical Tests,” Journal of Political Economy, 1973.
Fang, Li, Lu, Dong, Han & Liu, “Is Out-of-Distribution Detection Learnable?,” NeurIPS, 2022.
Fawzi et al., “Discovering Faster Matrix Multiplication Algorithms with Reinforcement Learning,” Nature, 2022.
Fedorova, Gammerman, Nouretdinov & Vovk, “Plug-in Martingales for Testing Exchangeability On-Line,” ICML, 2012.
Financial Times, “Why Hedge Fund Managers Are Happy to Let the Machines Take Over,” October 17, 2019.
Frazzini & Pedersen, “Betting Against Beta,” Journal of Financial Economics, 2014.
Freedman, Rozenberg & Bronstein, “A Theoretical Framework for an Efficient Normalizing Flow-Based Solution to the Electronic Schrödinger Equation,” AAAI, 2025.
Gal & Ghahramani, “Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,” ICML, 2016.
Gama, Žliobaitė, Bifet, Pechenizkiy & Bouchachia, “A Survey on Concept Drift Adaptation,” ACM Computing Surveys, 2014.
Gao, Jiang & Yan, “Detecting Lookahead Bias in LLM Forecasts,” arXiv:2512.23847, 2026.
Gelman & Loken, “The Garden of Forking Paths,” Department of Statistics, Columbia University, 2013.
Gibbs & Candès, “Adaptive Conformal Inference Under Distribution Shift,” NeurIPS, 2021.
Gibbs & Candès, “Conformal Inference for Online Prediction with Arbitrary Distribution Shifts,” Journal of Machine Learning Research, 2024.
Glasserman & Lin, “Assessing Look-Ahead Bias in Stock Return Predictions Generated by GPT Sentiment Analysis,” Journal of Financial Data Science, 2024.
Göbel, Sander, Schneider & Valencia, “Correlated Mutations and Residue Contacts in Proteins,” Proteins, 1994.
Gretton, Borgwardt, Rasch, Schölkopf & Smola, “A Kernel Two-Sample Test,” Journal of Machine Learning Research, 2012.
Grinold & Kahn, Active Portfolio Management, 2nd ed., McGraw-Hill, 2000.
Grossberg, “How Does a Brain Build a Cognitive Code?,” Psychological Review, 1980.
Gu, Kelly & Xiu, “Empirical Asset Pricing via Machine Learning,” Review of Financial Studies, 2020.
Gupta & Kelly, “Factor Momentum Everywhere,” Journal of Portfolio Management, 2019.
Han, “Non-Parametric Analysis of a Generalized Regression Model: The Maximum Rank Correlation Estimator,” Journal of Econometrics, 1987.
Han & Wellner, “Convergence Rates of Least Squares Regression Estimators with Heavy-Tailed Errors,” Annals of Statistics, 2019.
Hansen, “A Test for Superior Predictive Ability,” Journal of Business & Economic Statistics, 2005.
Hansen, Maddix, Alizadeh, Gupta & Mahoney, “Learning Physical Models that Can Respect Conservation Laws,” ICML, 2023.
Harvey & Liu, “Backtesting,” Journal of Portfolio Management, 2015.
Harvey & Liu, “False (and Missed) Discoveries in Financial Economics,” Journal of Finance, 2020.
Harvey & Liu, “Lucky Factors,” Journal of Financial Economics, 2021.
Harvey, Liu & Zhu, “…and the Cross-Section of Expected Returns,” Review of Financial Studies, 2016.
He, Lv, Manela & Wu, “Chronologically Consistent Large Language Models,” arXiv:2502.21206, 2025.
Horowitz, “A Smoothed Maximum Score Estimator for the Binary Response Model,” Econometrica, 1992.
Hron, Matthews & Ghahramani, “Variational Bayesian Dropout: Pitfalls and Fixes,” ICML, 2018.
Huang, Boyken & Baker, “The Coming of Age of De Novo Protein Design,” Nature, 2016.
International Organization of Securities Commissions, The Use of Artificial Intelligence and Machine Learning by Market Intermediaries and Asset Managers: Final Report, FR06/2021, September 2021.
Israel, Kelly & Moskowitz, “Can Machines ‘Learn’ Finance?,” Journal of Investment Management, 2020.
Jaegle, Gimeno, Brock, Vinyals, Zisserman & Carreira, “Perceiver: General Perception with Iterative Attention,” ICML, 2021.
Jegadeesh & Titman, “Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency,” Journal of Finance, 1993.
Johansen, “Statistical Analysis of Cointegration Vectors,” Journal of Economic Dynamics and Control, 1988.
Jumper et al., “Highly Accurate Protein Structure Prediction with AlphaFold,” Nature, 2021.
Kalbfleisch & Prentice, “Marginal Likelihoods Based on Cox’s Regression and Life Model,” Biometrika, 1973.
Katsaris, Hegazi & Goulet, “A Total Risk Measurement Framework for Hedge Funds and Funds of Funds,” SSRN 1546522, 2010.
Kelly & Malamud, “Understanding the Virtue of Complexity,” Swiss Finance Institute Research Paper 25-96, 2025.
Kelly, Malamud & Zhou, “The Virtue of Complexity in Return Prediction,” Journal of Finance, 2024.
Khandani & Lo, “What Happened to the Quants in August 2007? Evidence from Factors and Transactions Data,” Journal of Financial Markets, 2011.
Kim & Pollard, “Cube Root Asymptotics,” Annals of Statistics, 1990.
Krishnapriyan, Gholami, Zhe, Kirby & Mahoney, “Characterizing Possible Failure Modes in Physics-Informed Neural Networks,” NeurIPS, 2021.
Krizhevsky, Sutskever & Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” NIPS, 2012.
Kwiatkowski, Phillips, Schmidt & Shin, “Testing the Null Hypothesis of Stationarity Against the Alternative of a Unit Root: How Sure Are We That Economic Time Series Have a Unit Root?,” Journal of Econometrics, 1992.
Lakshminarayanan, Pritzel & Blundell, “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles,” NIPS, 2017.
Le Folgoc, Baltatzis, Desai, Devaraj, Ellis, Martinez Manzanera, Nair, Qiu, Schnabel & Glocker, “Is MC Dropout Bayesian?,” arXiv:2110.04286, 2021.
LeBaron, “Agent-Based Computational Finance,” in Handbook of Computational Economics, vol. 2, 2006.
Ledoit & Wolf, “Honey, I Shrunk the Sample Covariance Matrix,” Journal of Portfolio Management, 2004.
Lehmann & Romano, Testing Statistical Hypotheses, 3rd ed., Springer, 2005.
Lei & Wasserman, “Distribution-Free Prediction Bands for Non-Parametric Regression,” Journal of the Royal Statistical Society, Series B, 2014.
Li, Yang, Yang, Xu, Wang, Liu & Bian, “R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization,” NeurIPS, 2025.
Liu, “Learning to Rank for Information Retrieval,” Foundations and Trends in Information Retrieval, 2009.
Lo, “The Adaptive Markets Hypothesis: Market Efficiency from an Evolutionary Perspective,” Journal of Portfolio Management, 2004.
Lo, “The Statistics of Sharpe Ratios,” Financial Analysts Journal, 2002.
Lo & MacKinlay, “Stock Market Prices Do Not Follow Random Walks: Evidence from a Simple Specification Test,” Review of Financial Studies, 1988.
London, “Artificial Intelligence and Black-Box Medical Decisions: Accuracy versus Explainability,” Hastings Center Report, 2019.
López de Prado, “The 10 Reasons Most Machine Learning Funds Fail,” Journal of Portfolio Management, 2018.
López de Prado, Advances in Financial Machine Learning, Wiley, 2018.
López de Prado, “Building Diversified Portfolios that Outperform Out of Sample,” Journal of Portfolio Management, 2016.
Lopez-Paz & Oquab, “Revisiting Classifier Two-Sample Tests,” ICLR, 2017.
Lorden, “Procedures for Reacting to a Change in Distribution,” Annals of Mathematical Statistics, 1971.
Lu, Pestourie, Yao, Wang, Verdugo & Johnson, “Physics-Informed Neural Networks with Hard Constraints for Inverse Design,” SIAM Journal on Scientific Computing, 2021.
Lucas, “Econometric Policy Evaluation: A Critique,” Carnegie-Rochester Conference Series on Public Policy, 1976.
Mao, Weed & Rigollet, “Minimax Rates and Efficient Algorithms for Noisy Sorting,” Proceedings of Algorithmic Learning Theory, 2018.
Marks, Colwell, Sheridan, Hopf, Pagnani, Zecchina & Sander, “Protein 3D Structure Computed from Evolutionary Sequence Variation,” PLoS ONE, 2011.
Martin & Nagel, “Market Efficiency in the Age of Big Data,” Journal of Financial Economics, 2022.
Martineau, “Rest in Peace Post-Earnings Announcement Drift,” Critical Finance Review, 2022.
Mayo, Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars, Cambridge University Press, 2018.
McCloskey & Cohen, “Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem,” in Psychology of Learning and Motivation, vol. 24, Academic Press, 1989.
McLean & Pontiff, “Does Academic Research Destroy Stock Return Predictability?,” Journal of Finance, 2016.
Moreira & Muir, “Volatility-Managed Portfolios,” Journal of Finance, 2017.
Moustakides, “Optimal Stopping Times for Detecting Changes in Distributions,” Annals of Statistics, 1986.
Nagel, Machine Learning in Asset Pricing, Princeton University Press, 2021.
Nagel, “Seemingly Virtuous Complexity in Return Prediction,” NBER Working Paper 34104, 2025.
Nalisnick, Matsukawa, Teh, Görür & Lakshminarayanan, “Do Deep Generative Models Know What They Don’t Know?,” ICLR, 2019.
Navon, Shamsian, Achituve, Fetaya, Chechik & Maron, “Equivariant Architectures for Learning in Deep Weight Spaces,” ICML, 2023.
Novikov et al., “AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery,” Google DeepMind technical report, 2025.
Osband, “Risk versus Uncertainty in Deep Learning: Bayes, Bootstrap and the Dangers of Dropout,” NIPS Workshop on Bayesian Deep Learning, 2016.
Ovadia et al., “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty under Dataset Shift,” NeurIPS, 2019.
Page, “Continuous Inspection Schemes,” Biometrika, 1954.
Paindaveine, “A Unified and Elementary Proof of Serial and Nonserial, Univariate and Multivariate, Chernoff–Savage Results,” Statistical Methodology, 2004.
Peduzzi, Concato, Kemper, Holford & Feinstein, “A Simulation Study of the Number of Events per Variable in Logistic Regression Analysis,” Journal of Clinical Epidemiology, 1996.
Perdomo, Zrnic, Mendler-Dünner & Hardt, “Performative Prediction,” ICML, 2020.
Phillips & Perron, “Testing for a Unit Root in Time Series Regression,” Biometrika, 1988.
Popper, Conjectures and Refutations: The Growth of Scientific Knowledge, Routledge & Kegan Paul, London, 1963.
Rabanser, Günnemann & Lipton, “Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift,” NeurIPS, 2019.
Ramdas, Grünwald, Vovk & Shafer, “Game-Theoretic Statistics and Safe Anytime-Valid Inference,” Statistical Science, 2023.
Reason, Human Error, Cambridge University Press, 1990.
Reason, “Human Error: Models and Management,” BMJ, 2000.
Reason, Managing the Risks of Organizational Accidents, Ashgate, 1997.
Recht, Roelofs, Schmidt & Shankar, “Do ImageNet Classifiers Generalize to ImageNet?,” ICML, 2019.
Reed et al., “A Generalist Agent,” Transactions on Machine Learning Research, 2022.
Roberts et al., “Common Pitfalls and Recommendations for Using Machine Learning to Detect and Prognosticate for COVID-19 Using Chest Radiographs and CT Scans,” Nature Machine Intelligence, 2021.
Romano & Wolf, “Stepwise Multiple Testing as Formalized Data Snooping,” Econometrica, 2005.
Romano, Patterson & Candès, “Conformalized Quantile Regression,” NeurIPS, 2019.
Romera-Paredes et al., “Mathematical Discoveries from Program Search with Large Language Models,” Nature, 2024.
Rosenberg, Vedula, Romano & Bronstein, “Fast Nonlinear Vector Quantile Regression,” ICLR, 2023.
Sarkar & Vafa, “Lookahead Bias in Pretrained Language Models,” SSRN 4754678, 2024.
Schrimpf, Shin & Sushko, “Leverage and Margin Spirals in Fixed Income Markets during the Covid-19 Crisis,” BIS Bulletin No. 2, 2020.
Sherman, “The Limiting Distribution of the Maximum Rank Correlation Estimator,” Econometrica, 1993.
Silver, “Challenges of Antibacterial Discovery,” Clinical Microbiology Reviews, 2011.
Silver et al., “Mastering the Game of Go Without Human Knowledge,” Nature, 2017.
Stambaugh, Yu & Yuan, “The Short of It: Investor Sentiment and Anomalies,” Journal of Financial Economics, 2012.
Standards Board for Alternative Investments, “Backtesting: Key Questions for Investors to Ask,” SBAI Toolbox, Alternative Risk Premia Working Group, July 2020.
Strassen, “Gaussian Elimination Is Not Optimal,” Numerische Mathematik, 1969.
Strathern, “‘Improving Ratings’: Audit in the British University System,” European Review, 1997.
Suetonius, Divus Augustus, in The Twelve Caesars, c. 121 CE.
Sullivan, Timmermann & White, “Data-Snooping, Technical Trading Rule Performance, and the Bootstrap,” Journal of Finance, 1999.
Sutton, “The Bitter Lesson,” incompleteideas.net, 2019.
Tang, Chen, Yang, Mai, Zheng, Wang, Chen & Lin, “AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay,” KDD, 2025.
Tibshirani, Barber, Candès & Ramdas, “Conformal Prediction Under Covariate Shift,” NeurIPS, 2019.
Tóth, Lempérière, Deremble, de Lataillade, Kockelkoren & Bouchaud, “Anomalous Price Impact and the Critical Nature of Liquidity in Financial Markets,” Physical Review X, 2011.
U.S. Securities and Exchange Commission, Press Release 2024-36, March 2024.
United States Senate, Apollo Accident: Hearings before the Committee on Aeronautical and Space Sciences, Ninetieth Congress, 1967.
van der Ploeg, Austin & Steyerberg, “Modern Modeling Techniques Are Data Hungry: A Simulation Study for Predicting Dichotomous Endpoints,” BMC Medical Research Methodology, 2014.
Vane, “Inhibition of Prostaglandin Synthesis as a Mechanism of Action for Aspirin-like Drugs,” Nature New Biology, 1971.
Varoquaux & Cheplygina, “Machine Learning for Medical Imaging: Methodological Failures and Recommendations for the Future,” npj Digital Medicine, 2022.
Vaswani et al., “Attention Is All You Need,” NIPS, 2017.
Vedula, Tallini, Rosenberg, Pegoraro, Rodolà, Romano & Bronstein, “Continuous Vector Quantile Regression,” ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023.
Vovk, “Conditional Validity of Inductive Conformal Predictors,” ACML, 2012.
Vovk, Gammerman & Shafer, Algorithmic Learning in a Random World, Springer, 2005.
Vovk, Nouretdinov & Gammerman, “Testing Exchangeability On-Line,” ICML, 2003.
Weiss, Senouf, Vedula, Michailovich, Zibulevsky & Bronstein, “PILOT: Physics-Informed Learned Optimized Trajectories for Accelerated MRI,” Machine Learning for Biomedical Imaging, 2021.
White, “A Reality Check for Data Snooping,” Econometrica, 2000.
Wiese, Knobloch, Korn & Kretschmer, “Quant GANs: Deep Generation of Financial Time Series,” Quantitative Finance, 2020.
World Health Organization, Analysis of Antibacterial Agents in Clinical and Preclinical Development: Overview and Analysis 2025, 2025.
Xie, Dai, Chen, Dai, Zhao, Zha, Wei & Pfister, “Differentiable Top-k with Optimal Transport,” NeurIPS, 2020.
Zadok, Salzman, Wolf & Bronstein, “Toward Predicting Fine Finger Motions from Ultrasound Images via Kinematic Representation,” ICRA, 2023.
Zadok, Wolf, Bronstein & Salzman, “SonoRank: Toward Calibration-Free Real-Time Finger Flexion Detection from Forearm Ultrasound Sequences,” IROS, 2026.
Zhang, Bengio, Hardt, Recht & Vinyals, “Understanding Deep Learning Requires Rethinking Generalization,” ICLR, 2017.
Zuckerman, The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution, Portfolio, 2019.