Correlation analysis and portfolio diversification.

The diversification myth

Open any introductory portfolio textbook and you will find the same claim: hold more assets, diversify more. The intuition is correct in the limit but misleading in practice. A portfolio of fifty correlated assets behaves, in stress, like a portfolio of one. The relevant question is not how many assets but how many independent sources of return — and that is a question about correlation, not count.

Correlation, not concentration, is the diversification primitive. Two assets with a 0.95 correlation contribute approximately the risk of one position. Two assets at 0.20 contribute close to two. The arithmetic of variance reduction is dominated by the correlation matrix, and the rest is rounding.

This is the first reason institutional risk teams obsess over the structure of the correlation matrix rather than the marginal volatility of individual sleeves. The second reason is that the matrix is not stable. It moves with the regime — and it moves against you precisely when you need it most.

Static correlation is the wrong tool

The standard practitioner workflow estimates correlation from a long, full-history window, plugs it into a mean-variance optimiser, and treats the output as an allocation. Two empirical facts make this approach fragile.

First, full-history correlation is dominated by long stretches of normal-regime data and underweights the short, intense periods when correlation actually matters for portfolio survival. Second, correlation is non-stationary. The cross-asset correlation observed in 2017 has almost no predictive power for the correlation observed in March 2020, October 2008, or any other tail event.

Rolling correlation — typically a 60- to 120-day window — is the minimum upgrade. It surfaces correlation drift in real time and forces the optimiser to react to current rather than historical structure. But rolling windows still smooth across regimes. They detect change late and they over-fit short windows. The institutional answer is conditional, not rolling.

Conditional and regime-aware correlation

Conditional correlation measures pairwise correlation inside a defined regime — high-volatility days, drawdowns, central-bank surprises, geopolitical shocks. The DCC-GARCH family of models (Engle, 2002) and copula-based approaches make this tractable. The output is a separate correlation matrix for each regime, and the empirical pattern is consistent across asset classes:

Equity-bond correlation flips sign under inflation regimes. Cross-currency correlation collapses to one during dollar-funding stress. Credit spreads correlate with equity drawdowns far more strongly than with equity rallies. Commodities decouple from risk assets in supply shocks but couple tightly during demand shocks.

These are not academic curiosities. A portfolio that looks decorrelated under unconditional historical correlation can be highly concentrated under stress. The correct discipline is to size positions against the worst-regime correlation the portfolio is plausibly exposed to, not the average. Stress-conditional correlation is what a real risk budget needs to clear.

Hierarchical clustering: structure beyond pairs

Pairwise correlation hides higher-order structure. Three assets can each be moderately correlated pairwise yet load on the same single underlying factor, producing a portfolio with one effective bet rather than three. Hierarchical clustering — agglomerative or divisive, with correlation distance d = sqrt(1 - ρ²) as the metric — exposes this geometry.

The dendrogram makes the cluster structure of the universe visually explicit. A US-equity sector ETF block clusters together; developed-market sovereign bonds form a tight cluster; crypto sits as a near-isolated branch under most regimes. Two assets that sit in the same cluster contribute one bet, not two, regardless of their pairwise correlation.

Hierarchical risk parity (HRP, López de Prado, 2016) operationalises this directly: it allocates risk recursively along the dendrogram rather than across the raw correlation matrix, and is markedly more robust to estimation noise than mean-variance. It is the production-default in our portfolio layer for that reason.

From correlation matrix to allocation

Even a perfect correlation matrix is not an allocation. The translation step matters. Three approaches dominate institutional practice:

Mean-variance optimisation remains the textbook default — and remains the most fragile. It is exquisitely sensitive to expected-return inputs and tends to concentrate on whichever asset has the highest forecast Sharpe, regardless of how noisy that forecast is. Black-Litterman shrinkage and resampling can stabilise it, but the underlying brittleness persists.

Risk parity equalises marginal contribution to portfolio variance across positions. It eliminates the dependence on return forecasts entirely and produces well-conditioned allocations, at the cost of a structural overweight to low-volatility assets that can be levered into trouble.

Hierarchical risk parity and multi-objective Pareto search are the institutional state of the art. HRP imposes geometric stability via clustering; multi-objective optimisation explicitly trades correlation, variance, and expected return on a Pareto frontier rather than collapsing them into one objective. We use a custom NSGA-II implementation for the latter — see the methodology page for the algorithmic detail.

Common failure modes

Three failures recur in correlation-driven portfolios across institutional managers, and each is recognisable before it costs money.

Sample-size starvation. A correlation matrix with N assets has N(N-1)/2 free parameters. A 100-asset universe has 4,950. Estimating that from 250 daily observations is roughly fitting a model with twenty parameters per data point — pure noise. Shrinkage toward a structured prior (constant-correlation, factor-derived, or Ledoit-Wolf) is mandatory at scale.

Static recalibration. A correlation matrix updated quarterly is a forecast, not an estimate. A weekly or daily recomputation with proper exponential weighting catches regime breaks early enough to act on them.

Confusing decorrelation with diversification. Two strategies can be near-zero correlated on average and 0.9 correlated in the only regime that matters. Run conditional correlation against your defined stress windows, not just unconditional, before sizing.

How Divitae models correlation

Across all four live strategies — Zeus, Kronos, Ares, and Hermes — correlation is a first-class object in the optimiser, not an output of the backtest.

Subsystem signals are weighted by an NSGA-II Pareto search over Smart Sharpe, variance, and pairwise correlation simultaneously. The optimiser is constrained against concentrating effective risk in a single cluster of the hierarchical map, which prevents the seven-strategy portfolio from collapsing into one bet under stress. Eight discrete risk layers downstream re-test that the live correlation structure matches what the optimiser priced in; deviations trigger automatic deleveraging before drawdowns register.

The result is a portfolio whose decorrelation survives the regimes that matter, not just the back-tested average. Decorrelation is something we measure, budget, and defend continuously — never something we hope our position count will produce.