For Finance to serve society, it needs to evolve beyond wizardry, guru’s guesswork and flawed toy models. Just as modern physics cannot advance without facilities like the Large Hadron Collider (LHC), a useful Finance will require machinery up to the task.
IN-SAMPLE DREAMS, OUT-OF-SAMPLE NIGHTMARES
In a recent paper, I have argued that some of the most popular optimization techniques used in Finance are in fact detrimental. Take mean-variance optimization (MVO), the most commonly used portfolio construction technique, with its multiple upgrades and variations throughout the past 60 years. A great majority of academic papers apply MVO when the authors are faced with the dilemma of building a diversified portfolio. One would expect that such venerable technique would be among the best performing portfolio construction methods, right? Think again.
A number of studies have demonstrated that MVO portfolios underperform the so called “naïve portfolio”, that is the portfolio that splits assets equally among holdings (see here for example). And yet MVO is taught in every business school as one of the key results in Finance. Shouldn’t students be warned that MVO is detrimental, relative to a naïve allocation? How can a Nobel prize-winning theory lose to the most rudimentary scheme?
This is not a unique example. There are plenty of revered financial techniques that fail to perform as advertised. Cointegration models are known to lack robustness, in the sense that small changes on a few observations will lead to entirely different forecasts. This is particularly problematic in a discipline like Finance, where the signal-to-noise ratio is low and measurements are far from precise. Still, unstable econometric methods are routinely used by economists to forecast macro variables and by the Federal Reserve to inform their life-changing decisions.
In fairness, these methods were designed and vetted for academic consumption only. They are toy models, to be used for in-sample philosophical disquisitions, not in out-of-sample industrial applications (see here for the difference). Their authors never managed investors’ money, risked their own assets, or foresaw their widespread use outside academia. Real-world applications require a degree of complexity and robustness that simple models cannot satisfy.
OLD MATH AND TFLOPS-PAUCITY
Financial markets are complex networks of decision-making agents, interacting with each other. MVO is a very simple model, where every asset is a candidate to be replaced by another. A more sophisticated method would recognize the fact that some assets share features, such as their country of origin, sector, industry, default probability, liquidity, etc. In other words, like most applications of Geometry and Linear Algebra, MVO ignores hierarchical relationships. And because the method lacks structure, it lacks robustness. Conversely, graph theory embeds hierarchical information with minimal assumptions. But their development requires modern mathematical techniques absent from most economics/business curricula, hence graph-theory applications to Econometrics remain rare.
In many instances, simplicity is not a choice. Many researchers opt for dumb-down models because they lack the computational power required by more realistic ones. Should researchers have had access to more powerful computing machines, they would have proposed more realistic methods and theories.
REFORMULATING PROBLEMS IN REALISTIC TERMS
Portfolio construction techniques allocate assets in such a way that risk is minimized for a given return target. MVO can solve this problem as long as some unrealistic assumptions are met: First, returns must be IID Normal. Second, the covariance matrix must be invertible. But inverting a NxN covariance matrix with a minimal degree of success requires N(N+1)/2 independent observations. For a small portfolio, of say size 50 instruments, that would involve at least 1,275 independent daily observations, or over 5 years of data. As every practitioner knows, correlations do not remain stable over such horizon with any meaningful degree of confidence. The implication is, MVO gives the precise answer to the wrong question.
Alternatively, we could study the hierarchical relations between the 50 instruments and build a tree-graph with 49 edges. This machine learning approach, called HRP, does not require covariance inversion, or a large number of independent observations. Results are robust to noisy observations and HRP delivers a 31% increase in out-of-sample Sharpe ratio relative to MVO. HRP is just one example of how modern mathematical approaches can help reformulate a problem in realistic terms. Interestingly, HRP does not require more data than MVO, so the performance improvement is all due to processing the available data better.
HOW QUANTUM COMPUTING MAY SAVE FINANCE
There are numerous instances in which machine learning methods deliver better results than classical calculus or linear algebra applications. But machine learning often deals with NP-complete or NP-hard problems, which demand overwhelming computational power. One possibility is to use proper heuristics (as HRP’s clustering algorithm does) to make the problem amenable to digital computers (DC). Carefully applying heuristics to solve a NP-complete problem is better than dumbing-down the problem to the point that a closed-form solution exists. A better way is to replace heuristics altogether, and explore a large portion of the feasibility region.
This is the promise of financial quantum computing (QC). One day, in the near future, we will not need to dumb-down models, or rely on heuristics. We will develop models cognizant of reality’s complexity, and solve them in their NP-complete grandeur. Think about it: If HRP can improve your out-of-sample Sharpe ratio by 31% over MVO’s, what will the improvement be once you replace the DC+Heuristics tandem with QC+Completeness? Perhaps 50%? That means boosting your Sharpe ratio from 1.50 to 2.25, quite worth the management fee. Besides, QC can be used to save transaction costs and reduce the prevalence of high-frequency trading.
It is possible that one day Finance may offer real value to investors, beyond sales pitches, TV-show pundits and gurus’ guesswork. When cutting-edge technologies will be combined with realistic mathematical models to deliver strong, measurable benefits to savers and retirees. When financial firms will operate as research laboratories, and the alchemy of finance will be replaced with the chemistry of asset allocation. It will take a lot of qubits… but society will be better served.