The Economist: How machine learning is revolutionising market intelligence

21 November 2019

Intelligence-gathering is a labour-intensive business. It is thus ripe for automation. That this is happening in finance is also natural. There is a well-defined objective (to make money). There is a well-defined end-point (buy, sell or avoid). Without such clarity of purpose, intelligence is an endless river. It is one undammed thing after another.

For many people, the use of such technologies in finance is the stuff of dystopian science fiction, of machines running amok. But once you look at market intelligence through the eyes of computer science, it provokes disquieting thoughts of a different kind. It gives a sense of just how creaky and haphazard the old-school, analogue business of intelligence-gathering has been.

Analysts have used text data to try to predict changes in asset prices for a century or more. In 1933 Alfred Cowles, an economist whose grandfather had founded the Chicago Tribune, published a pioneering paper in this vein. Cowles sorted stockmarket commentary by William Peter Hamilton, a long-ruling editor of the Wall Street Journal, into three buckets (bullish, bearish or doubtful) and attached an action to each (buy, sell or avoid). He concluded that investors would have done better simply to buy and hold the leading stocks in the Dow Jones index than to follow Hamilton’s steer.

The application of machine-learning models to text-as-data might seem a world away from Cowles’s approach. But in concept, it is similar. The relevant text is sought. Values are ascribed to it. A statistical model is applied. Its predictions are tested for robustness. Of course, with bags of computing power and suites of self-learning models, the enterprise is on a different scale from Cowles’s rudimentary exercise. The endless expanse of the internet means far richer source material. The range of possible values ascribed to it will be broader than “bullish, bearish or doubtful”. And self-learning algorithms can test and retest the combinations that yield the best predictions.

Full article on The Economist (subscription required)


© The Economist