The emergence of Artificial Intelligence in the stock market

The emergence of Artificial Intelligence in the stock market

AI is a predictive model that looks at more than technical patterns of trading; it has the ability to identify financial features of companies that will make money in the long run

Technology can be used either to make our lives better or make money. With the former, you set up a technology company such as Grab to make it easier for consumers to get a ride. The latter is normally used in trading which is commonly done in a financial hub like Singapore.

Technology is constantly evolving and for this article, we will be focusing on the latter which is to use technology to make money. The general public would have already heard of high-frequency trading (HFT) which uses algorithms to decide their buy and sell trades based on preset criteria. HFT firms like Tradeworx make their money by being faster than the average trader by taking microseconds to place a trade.

HFT trading firms have been active since 1999 when the SEC authorised electronic exchanges in 1998. Today, the markets have realised that just being fast does not necessarily mean that you will make money. You will also have to be smarter than the rest of the market.

The New Edge

If HFT firms represent the old-age of algorithm trading, firms like Aditya and Sentiment Technologies are the new edge of algorithm trading infused with Artificial Intelligence (AI). HFT algorithms are static in searching for pre-determined trading patterns that is beyond human comprehension in both scale and speed. It will work for a week, a month or a year before the rest of the market catches up with it.

Then the quants of HFT firms will have to develop new algorithms to find another set of pre-determined trading pattern. AI algorithms have the ability to learn from the changing patterns of the market. So when the old profitable trading patterns stopped making money, the AI can switch effortlessly into a new trading pattern.

In the one month or more that HFT firms used to identify new patterns and code their algorithms manually, AI firms have the advantage of trading in a relatively uncrowded market.

Artificial Intelligence – Cross Disciplinary Approach

The challenge for AI firms would be to create the versatile and adaptable algorithm in the first place. Aditya’s Chief Scientist, Dr. Benjamin Goertzel, is a mathematics professor who combined his expertise in machine learning, computational linguistics, natural language processing, computational finance, bioinformatics, complex systems and cognitive science with a team of talented scientists to create the AI algorithm.

Sentiment’s Chief Scientist Babak Hodjat was behind Apple’s Siri platform and a prominent developer. Sentiment used models based on biological evolution and deep learning to create their proprietary software named ‘Evolutionary Intelligence’.

AI is a predictive model that looks at more than technical patterns of trading. It has the ability to identify financial features of companies (e.g. price to earnings ratio, long term business loans) that will make money in the long run. This requires capabilities from different areas of study and massive computational power which is why it is only prevalent in recent years.

Aditya was started in 2011 in Hong Kong and started its first fund after three years’s of effort. Sentiment started their AI project in 2007 and only emerged in late 2014 with their product, Evolutionary Intelligence.


HFT is designed for short-term trades which lead to the Flash Crash of 2010. This is also a system which incurs a high transaction cost to make money. AI trading allows for long term trades that would make money over the next year.

For instance, AI can search through countless analysis of the yen and predict if Toyota will make or lose money before they publish their results three months later. The market would gain more stability and this appears to be the new way forward.

Will AI take over the stock market? Robotic stockbrokers are starting to predict  changes in share prices better than humans

Will AI take over the stock market? Robotic stockbrokers are starting to predict changes in share prices better than humans

  • Japanese brokers are testing artificial intelligence to predict the market
  • Their algorithms has got its predictions correct 68 per cent of the time
  • Most human analysts can only get 48 per cent of their forecasts right
  • It has raised the prospect of AI computers taking over share dealing

The stock market is notoriously unpredictable – for those who can make the right calls, vast sums of money can be made, but get it wrong and the losses can be equally huge. Now some stockbrokers are hoping that artificial intelligence will tip the balance in their favour. Japanese researchers have developed a computer capable of predicting the rise and fall of the market correctly 68 per cent of the time.

It suggests learning algorithms, which assess huge volumes of data to predict stock prices and economic fortunes, could transform the markets.

While computer algorithms have been used in share trading for decades now, allowing transactions to be timed to a fraction of a second, the use of AI has been slow. Dr Junsuke Senoguchi, a senior equity strategist at Mitsubishi UFJ Morgan Stanley Securities in Tokyo, however, has being testing a new robotic stockbroker. Every month, for four years it has produced a single simple prediction – whether the Nikkei 225 Stock Average index will be higher or lower in 30 days time. Dr Senoguchi told Bloomberg Business that the robot has been getting more and more accurate. He said: ‘Artificial intelligence gives you much better results than conventional statistics. ‘Sometimes the structure of the market changes greatly.
‘The ability to change the model when this happens is a big difference from previous approaches.’

According to Dr Senoguchi, since March 2012, his robotic stockbroker has been right about the changes in the market 32 times out of 47 – giving it a hit rate of 68 per cent.
Although this is still a relatively small number of tests, it exceeds the figure that would be expected by chance. Indeed, research compiled from market forecasters since 1998 by CXO Advisory Group revealed humans were only able to accurate predict the market direction 48 per cent of the time.

With a prediction hit rate of 68 per cent, the machine is already outperforming human stockbrokers, although the number of forecasts the computer has made are still small. Some experts believe artificial intelligence could transform how trades are made. Stock image of share prices on a computer pictured

Dr Senoguchi’s algorithm combines 92 economic indicators over multiple timeframes and uses sets of rules to eliminate those that it sees as being least predictive over the past 48 months.
It then uses this data to produce model to predict what will happen in the coming month. Each month it starts afresh and takes into account any changes.

However, the AI stockbroker is still far from being perfect. According to Bloomberg it utterly failed to predict the huge fall in share prices around the world that marred the start of 2016.
It said the period ending January 10 would see the Nikkei 225 rise. Instead it fell to one of its lowest levels on record.
Regardless, artificial intelligence experts have described the approach being taken by Dr Senoguchi as ‘extremely new’ and said the computer’s level of success was ‘pretty high’.

The AI stockbroker failed to forecast plunging markets at the start of 2016, sparked by massive falls in share prices on China’s stockmarket (pictured) which reverberated around the world

How well does a “robot AI” predict the Japanese stock market?

How well does a “robot AI” predict the Japanese stock market?

A recent Bloomberg article reported on the work of Junsuke Senoguchi, who has developed a“robot” artificial intelligence-powered computer program that forecasts the Japanese stock market, in particular the Nikkei-225 index.
Senoguchi, who currently works for Mitsubishi UFJ Morgan Stanley Securities in Tokyo and who has previously worked for Lehman Brothers and also the Bank of Japan, has a Ph.D. in artificial intelligence (AI), and his new investment program employs AI techniques. Senoguchi is delighted when it is working well, “because I feel I can predict the future“.

While we certainly wish Mr. Senoguchi well in his efforts, we need to check his results to date. Bloomberg reports that in the four year period from March 10, 2012 through January 10, 2016, his program has predicted the next month’s movement of the Nikkei-225 index, up or down, 32 times out of 47, or 68% of the time. How does this compare with a “random” strategy?

The Halloween indicator and the January barometer

In previous columns, we discussed the Halloween indicator and the January barometer.
The January barometer is the claim that whether the S&P500 index goes up or down for the year is well predicted by whether it goes up or down in January. This has been correct 62 of the last 85 years, or 73% of the time. This may sound significantly better than a coin-flip, but we must keep in mind that the proper comparison is with a weighted coin-flip, since the S&P500 has gone up 63 out of these 85 years. If one compares the January barometer with a coin-flip weighted to turn up heads 74.12% of the time, then it is almost as good as the January barometer.

Analysis of Senoguchi’s results

So let us take a closer look at Senoguchi’s results. First of all, note that in 47 trials, a 50-50 coin flip will be correct on average 23.5 times. The standard deviation here is 3.4, so Senoguchi’s 32 successes out of 47 is a roughly two-sigma result, which corresponds to only a 5% chance that it is due merely to random chance.

But, as we noted above, it is really not appropriate to compare with a 50-50 coin flip, since in this case the Nikkei-225 index has increased 27 out of the past 47 months since March 2012. If we then consider a weighted coin with probability of heads = 27/47 = 0.574468…, then we obtain the result that a coin-toss experiment with 47 trials will be correct slightly more often —24 times.

But even this experiment, one might argue, is not quite appropriate, since beginning in March 2012 one would not know the performance of the Nikkei-225 over the next four years.

So let us consider the following additional experiment: each month (on the tenth day), beginning in March 2012, look back over the previous four years and tally the fraction of times the Nikkei-225 has gone up in those 48 months, then toss a coin with this weighted probability as the guess of the market’s behavior in the next month. We have done this with a computer program, based on historical market data. On average, our program is correct approximately 24 times on average out of 47, the same as above.

One way or the other, Senoguchi’s result is a roughly two-sigma result. While promising, in our view it is not yet time to break out the finest champagne.


As we noted in our blog on the January barometer, insomuch as models of this type rely on market indices such as the S&P500, the Nikkei-225, the FTSE100 or the DAX-30, particularly if only taken at an interval of months or years, it is very difficult to avoid backtest overfitting — there simply is not nearly enough data to make any statistically meaningful statements.

Data over a longer time horizon — 25, 50 or more years into the past — is certainly available, but it is of highly questionable relevance today. This is especially true given the rise of computerized trading in recent years, which means that any trends or patterns that may exist in data are spotted (and neutralized) by sophisticated computer programs long before human analysts see them.

In short, while we wish Mr. Senoguchi well, it is premature to say whether his system has any fundamental, persistent “skill,” particularly in today’s highly mathematical, computerized market. There are no slick tricks to wealth, and no substitute for patient, long-term, low-cost investing.

Artificial Intelligence will be crashing the Stock Market in 3,2,1…

Artificial Intelligence will be crashing the Stock Market in 3,2,1…

A few weeks ago, Stephen Hawking opened the world’s eyes to the dangers of Artificial Intelligence (AI), warning that it has the potential of outsmarting humans in the financial markets. But few people realize that we are already in imminent danger of this happening.

The stock market is a system for assigning value to companies through the buying and selling of stock. It’s a human-based system, assigning human value, to corporations owned and operated by humans. Well, at least that is how it was supposed to work until the machines started taking over.

In the 1960’s, an average share of stock was held 4 years. By 2000, average ownership dropped to 8 months, and in 2008 it dropped even further to 2 months.

Today the average share is held a scant 20 seconds and within a few months, it will drop to less than 10 seconds.
At the center of this rapid buying and selling of stock are a series of high-frequency trading machines run by the quants, the math-whiz kids who are a type of hackers only on Wall Street.

Without having people at the center of these trades, we have lost the core ingredient, our ability to accurately assess value.

The invasion of high-frequency trading machines is now forcing capitalism far away from anything either Adam Smith or the founders of the NYSE could possibly find virtuous.

We’re not about to let robots compete in the Olympics, driverless cars race in the Indianapolis 500, or automated machines play sports like football, basketball, or baseball. So why is it we allow them to play a role in the most valuable contest of all, the world wide stock exchange?

With crude forms of AI now entering the quant manipulator’s toolbox, we are now teetering dangerously close to a total collapse of the stock market, one that will leave many corporations and individuals financially destitute.

Here is why this should be ringing alarm bells all over the world.

The Flash Crash of 2010

Few things explain the dangers of AI manipulations of the stock market quite like the Flash Crash of 2010. Here’s a quick overview:

The date was May 6, 2010 and by all outward appearances, everything on the markets appeared to be normal. Yes, people in Greece were protesting austerity but there were no other indicators of what was about to happen.

Cheap credit had been pushing stocks higher for months, but the mood was changing. Every time disgruntled workers in Athens hits the TV screen, the Dow Jones would drop a bit more. By 2:30 p.m., it was down 2.5%, a moderately bad day, but that’s when everything was about to go crazy.

It began as a ripple in the price of E-mini futures contracts, traded on the Chicago Mercantile Exchange, but almost nobody noticed. This tiny ripple quickly morphed in a major ripple, and with very little forewarning, the tail quickly started wagging the dog to pieces.

Within seconds, the Dow has lost 100 points, then 200, 300, 400, and 500.

It’s important to know that the internal self-limiting mechanisms designed to halt trading after unnatural price swings only works until 2:30 p.m. EST.

At 600 points down, the Dow had fallen further than it did on news of Lehman Brothers’ collapse in 2008. But that crash took a full day, this was killing the market in a matter of seconds.

When the market goes into a freefall, traders start to panic, vomiting into trashcans and mentally preparing themselves to leap off tall buildings. Even the 9/11 disaster hadn’t rocked the market like this.

Those close to the action started demanding that someone “break the glass and hit the emergency stop button on the wall,” but this button doesn’t exist.

At 2:47 pm, with the Dow racing towards an unprecedented 1,000-point loss and almost $1 trillion being wiped from the balance sheets, an even bigger surprise happened. The market switched directions and began to rise.

The 600-point loss suddenly became 400, then 300, and 200.

The craziest part of all was that this entire episode, the most dramatic in stock market history, had occurred in just 10 minutes.

After a short period, rumors began pointing fingers at the supposed culprit – Waddell & Reed, an asset management firm based in Kansas. But this was later disproven.

In the background were the quant-hackers managing the strings for some other puppet-master. These people are also experts in blame-shifting and deception so we will probably never know what actually happened.

(Here’s a more detailed account of the Flash Crash of 2010.)

Killing the Goose that Laid the Golden Eggs

Even the most egregious abusers of the stock market have no interest in killing the system that has become their central playground, the goose that continually lays the golden eggs. But they have no problem with extracting wealth from it at every turn, often harming individual companies quite severely in the process.

The tactics used in the flash crash of 2010 are similar to a denial of service (DoS) attack. On this day servers became so overloaded that quotes for some shares experienced as much as a 36 second delay.

This is a black-hat technique known as “quote stuffing” where machines begin placing and canceling unrealistic orders 10,000 times in a second, or stepping from one share all the way to 100, one at a time, and then marching back down again in milliseconds, over and over and over.

Keep in mind this is a very crude form of artificial intelligence. Imagine the type of exploitations that will be possible when higher forms of AI begin entering the system.

Manipulations like this are only a problem if we continue to let the puppet-masters guard the marionette stage. Much of this problem quickly goes away if we return to a system where human “authority” is used to curb abusive practices.

The Need for Human Authority

Many of our free-market thinkers have long advocated automating human authority out of the equation.

Trades can happen faster and in greater volume if we remove the gatekeepers from the system. However, we still need humans to oversee the system. It may be automated human, but still humans.

As an example, Google’s largest computer data centers are built around thousands and thousands of flawed machines that individually fail time and again. With systems for circumventing failures when they occur, the overall machine, in its entirety is more than a little impressive.

People are very similar. We are all flawed individuals, mired in an ocean of personal chaos. But the same imperfections we see on the micro scale change dramatically when we transcend to look at humanity on a macro scale.

In much the same way that Google operates a massively complex machine by changing out individual units on the fly, we will eventually be able to create superhuman intelligence by connecting our own individually flawed brains with a massively coupled super brain.

No, this would not be a 24/7 link that limits our individuality or free will, but the human equivalent of a moral machine that we can choose to be part of. I’ll save the details of this for a later discussion.

Final Thoughts

We are currently walking on dangerously thin ice. Artificial intelligence is already creeping into our lives on a daily basis, but even with its current Neanderthal-level intellect, it can do incalculable levels of damage.

I would compare the current stock market to The Borg on Star Trek, but it’s even more sinister than that. The deeper we probe, the more we appear to be pawns on someone else’s financial chessboard.