Why did flawed trade software not trigger a shutdown?


Faulty deals persisted for over 30 minutes and lost a fortune, write JESSICA SILVER-GREENBERG, NATHANIEL POPPERand MICHAEL J DE LA MERCED

WHEN COMPUTERISED stock trading runs amok, as it did this last week on Wall Street, the firm responsible typically can jump in and hit a kill switch.

But as a torrent of faulty trades spewed on Wednesday morning from a Knight Capital Group trading program, no one at the firm managed to stop it for more than a half-hour.

Some Knight employees and New York Stock Exchange officials noticed the blizzard of erratic orders in the minutes after trading started and sent alarmed messages to Knight managers, according to the exchange and Knight employees who declined to be identified discussing the matter.

As Knight struggled to survive last Friday, employees at the company, market overseers and other electronic trading firms were asking the same basic question: where was the “off” switch?

Several market insiders said they were bewildered, because in a market where trading losses can pile up in seconds, executives typically have a simple command that can immediately halt trading.

“Even just a minute or two would have been surprising to me. On these time scales, that is an eternity,” said David Lauer, a trader at a high-speed firm until a year ago. “To have something going on for 30 minutes is shocking.”

Regulators are planning to look into why there was such a lag between the discovery of the problem and when Knight’s trading ceased, according to people with knowledge of the discussions. But so far the company has not provided any answers, even to its own staff, employees said.

On Friday, Knight, which in the past decade grew into a leading broker for US stocks, climbed off the mat, securing emergency financing that allowed it to continue operating for the day. It also enticed some of its customers to resume sending client stock trades, two days after Knight disclosed a possibly fatal $440 million loss from the software problem. But it faced a desperate weekend of manoeuvring to find a more permanent solution for its woes. Knight’s short-term financing was meant to keep it alive until today, when its executives and advisers hope to have deals completed to remove any doubt about the firm’s future.

Advisers, including Sandler O’Neill Partners, have been talking with Knight rivals and private equity shops about either buying divisions of the firm or investing in the business.

Among the businesses that Knight is in discussions about selling is its futures brokerage unit, largely made up of operations the firm purchased only in May, said industry insiders. Potential buyers for the business include RJ O’Brien, based in Chicago and one of the oldest futures clearing firms.

Others that have expressed interest in potential investments or deals include Knight rivals such as the Citadel Investment Group, Virtu Financial and Peak6 Investments, as well as private equity firms like Kohlberg Kravis Roberts and TPG Capital, these people said. Knight is also working with Goldman Sachs to help unwind the trades behind its extensive loss, according to people briefed on the matter.

Goldman has agreed to buy, at a discount, the shares that the trading firm had accumulated. Such a move would help Knight by taking the portfolio off its hands and freeing up capital.

Two major brokerage firms, TD Ameritrade and Scottrade, said on Friday they had begun sending client orders to Knight. Others, like E(AST) Trade Financial, had yet to resume doing so.

“Knight is one of many order routing destinations for us and has long been a good and trusted partner,” said Fred Tomczyk, TD Ameritrade’s chief executive.

Toward the end of the trading day on Friday, employees in the Jersey City, NJ, offices gathered around television screens and cheered every bit of good news. Shares in Knight leapt 57 per cent on Friday, closing at $4.05. But they remain down more than 60 per cent for the week.

Coming after a number of previous market mishaps caused by faulty computerised trading, Knight’s trading problems rekindled a broader discussion about the vulnerability of an increasingly complex and fragmented stock market.

Chairman of the Securities and Exchange Commission Mary L Schapiro called the Wednesday episode “unacceptable” and said that her staff would “convene a round table in the coming weeks to discuss further steps that can be taken to address these critical issues”.

Duncan Niederauer, the chief executive of the New York Stock Exchange, said in a conference call with investors that the incident was a “call to action” and that the exchange was prepared to lead the way on reforms.

“We are all understanding – meaning we, market participants, and most importantly the regulators – are understanding that speed is not always better,” Mr Niederauer said.

Within the financial community, much of the attention was still focused on what happened on Wednesday morning. While the New York Stock Exchange has said that there was “irregular trading” in only about 140 stocks listed on its exchange that day, Knight’s trading in those stocks was so extreme that it was visible in the volume of trading in all stocks.

A New York Times analysis of New York Stock Exchange volume on Wednesday morning showed that, during the first minute of trading, there was 12 per cent more trading in all stocks than there had been on average during the previous seven days. By the third minute of trading, there was 116 per cent more trading than the previous week’s average. The difference reached a peak at 9.58am, when the volume was six times greater. After that, trading volume fell off sharply, nearing the recent average at 10.15am.

Mr Niederauer said the exchange had noticed the problem and contacted Knight “within minutes” of the 9.30am opening bell.

Knight’s failure to respond sooner was particularly mystifying to other traders because on Wednesday the firm had introduced new trading software. Industry experts said that this would normally be the cause for programmers and other employees to be on high alert.

Once the problems began, many traders said it would have made sense if the firm’s employees had not caught the problems for the first minute or so, given the speed at which Knight’s program was firing off orders. After that, though, the problems were visible for all to see.

Howard Tai, an expert in high-speed trading at the Aite Group, said that at all the firms where he worked there were several warning signals built into every computerised trading system. When all else failed, there was always the “automatic kill switch” that could immediately stop trading.

Mr Lauer said, “It’s kind of mind-boggling that it got so out of control.”

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
Error Image
The account details entered are not currently associated with an Irish Times subscription. Please subscribe to sign in to comment.
Comment Sign In

Forgot password?
The Irish Times Logo
Thank you
You should receive instructions for resetting your password. When you have reset your password, you can Sign In.
The Irish Times Logo
Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.
Screen Name Selection


Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
Forgot Password
Please enter your email address so we can send you a link to reset your password.

Sign In

Your Comments
We reserve the right to remove any content at any time from this Community, including without limitation if it violates the Community Standards. We ask that you report content that you in good faith believe violates the above rules by clicking the Flag link next to the offending comment or by filling out this form. New comments are only accepted for 3 days from the date of publication.