A Brief History of Our Application of DoE to Automated Trading Systems

Although we have been believers in Design of Experiments for nearly twenty years and builders of automated trading systems for nearly ten, we never put the two together until recently. A designed experiment simply our trading system would have taken years as recently as three years ago. As a result, we did like everybody else does. That is, we set our system parameters using intuition and experience, and tweaked those each time we reviewed the findings from any test run. We made significant progress in that way, but it was plain to see that we could learn more faster.

We started applying Design of Experiments to automated trading systems about a year ago, but the tests were taking a long time. We were working with intraday data, and that is really dense. It took many months to complete a study that used just two years of market data. Even at that cost, we were able to run a couple of studies and became convinced that the approach was valid. Unfortunately, the systems were proprietary and secret so we were having a little trouble talking about our approach in any detail without betraying a confidence.

A couple of months ago, Ernest Chan suggested that we test something from the book Short Term Trading Strategies That Work. We needed to talk about something other than our own systems. There were two major surprises.

The first was the speed of the tests. Our tests for the options trading systems that we built took six hours for each trial for each expiry (month). An eighteen month study takes about three months running ten of these tests at a time in parallel. The strategies in the book used only end-of-day data. As a result, a test on a twenty-year set of data takes one quarter of a second! We had the study completed, and article on it accepted by a publisher in less than a week.

This is mind-boggling because in most experimentation the cost of the trials is the overwhelming majority of the cost of the experiment. The challenge of most of Design of Experiments is to get as much information out with as few trials as possible. One of the problems with our business model when we built Strategy was that there very few people who needed enough experiment designs to keep us in business because they would get a design, take months or even years to run the experiment, then need our help or software for about two hours to interpret the results. When the trials are free and run this fast, the design of the experiment and the analysis become significantly more difficult and time-consuming. That’s fine with us because we have some unique advantages there.

The response to our efforts was the second surprise. We had never heard of this book, but it turns out to be very popular in the trading community. Many people have read, and those that haven’t, had at least heard of it. Many people had a common basis for understanding our work, and we are starting to have real conversations about applying this technology. We are moving on to models that we couldn’t even consider when the cost of trials was an issue.


2 comments on “A Brief History of Our Application of DoE to Automated Trading Systems

  1. Jakejake says:

    This is really interesting, how has it progressed?

    • abcorwin says:

      Sorry, I didn’t notice your comment.

      The results are mixed. We have verified some things that already shape the way we bet, but there is little that one can count on in a changing world. Learning about the past is illuminating, but the main thing it teaches one is to be careful.

      We have had a little luck lately testing folk sayings like “Cut your losers and run with your winners.” This particular saying has proven very useful, but when to cut losers and how long to let winners run are subjects of continued study.

      It’s fun, but it’s like any gambling operation. The bookies make all of the money. You have to be very good and very lucky to win more than the house takes out of your hide. ETFs have almost no vig so you only need to win a little bit to pay expenses.

      By the way, if you are @SixSlayer, I enjoy your stream on Twitter.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s