Internal and External Risk Management

Internal risk management is about disaster aversion. External risk management is about the probability of loss.

External risk management is strictly about the relative size of your bet. Strictly, it is about the degree of affordability of the worst possible loss. A prudent external risk management plan ensures that each individual bet and each collection of correlated bets risks less than half of your available bankroll. The external risk management process should be the same for all bets. The primary question you should always be asking yourself is how much you can lose and how much you can afford to lose.

External risk management is far more important than internal risk management. Good external risk management keeps you in the game when things go wrong. Never forget that eventually things will go wrong.

Internal risk management is disaster identification and mitigation. It’s all about what can go wrong, and what can be done to prevent it. Internal risk management starts with a list a list of the possible disasters. A prevention and mitigation strategy is then developed for each risk. The process is simple and straight-forward. Just going through the process gets many of the risks under control, and it is essential to getting the risks solidly under control.

The quality of internal risk management is a factor in the execution of external risk management. The more the internal risks are kept under control, the less risky the overall bet is. Always consider the internal risk management when evaluating the probability of loss.

Research Time and High Frequency Trading

The primary cost of doing most trading system research is the cost of the time. Time is used to plan an experiment, to run an experiment, and to analyze an experiment. When you need answers in two days, you simply cannot use an experiment that takes six weeks to run.

In an ordinary research environment, the time to design an experiment uses a tiny fraction of the time. Analysis takes slightly longer. Time to set up and execute the experiment takes far more than ninety percent of the total time in the process. When a problem requires an answer in significantly shorter amount of time, the only hope is to reduce the time it takes to execute the tests. If you need an answer within two days, you need to reduce the execution time for each trial to about a minute or less.

You can get incremental improvements in the speed of trading system tests by getting faster and faster computers, but the real secret of speeding up trials is to consider less data. Each piece of data requires processing, even if the process is simply to ignore that piece of information. Do you need or even want ten years of data if you are testing a system that will trade a thousand times a day? Do you need six months of test data? Can you find what you are looking for in the last week’s data?

A week is the shortest time frame that really makes sense in the context of the problems we have looked at, but your problem might allow even shorter time frames. If you are talking about tick data, even a common PC can process a week of tick data is less than a minute. While you might be able to find your answers with even less data than this, it is hard to see why you might want to.

Once you get the total execution time of an experiment down in the neighborhood of a single day, the dynamics change. If you are taking a couple of days to design the experiment and a week to analyze it, that becomes the place to save time. An experiment that takes six months to run demands two days of design and at least a week of analysis.

Any environment like high frequency trading that have severe enough time demands to drive the setup and execution time for experiments down to a scale of hours will also demand rapid design and analysis.

More rapid design is fairly easy to achieve. The painful and tedious cost of most experiment design is deciding on the compromises. You need to compromise because of the cost of executing each trial of an experiment. For a trading system, the cost of executing these trials is strictly the cost of the time. Once that problem is solved, additional trials do not impose additional costs. Actually generating a design once you have decided on the goals of the experiment only takes a few minutes with today’s tools.

Some research organizations devote weeks or months to analysis. This is entirely justified when an experiment takes six months to run because you are obligated to wring every possible bit of information from each experiment. Experiments using trials that have little or no costs to them generate far more data than experiment

However, everyone who has ever analyzed an experiment will tell you that some results are obvious at a glance and other results are only obvious after a particular perspective has been imposed on the data. Finding an answer to a particular question is usually not that time consuming. Primary analysis does not have to take days or weeks. Even the mountain of data generated by a high speed experiment can be parsed rapidly if that is done with a narrow focus.

Secondary analysis still can and should be done. Unexpected results are often only found in the secondary analysis, and discovery is all about unexpected results.

What is My Opponent Holding?

Having spent far more hours playing poker than sanity allows, I like to think that I have more than money to show for it. I hope that I have come away with lessons that apply to business, investment, and life in general. That may be a stretch, but I cling to that illusion.

I am constantly trying to figure out why the best players are that good. It’s easy enough to see why the bad players are so bad. They bluff too often, call too often, don’t raise to protect their hands, don’t control their emotions, and have no understanding of position. You can become a mediocre to good player simply by addressing these problems, but great players have something else. As a matter of fact, even the great players still have some of these problems on an occasional basis.

Something else makes them great. I don’t think it is that the best players have more heart, more brains, or more luck, though everything does go easier when you have more advantages on your side. I think what separates the great players is their awareness that poker is a relative game. The great players spend most of their attention focusing on what their opponent has. Bad players often don’t think at all about the opponent’s cards, good players think about them at the time of decision, and great players think about them even when they are not in the hand. Every time you show a hand down, you are giving a great player another weapon to beat you with.

The great player knows that there is no greater advantage in poker than knowing your opponent’s cards. We could all be poker millionaires if we knew what our opponents have and they don’t know what we have. One of the best examples of this was a couple of years ago in the main event at the World Series of Poker. Sammy Farhar called a massive pre-flop raise early on the first day with only a pair of deuces. He knew that the nervous amateur making the bet would not have made that bet with anything other than a pair of aces. When a deuce came on the flop, Sammy got the rest of the amateur’s chips. I’m don’t think that the implied odds that he got really justified Sammy’s call, but know just exactly where he stood was a powerful incentive that he could not resist. He made it pay off.

Daniel Negraneau is always talking about what his opponents have when he is heads-up in a hand. I am sure that he only refrains from discussing what other people have in their hands in other situations because that would be outside the etiquette of the game, but I am sure he is thinking about it all of the time. I find Daniel the most instructive of the leading pros to watch for just this reason. Listen to the questions that he asks himself.

Why did he bet that much? Why did he check? What did that card do for him? How likely is it that my opponent is bluffing? Is this a guy who only bets the nuts? How nervous is this person about this bet? How many chips does this opponent have left? What is this opponent likely to do if I raise?

Daniel doesn’t always come up with the right answers and he’s quick to notice when he is wrong, but he is always asking and answering the questions. That gives him many chances to be right than he would have if he never thought of those questions.

It’s just like that in business and investing. Asking all of the right questions doesn’t guarantee that you will win, but it does increase your chances dramatically. There are never any guarantees, but there are lots of edges if you look for them.

Just spend a few minutes thinking about the last few years at some companies disappeared.  They disappeared not because they came up with the wrong answers; many disappeared because they simply didn’t ask the right questions.

Your Linked-In Connected Blog

One of the things that you can do on your Linked-In profile is link to a blog. I started this blog on WordPress.com because Linked-In offered it and said that it would show up on my profile page.

It would be really nice if Linked-In Updates would report when I post another entry, but they don’t. You actually have to go to my profile page to see that I even have a blog, and that doesn’t show to people outside your network unless you have an upgraded (not free) account. Upgrade to the business account, and people will see your blog on your profile if you so desire.

Always remember that even if no one reads your blog for months at a time, somebody will when you go to look for a job or attempt to sign up a new client.

That means that you don’t write about politics unless you want to make about half of the people out there want to avoid you for your political leanings, you don’t want to talk about religion unless you are in the religion business, and you don’t want to talk about your partying life unless you are selling your partying life. You wouldn’t talk about your bad habits in an interview. A blog will get to your interview before you do so keep that in mind.

In fact, one of the nice things about having a blog is that it is like an interview where you are allowed to ask the questions and give the answers, but someone who did not participate in that process gets to judge it. Make sure that you ask questions that are interesting to your perspective interviewers or customers, and make sure that you provide thoughtful answers. They can’t always be the right answers, but they can always be thoughtful. And because it is in writing, you don’t have to show your thought until it is complete.

If you can write, you should. Everybody needs people who can write, and the more technical your field is, the more your ability to write will be appreciated. However, you do need to remember that your Linked-In persona is the one you are taking to work. Your Linked related blog is not where you want to rant and rave.

You should only write about things that relate to the way you earn your living or want to earn your living. That gives lots of us a lot of latitude as long as we are creative. For example, I am a process professional who builds software, directs research, and markets that research. I’ve been an entrepreneur for forty years. I can write about anything from at least one of those perspectives, but I do have to couch it in that perspective.

For example, I love to play golf, but I’m not in the golf business. That means I can talk about the marketing of Tiger Woods (a topic I find fascinating), but my Linked-In related blog is not a place to discuss potential cures for my hook. If I was in the golf business, I’d have a little more latitude. If I was a golf architect, I could do course reviews or blogs on hole design. If I was a club maker, I could write about swing dynamics. The point is that you should only write about things that have a chance to help your business.

So how does this and other articles on Linked-In fit in for me? Linked-In is a tool that I am trying to exploit to its fullest extent. Part of my role and part of my interest is always on marketing, and Linked-In is a marketing tool. You have to understand marketing to some extent no matter what your job is because unless someone is buying, you may not be getting paid for long.

A Brief History of Our Application of DoE to Automated Trading Systems

Although we have been believers in Design of Experiments for nearly twenty years and builders of automated trading systems for nearly ten, we never put the two together until recently. A designed experiment simply our trading system would have taken years as recently as three years ago. As a result, we did like everybody else does. That is, we set our system parameters using intuition and experience, and tweaked those each time we reviewed the findings from any test run. We made significant progress in that way, but it was plain to see that we could learn more faster.

We started applying Design of Experiments to automated trading systems about a year ago, but the tests were taking a long time. We were working with intraday data, and that is really dense. It took many months to complete a study that used just two years of market data. Even at that cost, we were able to run a couple of studies and became convinced that the approach was valid. Unfortunately, the systems were proprietary and secret so we were having a little trouble talking about our approach in any detail without betraying a confidence.

A couple of months ago, Ernest Chan suggested that we test something from the book Short Term Trading Strategies That Work. We needed to talk about something other than our own systems. There were two major surprises.

The first was the speed of the tests. Our tests for the options trading systems that we built took six hours for each trial for each expiry (month). An eighteen month study takes about three months running ten of these tests at a time in parallel. The strategies in the book used only end-of-day data. As a result, a test on a twenty-year set of data takes one quarter of a second! We had the study completed, and article on it accepted by a publisher in less than a week.

This is mind-boggling because in most experimentation the cost of the trials is the overwhelming majority of the cost of the experiment. The challenge of most of Design of Experiments is to get as much information out with as few trials as possible. One of the problems with our business model when we built Strategy was that there very few people who needed enough experiment designs to keep us in business because they would get a design, take months or even years to run the experiment, then need our help or software for about two hours to interpret the results. When the trials are free and run this fast, the design of the experiment and the analysis become significantly more difficult and time-consuming. That’s fine with us because we have some unique advantages there.

The response to our efforts was the second surprise. We had never heard of this book, but it turns out to be very popular in the trading community. Many people have read, and those that haven’t, had at least heard of it. Many people had a common basis for understanding our work, and we are starting to have real conversations about applying this technology. We are moving on to models that we couldn’t even consider when the cost of trials was an issue.

Is Complexity a Bad Thing?

Complexity is taking a lot of beatings lately. It’s a carrier of misunderstanding, and we don’t want any misunderstandings – especially when we are full of fear. And there is no doubt that humans can very rapidly make things far more complex than they can understand and control. Our very capabilities carry the seeds of our own destruction. However, as long as you can understand and control complexity, it is your friend.

I’m no fan of complexity myself. I want everything not only simple enough so that I can understand it, but also so simple I can’t misunderstand it. I want it simple enough so that I can learn everything about it. Yet it is complexity that makes our modern life modern. Cars are more complex than walking, computers are more complex than pencil and paper, and a steel driving machine is more complex than a hammer.

A monkey understands a hammer. You could teach a monkey to use a steel-driving machine, but not to build or maintain one. Most of us are like that monkey. That is, we can only make use of a vast amount of the modern world because someone has found a way to package complexity in an easy-to-use wrapper. We can exploit complexity because someone else has taken care of all of the details. In fact, complexity beats simplicity on a wide variety of measures.

Once complexity gets out of control, it can be very dangerous. A drunk behind the wheel of a car and a madman with a nuclear weapon are just two such examples. Complexity is powerful, and control is easily lost. Simplicity may win in the long run, but complexity wins time after time in the short run. Are you going to bet on the man with the hammer or on the steel-driving machine?

I think we are all going to bet on the machine. It has too many advantages, each of which is an encapsulation of complexity. We want complexity on our side.

I think this is true when you are following or building an investment management system or a trading system as well. If you want to beat the other systems that are out there, you have to have more complexity on your side. You need to add the right kind of complexity, you need to keep it under control, and you need to deliver in an easy to use package even if you are the only user. The more details you need to keep in mind to use your system, the more you are likely to misuse it.

Let’s look at a simple investment plan. Buy real estate. Lots of people have made lots of money with a plan no more complex than this, but it is easy to see how a little complexity could make it better. For example, it gets considerably more complex if the plan is simply expanded to add one condition: “Buy real estate if you can afford it.” The added condition encapsulates a significant amount of complexity, and that complexity should make the plan work better.

If you actually want to make sure that you are getting a good deal, that complicates the plan even more. “Buy real estate when you can get it for less than it is worth for a price you can afford.” It still looks like a pretty simple plan, but we can all attest that our ability to state simply and plainly what needs to be done doesn’t really mean that it is simple or easy.

There isn’t really much simplicity around until we impose it by not thinking about most of what is going on. That’s not a problem for us. People are very good at ignoring most of what is going on. It’s how we get through every day.

A Thousand Trials

Usually, when you run an experiment, the primary cost of the experiment is the cost of running the trials. Designing the experiment is usually fairly simple, and never very expensive. The analysis requires some moderately specialized skills, but you can usually afford the analysis if you can afford the trials. Now, for the first time in our experience, we have discovered a field of investigation where the cost of the trials is almost nothing. This has profound side effects that we did not anticipate.

We should have because the most complex experiments were always run by people who had the least expensive trials, but even testing print cartridges makes a noticeable dent in a budget when you want to run a hundred trials. At the very least, the tests will still take hours, perhaps days, to run.

When we started applying Design of Experiments to financial engineering, we were strictly interested in intraday trading systems. Testing these systems had a moderate cost. For example, if we wanted to run sixteen trials on three years of data, it did not cost us much money, but it did take several months. That has a significant cost when you want to know the answers right now.

We wanted to talk about those experiments, but a number of factors prevented that We looked around for another system that we could experiment on. Ernest Chan recommended that we look at systems in Short Term Trading Strategies That Work and High Probability ETF Trading, two books by Larry Connors and Cesar Alvarez. As Ernie correctly perceived, Connors and Alvarez described systems that lent themselves very well to a Design of Experiments. We came up with lots of ideas for experiments right away, and we picked one out to try it out.

My partner Ron is very fast at this kind of thing, and he had a simulator ready to run the test after about a day of work. He had to organize the data because we had never worked with an end-of-day system before. Then he started running the test. We knew they would go quickly, but they only took about a quarter of a second to run a trial on fifteen years of data. It took far less than half a minute to run 56 trials.

This is as close to instant gratification as an experimenter will ever get! It took a few hours to analyze the results, but it was essentially “same day service”. You usually have months separating the planning of the experiment and the analysis.

Immediately, we began to think of more complex experiments looking at more factors and using higher order models. Since the higher order models required more trials, we rarely even thought in that direction. The cubic model was the highest order model supported by the software we were using, and we very rarely had the opportunity to use that. Most people are looking for experiment designs with less trials, not more. In fact, the point of most experiment design software on the market today is to get the most information out of the fewest number of trials.

Notice how the goal changes when the cost of the trials approaches zero. The goal in this environment is to get the best possible predictions without regard to the number of trials it requires. Another fifty trials? Another hundred trials? How about another thousand trials? We don’t care when the trials are free.