As anyone who has ever tried to do optimization on automated trading systems will tell you, there are many problems. The optimal numbers from the past almost certainly won’t be the optimal numbers for the future. In fact, every data set you test will produce different numbers. Why then would you do system optimization?
We believe that the most important output from these efforts is the discovery of relationships between decision factors. It’s not the price you pay when you buy or the price you get when you sell that determines profit, it’s the difference between the two.
That means that the most important relationships we are looking for are the relationships between selection factors and exit factors. We like using response surface graphs to show these because they show the relationships clearly, and the general picture they show differs little when we try it on different data sets. This leads us to believe that the relationships are robust even though the optima are often simply coincidental.
Response surface graphs also show which settings are clearly dysfunctional. It becomes pretty clear after looking at just a few sets of results that there is a range of settings that might work, and a range that is clearly disastrous. This is true for both the individual factor settings and for the relationships between them. Note that findings of dysfunctional settings tend to be far more robust than findings of optima. Knowing what to avoid doesn’t tell you how to do the job right, but there is clearly value in avoiding known disaster area.
In addition to interactions between selection factors and exit factors, there are also clearly interactions between various selection factors and interactions between various exit factors. For example, an either OR rule for selection factors means that you get into more trades while an AND rule reduces the number of trades. We know that without performing any tests, but more trades or less trades is not the critical answer we need. We need to know whether an OR rule gives us a higher winning percentage and greater profit, and whether an AND rule gets us into more trades that lead to more profits.
The same is true with exit rules. Do they work well together or do they get in each other’s way? Generally speaking, there is a desired exit process and a set of fall-back exit processes. You don’t want the fall-back processes to unduly interfere with the desired process, but you can’t let your primary exit process to be so dominant that the backup plans don’t go into effect until it is too late. Response surface graphs show these relationships very clearly.
Not all of the critical interactions are two-factor. There can be three, four, or even more factors that are working together. The more complex an interaction, the less likely it is to be robust, but many three-factor reactions are robust. Showing three-factor interactions is a minor problems that we address currently with animations. Here is an example.
We show the most powerful factors on the axes. Here this is the critical selection factor and the primary exit factor, and that is generally the case. The third factor, the backup exit strategy, we show with the animation. The animation shows that the third factor changes the degree but not the nature of the reaction between the first two factors. This presents a fairly clear picture of what is going on in the trading system, and we can see that these pictures will look very similar if we apply them to a variety of data sets.
It is important to see whether the charts you get out of the data support or contradict your basic theories. The basic theory here is that a move in one direction is most likely to be reverse if it swings far enough, and most likely to make a solid profit when you are looking for the swing back to be half as large as the initial move. The size of the swings varies considerably, but the relationship between size of the initial swing and size of the secondary swing is pretty consistent.
The fallback exit factor is of less importance. By itself, it never determines whether the system will win or lose, but does affect how much it will win or lose.
What settings should we choose for our working system? It’s still something of a crap shoot, but we have a reasonable idea of the direction to shoot in. The chances of picking the best values is only slightly more than zero, but the chances of picking pretty good values is very high. There may be other response surfaces that you can look at like days in position or winning percentage that will push you in one direction or another, but you still are moving from analysis of the past to prediction of the future. The stock market will never be a chemistry problem.