Beyond Trading System Optimization

As anyone who has ever tried to do optimization on automated trading systems will tell you, there are many problems. The optimal numbers from the past almost certainly won’t be the optimal numbers for the future. In fact, every data set you test will produce different numbers. Why then would you do system optimization?

We believe that the most important output from these efforts is the discovery of relationships between decision factors. It’s not the price you pay when you buy or the price you get when you sell that determines profit, it’s the difference between the two.

That means that the most important relationships we are looking for are the relationships between selection factors and exit factors. We like using response surface graphs to show these because they show the relationships clearly, and the general picture they show differs little when we try it on different data sets. This leads us to believe that the relationships are robust even though the optima are often simply coincidental.

Response surface graphs also show which settings are clearly dysfunctional. It becomes pretty clear after looking at just a few sets of results that there is a range of settings that might work, and a range that is clearly disastrous. This is true for both the individual factor settings and for the relationships between them. Note that findings of dysfunctional settings tend to be far more robust than findings of optima. Knowing what to avoid doesn’t tell you how to do the job right, but there is clearly value in avoiding known disaster area.

In addition to interactions between selection factors and exit factors, there are also clearly interactions between various selection factors and interactions between various exit factors. For example, an either OR rule for selection factors means that you get into more trades while an AND rule reduces the number of trades. We know that without performing any tests, but more trades or less trades is not the critical answer we need. We need to know whether an OR rule gives us a higher winning percentage and greater profit, and whether an AND rule gets us into more trades that lead to more profits.

The same is true with exit rules. Do they work well together or do they get in each other’s way? Generally speaking, there is a desired exit process and a set of fall-back exit processes. You don’t want the fall-back processes to unduly interfere with the desired process, but you can’t let your primary exit process to be so dominant that the backup plans don’t go into effect until it is too late. Response surface graphs show these relationships very clearly.

Not all of the critical interactions are two-factor. There can be three, four, or even more factors that are working together. The more complex an interaction, the less likely it is to be robust, but many three-factor reactions are robust. Showing three-factor interactions is a minor problems that we address currently with animations. Here is an example.

We show the most powerful factors on the axes. Here this is the critical selection factor and the primary exit factor, and that is generally the case. The third factor, the backup exit strategy, we show with the animation. The animation shows that the third factor changes the degree but not the nature of the reaction between the first two factors. This presents a fairly clear picture of what is going on in the trading system, and we can see that these pictures will look very similar if we apply them to a variety of data sets.

It is important to see whether the charts you get out of the data support or contradict your basic theories. The basic theory here is that a move in one direction is most likely to be reverse if it swings far enough, and most likely to make a solid profit when you are looking for the swing back to be half as large as the initial move. The size of the swings varies considerably, but the relationship between size of the initial swing and size of the secondary swing is pretty consistent.

The fallback exit factor is of less importance. By itself, it never determines whether the system will win or lose, but does affect how much it will win or lose.

What settings should we choose for our working system? It’s still something of a crap shoot, but we have a reasonable idea of the direction to shoot in. The chances of picking the best values is only slightly more than zero, but the chances of picking pretty good values is very high. There may be other response surfaces that you can look at like days in position or winning percentage that will push you in one direction or another, but you still are moving from analysis of the past to prediction of the future. The stock market will never be a chemistry problem.

Advertisements

What is My Opponent Holding?

Having spent far more hours playing poker than sanity allows, I like to think that I have more than money to show for it. I hope that I have come away with lessons that apply to business, investment, and life in general. That may be a stretch, but I cling to that illusion.

I am constantly trying to figure out why the best players are that good. It’s easy enough to see why the bad players are so bad. They bluff too often, call too often, don’t raise to protect their hands, don’t control their emotions, and have no understanding of position. You can become a mediocre to good player simply by addressing these problems, but great players have something else. As a matter of fact, even the great players still have some of these problems on an occasional basis.

Something else makes them great. I don’t think it is that the best players have more heart, more brains, or more luck, though everything does go easier when you have more advantages on your side. I think what separates the great players is their awareness that poker is a relative game. The great players spend most of their attention focusing on what their opponent has. Bad players often don’t think at all about the opponent’s cards, good players think about them at the time of decision, and great players think about them even when they are not in the hand. Every time you show a hand down, you are giving a great player another weapon to beat you with.

The great player knows that there is no greater advantage in poker than knowing your opponent’s cards. We could all be poker millionaires if we knew what our opponents have and they don’t know what we have. One of the best examples of this was a couple of years ago in the main event at the World Series of Poker. Sammy Farhar called a massive pre-flop raise early on the first day with only a pair of deuces. He knew that the nervous amateur making the bet would not have made that bet with anything other than a pair of aces. When a deuce came on the flop, Sammy got the rest of the amateur’s chips. I’m don’t think that the implied odds that he got really justified Sammy’s call, but know just exactly where he stood was a powerful incentive that he could not resist. He made it pay off.

Daniel Negraneau is always talking about what his opponents have when he is heads-up in a hand. I am sure that he only refrains from discussing what other people have in their hands in other situations because that would be outside the etiquette of the game, but I am sure he is thinking about it all of the time. I find Daniel the most instructive of the leading pros to watch for just this reason. Listen to the questions that he asks himself.

Why did he bet that much? Why did he check? What did that card do for him? How likely is it that my opponent is bluffing? Is this a guy who only bets the nuts? How nervous is this person about this bet? How many chips does this opponent have left? What is this opponent likely to do if I raise?

Daniel doesn’t always come up with the right answers and he’s quick to notice when he is wrong, but he is always asking and answering the questions. That gives him many chances to be right than he would have if he never thought of those questions.

It’s just like that in business and investing. Asking all of the right questions doesn’t guarantee that you will win, but it does increase your chances dramatically. There are never any guarantees, but there are lots of edges if you look for them.

Just spend a few minutes thinking about the last few years at some companies disappeared.  They disappeared not because they came up with the wrong answers; many disappeared because they simply didn’t ask the right questions.

The Anatomy of a Quantified Trading Process

A quantified trading system is a system that can be automated, but it is not necessarily an automated system. A quantified process is precisely defined, uses numeric inputs, and produces measureable outputs.

Each quantified trading system starts with a Profit Plan. A profit plan is a theory that describes the actions that will be followed under the plan and explains how those actions will be converted into profit. Never forget that your profit plan is a theory. If it works, it is a good theory; if it doesn’t, it’s not. Theories of profit rarely become laws.

All theories start with observations. For example, you might first observe that stocks tend to bounce back after a sell-off. That makes you look closer, and you find this behavior seems to be a lot more prevalent in stocks that are doing well than in stocks in the general process of tanking. Your conceptual theory might then become “If a stock is doing well overall, you can make money buying on the dips Dump them as soon as the price bounces back.”

That’s a theory, but not a quantified theory. You can make it a quantified theory, as Connors and Alvarez did, by adopting numbers that define “doing well overall”, “the dips”, and “bounces back”. I am sure there are a lot of traders out there that did well enough with this rule with no quantification, but quantification brings two specific advantages. Once a theory becomes quantified, it can be executed by a computer, and once it can be executed by a computed, it can be tested exhaustively.

The process that Connors and Alvarez define on page 62 of Short Term Trading Strategies That Work is a good example of what is needed.

.

1. The S&P 500 Index is above the 200-day moving average

2. The 2-period RSI of the S&P 500 Index closes below 5.

3. Buy the S&P on the close.

4. Exit when the S&P closes above its 5-period moving average.

Step number one checks to see if the S&P is “doing well overall”. Is the 200-day moving average the best test of this? You can test for this. (We did, and found that every long-term average was almost equally effective so 200 is as good a number as any.)

Step number two is a quantified definition of how to spot a dip. You can now test whether this works, Is this the best indicator? You don’t know, but you can now test it against other alternatives. Quantification does not start with the assumption that you know the right numbers; it starts with the assumption that you can test and discover the right numbers.

Finally, step four quantifies what a bounce back is. Is the 5 period average the right indicator? Again, you can test for that.

A quantified system has rules that can be automated and tested. If you are automating a quantified trading system, keep in mind that the ability to run tests is likely to be the most important contribution of such a system.

High Probability ETF Trading: 7 Professional Strategies to Improve Your ETF Trading

If you are using any kind of a system based on technical analysis, you need to own and read this book as well as the earlier work by the same authors (Larry Connors and Cesar Alvarez), Short-Term Trading Strategies That Work. Both books are clear and understandable, and both lay out approaches that really do work.

The difference between the work of  Connors and Alvarez and almost everyone else is that they show you exactly how to APPLY commonly available statistical tools.  Everyone who has spent a couple of hours watching CNBC knows what the VIX is, for example, but it is just noise to the vast majority of people.  Connors and Alvarez provide explicit instructions that enable the application of this number to buying and selling decision making..

Take the 200 day moving average, for example.  A fine number, easily available from a number of sources, but specifically how should it affect your trading strategy?  Connors and Alvarez use this measure very simply and directly.  If the target security is above the 200 day moving average, only consider long moves.  If it is below that average, only consider short moves.

This is simple and measurably effective.  You can test any of the strategies in any of their books with and without this rule, and the difference is clear.  The strategies work when this rule is included; they don’t when it is not.

This book introduces their approach to short selling and money management tactics. The short selling tactics are a logical extension of their long strategies, but the money management is a new wrinkle. Like most of their presentation, it’s not fancy but it does offer a quantified approach for averaging into a position. The best aspect of everything that these books offer is a solid foundation for quantification because without that foundation, experimentation is useless and progress is uncertain.

The only quibble  that I have with either book is over the definition of the word “strategy’. As they say more than once in this book, it’s all about selling into overbought conditions and buying oversold securities. One strategy for which they present several effective tactics. I’m not knocking this. It’s a good strategy, and they make it better when they only buy in healthy markets and sell in overall weak markets.

They present a lot of useful statistics about a wide variety of indicators, but they are only looking for indications of overbought or oversold markets. The indicators they suggest are pretty reliable and work effectively when used as they suggest, but they are not different strategies. For example, you could not diversify by using several of their different strategies because they all tend to lead you into the same trades. That is, security A is overbought or oversold. Several indicators may reveal this, but that does not mean that there are several different opportunities.

A really good book makes you ask questions. For example: Which indicator is the best? Is there a combination of these indicators that is more effective than any single indicator on its own?

Connors and Alvarez also present a variety of indicators to tell you when to get out of a position. Each of these indicators point to the market no longer being oversold or overbought. Notice that the strategies are not looking to profit further in that direction. The end-of-day aspect of the exit rules may be trying to catch a little of that over-bounce. They make a point of saying that you shouldn’t get out during the day if you hit the target for that day. I suspect they are right, but that is one of the few assertions that they do not back up with statistics.

Engineering is all about gathering and evaluating measurements, then putting those measurements to work.  This book, like its predecessor, provides a good hands-on introduction to the basics of financial engineering.

Review of Short-Term Trading Strategies That Work

I’ve just finished reading Short-Term Trading Strategies Work for at least the tenth time. That may make it the most read book of my life. Of course, it short (less than 150 pages with lots of charts), and that helps considerably. Thanks to Ernest Chan for recommending it in his blog.

The book is good solid science. The approach they take is totally quantified and verifiable. They describe everything completely enough so that you can test every assertion, every rule, and every strategy. We tested almost everything, and we found that they all hold up under close scrutiny. These are short-term strategies that work.

You may be asking yourself why anyone would give away a money-making strategy that works. My guess would be that they have short-term strategies that work better, and they are keeping those to themselves. Also, technical analysis is not a privileged information approach like arbitrage that gets less valuable with broader visibility; rather, when there are enough people who believe in the same technical analysis, it becomes a self-fulfilling prophecy.

That’s no guarantee of how well these strategies will work in the future, but as they consistently and correctly point out, quantified analysis is about getting an edge, not a guarantee. This book is about the simple basics of getting and exploiting an edge. They follow a simple engineering process.

1. Observe the data.

2. Measure anything of potential relevance.

3. Analyze those measurements.

4. Look to turn those measurements into strategies.

5. Test the strategies thoroughly.

For example, they observe that stocks that hit short-term lows tend to rebound from there as long as those short-term lows are above the long-term moving average. They develop a precise definition of a short-term low (has to be precise to feed it to a test program) and adopt a precise definition of a long-term moving average. That allows them to search for specific trade opportunities. They add a specific exit strategy and test it using historical market data.

As an aside, one reviewer of this book labeled their approach as mean reversion theory in a new wrapper. I see no such underlying bias. This strategy, for example, could appeal as much to people who believe that the long-term trend dominates the short-term trend as to those who believe in mean reversion. Besides, the real question is whether or not a strategy works rather than what kind of underlying theory supports it.

There will be many people who find that simply adopting these strategies will be a big improvement over their current approach, but the proper perspective is that here are a set of basic building blocks. If you want better performance than what Connors and Alvarez had you on a platter, there is still a lot of work to do.

How To Foster the Starting of New Businesses

The only government program that I have seen that really ever seemed to do any good as far as creating new businesses and new jobs was the plan that allowed people to take their unemployment compensation and use it to start a new business. I am sure that most of those businesses failed because most new businesses fail, but some surely succeeded and employed people. Everything else I see coming out of the government really is not much help to the really small business person.

First, the government needs to stop doing things that get in the way. In particular, the government needs to stop getting in the way of new businesses trying to raise capital. For example, the changing of a “qualified investor” was made five times as stringent during the last year of the Bush administration. Maybe it was time to do this if you believe that the old law made sense because inflation has surely changed the notion of what is a comfortable nest egg, but the old law limits the right of ordinary Americans to be capitalists as much as any law on the books.

Making this law more stringent is a huge case of Nanny-state. You can’t foster new businesses, i.e. encourage people to take risks, by saying that they are unqualified to take them. The government does far more to prevent people from investing in legitimate new businesses that should have a beneficial effect for the whole society than it does to put people like Bernie Madoff out of business. Whether or not you believe that protecting people from themselves is a legitimate role of government, you have to conclude that the current rules and processes do not do everything they can to promote new business. Let’s take a look at what else they could do.

Create a Special Tax Break for Providers of Start-Up Capital

Should the guy who makes a hundred grand betting on Joe’s Widgets pay the same tax as the guy who made a hundred grand on Bank of America? Not if you want people to be eager to participate in new ventures. Is this fair to Bank of America, Microsoft, and IBM, to name a few? Not really, but they don’t need start-up capital. The cost to them is small and likely is a less than appropriate tax on their hugeness. Do you want create more small businesses or cater to the status quo? If you want a vibrant economy, you better not be catering to the status quo.

I’d like to see the first hundred percent return on such investments be tax free. That is, if you put up a million dollars to get Joe’s widgets started, you would not pay any tax on the first two million you took back out after Joe hits it big. That is, you don’t get taxed on getting your money back, and you don’t get taxed on the first matching win. I’d go even further and tax only fifty percent of the next hundred percent, and only seventy-five percent of the next hundred percent.

I don’t think that this is overly generous since most new businesses fail without ever making a profit for the investors, but it is a heck of a lot better than the deal angel investors get today. (They call them angels because they require so much faith.) The only way to get more money into these operations is to change the risk/reward profile. Most investors are not math-impaired.

Should investors pay a higher tax rate if they made their money from an organization that got bailed out by the government? They should, and actually they already are because the government is taking a big cut.

Problem/Opportunity Funding

Our country faces many problems and many opportunities. There is even occasionally some degree of consensus on what is a problem and what is an opportunity. I think the government and/or private agencies need to be able to support new businesses that can demonstrate that they are solving a problem for the community or opening an opportunity that will increase the prosperity of the community. The problem with such programs are that they disrupt the status quo, but a healthy capitalist economy is one that fosters change. Change is always going to come, and it will run you over if you are not ready for it on every level.

Foster Competition

It was a sad day when Packard went out of business, but it wasn’t devastating. Packard was just one of many auto companies, and they failed all the time. I’m not sure how much GM and Chrysler ended up with, but we could have started hundreds of new car companies at one hundred million dollars a shot. Wouldn’t that be a better use of our money than giving it to people who have proven they can’t manage a company? Wouldn’t that provide more jobs for auto workers? Wouldn’t that ensure that we would offer the world a broader choice in automobiles? Wouldn’t that mean that all cars would get better. The answer to all of these questions has to be yes if you believe in capitalism.

The auto bailout is exactly the wrong thing to do if you want to solve the problem. If it works, we end up with companies that should have failed and which are almost certain to face the same kind of problem in the future. If it doesn’t work, we are out billions and still have the same economic problems we were trying to prevent. A future with fifty car companies is a future far less prone to economic stresses than a future with three.

Redefine Small Business

A Company with $250 M in revenues is not a small business that needs help from the government. By the time a company is generating $10M a year in revenue, it should be able to stand on its own and raise capital on its own. A company needs the most help when it is just a business plan, or a business that is attracting attention and needs a little help to get to the next level. These are the companies that will increase employment, but they are also the companies that have the hardest time getting money from the banking industry. We need a government-guaranteed loan program that focuses on really small businesses that need to get to the point where they can inspire faith.

The most explosive growth of business in the micro-capitalism sphere, and there is a huge gap between that level and the level where the government will even recognize you. We either need to come up with a private solution to this, or the government ought to jump in.

Conclusion

I believe that we can make American business young again with just a few simple changes in our tax law. If we let everyone use their own judgment about these bets, it will help even more.

The government shouldn’t need to participate beyond that, but if it does stick its nose in, the focus needs to be on solving the problem, not in protecting the status quo – the path of least resistance that government always follow.

Nobody Saw It Coming

It doesn’t matter what you are talking about – the dot com crash, the subprime crash, or the tulip bubble. The years after such a crisis are marked by protestations that nobody saw the crash coming. In fact, those protestations are almost always false. True, almost everyone did drink the Kool-Aid, but there were always those that said: “No thanks!”

One of the good things about CNBC is that it creates a record of these events. They know who drank the Kool-Aid and who did not, and they have it on tape. When Peter Schiff was running around telling people that the subprime bubble was the biggest bubble of all time, both CNBC correspondents and their guests howled with derisive laughter. The Daily Show was kind enough to put together a collage of these interviews, and you can see that he was telling the straight story and was being treated like he was a total clown by people who were being total clowns themselves at the time.

Peter may not be the most charming person you ever saw on TV, but even if he had been wrong, he would have deserved significantly better treatment. Given the fact that he was right and his detractors were nothing less than fools, you would think there would have been quite a few public apologies, but I guess Wall Street doesn’t work that way. While lots of people on Wall Street are wrong most of the time, admitting in public that you were wrong is rather exceptional behavior.

There’s always a Peter Schiff, the guy who has it right. He’s always going to be the one who says the bubble will burst, and the guys who have it wrong will be saying, “that will never happen.” The guys who have it wrong are almost always hoping that it will never happen, but most of them don’t believe in never any more than you or I do. They mean “That just can’t happen while I am making all of this money.”

You should pay close attention whenever you hear that phrase or anything like it on CNBC, The more times you hear it said about any possibility, the more likely it is to happen. Wall Street should keep a “That will never happen” Index so that we could all track these events. They wouldn’t be talking about it if it wasn’t a possibility, and the more they talk about it, the more likely a possibility it has become.

Does that mean that any particular disaster will happen in a predictable time frame? Not necessarily, there are a lot of People that always say things like Peter Schiff, and most of them are wrong far more often than they are right. However, if the doomsayers are talking about your favorite investment or something that directly affects your favorite investment, you do need to sort through the doomsayers with an open mind. You don’t just want to say “That will never happen” or you soon may be saying “Nobody could have seen this coming.”