Importance of Benchmarking
As traders, we can sometimes fall into the relevancy trap, focusing on and trading only those strategies developed by our own hands. We want our efforts to be rewarded. The best way to do that is to actually trade the strategy. However, for most traders who inadvertently fall into the twin traps of curve fitting and data mining, their best efforts fall far short of what is necessary to succeed in real markets, with real money. This is why it’s always important to benchmark your strategy efforts against well established and robust methodologies. Methodologies that have worked well in the past and in all likelihood will work well in the future. The benchmark strategy should be the minimum hurdle your own strategy must attain before consideration is given to trading it. In my opinion, one of the best publicly available strategies to benchmark against is the Turtle strategy. If your own efforts can’t surpass the Turtle strategy then you are much better off trading it, not your own creation. Please refer to the following link to read a more detailed discussion on the importance of benchmarking.
Benchmarking Key Level vs Turtle
Let’s benchmark Key Level against the Turtle strategy. Benchmarking should comprise two parts;
- Robustness Analysis and
- Performance Analysis
Robustness Analysis
A strategy’s robustness is dependent upon its amount of out-of-sample performance, level of curve fitting and degree of data mining.
A robust strategy will generally have plenty of positive out-of-sample performance, little curve fitting and no data mining.
Let’s see how Key Level holds up when it’s compared to the Turtle benchmark strategy?
Apart from the release date, Key Level compares favourably to the Turtle strategy on a robustness comparison. Although Key Level looks relatively young (2008) when compared to the Turtle strategy (1983) we should not dismiss its youth when you understand the majority of strategies rarely make it to their first anniversary. Its “youth” should be applauded for the significance of its out-of-sample performance. Being a pattern based strategy underpins it soundness where it doesn’t rely on a collection of indicators with subjective and optimised variable values. Like the Turtle strategy Key Level has only 2 variables which have the same value across all markets and across both buy and sell setups. This definitely minimises any level of curve fitting. Also Key Level, despite being designed to trade currencies, is profitable across an out-of-sample portfolio of diversified markets. This absence of data mining and minimalist curve fitting all up makes Key Level a robust strategy.
Performance Analysis
If we’re satisfied both strategies are robust and will (hopefully) continue to enjoy a positive upward sloping equity curve, the next question we have to ask ourselves is which strategy is superior?
For this we need to review the numbers.
Well on first glance, comparing their individual equity curves, it appears the Turtle strategy is hands down superior making over $1.5m net profit compared to Key Level’s $0.3m. On a net profit criteria the Turtle strategy romps it home, or does it? Let’s go through the metrics I focus on to see what we can learn.
Survival
My number one objective as a trader isn’t what you’d expect. Yes, I do want to make money. However all my energy and concentration is purely focused on surviving the markets. Nothing else comes a close second. I think of myself as a Risk Manager first and foremost rather then a trader. Anyone can make a profit in trading. What many cannot do is manage a string of losses. Banking profits is effortless. Winning trades generally take off, look after themselves and don’t give you any grief. They’re effortless. However losses are a totally different story. Suffering, recording and funding losses is exhausting, both on your trading account and on your confidence, damaging both your mental and trading capital. It damages your trading soul. It’s what can drag you down and cause all sorts of strife. It’s the pain of trading.
So my first priority is to ensure my risk-of-ruin (ROR) is at 0% and that it remains stable. Not fluctuating. Not shifting upwards. In my opinion ROR is the most important concept in trading. Nothing comes a close second. If you’re not familiar with ROR then I’d suggest you stop trading immediately and become familiar with it. A good place to start is my book The Universal Principles of Successful Trading (Wiley 2010).
With 50 units of money combined with a positive expectancy over 20% both strategies have a 0% ROR, so that is a positive start.
In addition their respective 0% ROR should remain stable as both strategies to date have been robust.
Risk/Reward
On a single contract basis we know the Turtle strategy makes over four times what Key Level does however it does it with a larger drawdown of around -$86,000 compared to Key Level’s -$37,000.
Rather then looking at the raw net profit and worst drawdown my focus is always on the old fashion risk/reward trade off. What do I receive for the potential risk? On this measure the Turtle strategy, even with its large drawdown has a superior risk/return ratio of 18 compared to Key Level’s 10. On a risk/adjusted basis it is also superior with an Ulcer Performance Index of 2.3 compared to Key Level’s 0.7.
So although the Turtle strategy has a larger absolute drawdown, it does so by making more profit per dollar of drawdown then Key Level does. It generates more bang for each draw down dollar.
[As a sidebar I should note its not really surprising. Breakout strategies are generally superior to retracement models, so its not a surprise to me the Turtle strategy is ahead on this measure. Now although Key Level is behind its not a reason to disregard the approach as Key Level does capture a different part of market structure that the Turtle or breakout strategies do not.]
Risk
The next insight I’m keen to focus on is the average risk, or average stop per trade. Naturally my preference is for a lower risk and this is for two very important reasons. One, I naturally prefer to risk less capital then more and secondly, its cuts to the efficiency of the strategy in making money.
No doubt you may have heard or read the comment where it’s suggested that getting into a trade is not important. What is important is where you get out? Don’t worry about the entry, its all about the exit.
For me that’s a red flag. It suggests to me the presenter or author in all probability doesn’t trade because the entry and stops/exits are both terribly important. You can’t rank one above the other as your entry and stop/exit represents the risk per set-up and is the lynch pin to your position sizing according to your money management. It’s the lynch pin to the potential money you can make.
Remember money management is the secret behind both our survival (lowering our ROR) and our prosperity. As our account balance grows money management allows us to increase our position size thereby increasing our net profit for a positive expectancy strategy.
So the higher average risk or stop per set-up means a trader will have a lower position size relative to a strategy which has a lower average risk or stop per setup.
The biggest draw back of the Turtle strategy is the large size of its stop. It uses a 2 week stop giving it a very high average risk per trade of 4.5%.
Key Level has a much smaller average risk per trade of only 0.6%.
According to their respective average risk measures, Key Level should be a far more efficient strategy in making money since its lower average risk per trade allows for larger position sizes compared to the Turtle strategy.
Let’s review the next performance metric to have a look.
Efficiency (with Money Management)
IF we survive in trading our second objective is to make money. And since we know the secret behind earning big money is money management we as traders want to know how efficient a strategy is when the money management strategy is applied.
In this example I have applied the Fixed Percentage money management strategy where I’ve assumed a starting account balance of $50,000 where 2% of the account balance is risked per set-up.
Now on a single contract basis we know the Turtle strategy is superior to Key Level by making over four times the net profit ($1.5m v $0.3m).
However, looking at a single contract result is misleading, incorrect, short sighted and bereft of traps for the unwary trader.
And why?
Because single contract results can hide the existence of big stops.
Many strategies only look good because they use large stops, where the developer looks to avoid being stopped out, taking profits quickly, or only when a profitable close occurs, regardless of how many days, weeks, months have gone by!
By examining the efficiency of a strategy, by reviewing their profitability with money management applied, the existence of big stops can no longer be hidden and the real power of a strategy (or lack of power) is revealed.
As you can see Key Level, when a conservative 2% Fixed Percentage money management strategy is applied, has hypothetically made $387m compared to the Turtle strategy’s $419m giving Turtles only a small advantage over Key Level.
Key Level is a very efficient and profitable strategy.
Remember it’s always important to look under the hood!
Difficulty in Trading
Each model has a high level of 23/19 consecutive losing trades. So neither is easy to trade. But then trend trading is never easy. However these losing streaks can be reduced by adding a complimentary and diversified trading strategy to your portfolio of strategies, such as Key Breakout (breakout trend trading),d Key Swing (Mean-reversion trend trading) and Key Exhaustion (top and bottom picking).
Key Level certainly has a higher worst percentage drawdown and longer worst drawdown period, however it has a much lower absolute worst drawdown of -$37,000 compared to the Turtle’s strategy of -$86,000.
In my opinion Key Level is less difficult to trade.
Metrics
The metrics are self explanatory however I just want to highlight how efficient Key Level is as its efficiency ($387m) is based on almost half the number of Turtle trades with only half the average profit! It also does it with less exposure to the market where Key Level’s average holding period for winning trades is only 30 days compared to the Turtle strategy’s 46 days. Key Level is impressive.
Benchmark Summary
In my opinion, although the Turtle Strategy has by far more out-of-sample performance data compared to Key Level, I feel the latter is superior.
Key Level has been performing well with a positive equity curve since its release in 2008. Not only on the currencies, but also on a diversified portfolio of markets it was never designed to trade. It’s also impressive knowing most strategies do not make it to their first anniversary! Like the Turtle strategy its pattern based with only 2 variables so it doesn’t rely on overly optimised values for profitability. Key Level has a 0% ROR and although it has a lower risk/return trade off on a single contract basis, it’s by far the superior strategy in terms of efficiency ($387m) due to its much lower average risk per trade (-0.6% v -4.5%).
Key Level is less difficult to trade with a lower absolute drawdown.
For me, Key Level is the superior strategy. My efforts have paid off.
You will need to carry out a similar benchmark of your own strategies and if they’re not superior to the Turtle strategy you should consider trading it over a portfolio that you can afford to fund, or even better, consider purchasing yourself a copy of Key Level!