Check out this new paper on SSRN. All that Glitters Is Not Gold: Comparing Backtest and Out-of-Sample Performance on a Large Cohort of Trading Algorithms
It seems that everyone backtesting algorithms are simply wasting their time, as it is just an exercise in optimizing their results, which doesn't work! Sharpe ratios also are no good.
What does work? Higher order movements (like volatility and maximum drawdown), show significant predictive value of relevance to quantitative finance practitioners. Higher order movements. Hum..... I wonder if I am on to something..... Like Directional Trading using Volatility.....
Download the paper here: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2745220