Famous techno inventor Ray Kurzweil is interviewed in Fast Company, about his new book on living forever (lots of supplements...and nanobots), and also his new strategy for a hedge fund. I'm a big fan of anti-oxidants, as they minimize mutations in DNA that appear related to aging. I'm not a big fan of his investment philosohy. As Kurzweil describes it:
For static problems, neural nets will get you the right model, no matter how nonlinear, in the long run (ie, when a bunch of monkeys finish typing Hamlet). You don't even need a theory, the data leads you there. The problem is that in the nonphysical world "degrees of freedom" are very few, much less than you think. For example, if you were predicting defaults, a simple model that merely had a dummy variable for being in the telecom sector in 2000, and being in Hotels in 1990, would do as well as S&P ratings. That's clearly a stupid model, but for only two parameters, you get a lot of statistical power!
Neural nets explain everything and nothing. Their finite sample properties are lousy, especially in the real world where data are not iid, they are correlated in very nonstationary ways over time and industry. There is no transparency that allows one to understand the interactions and thus rectify errors, and the "fit-test-validate" approach is really merely a test of fit and subsample stability, since inevitably one mines the data until the validation performance is similar to the test performance (do this 100 times and 5 will be 'significant' no matter what). I think of it as a fancy way to run a regression with too many variables, their squares and cubes, and the product of all these terms.
Then there's this interesting note on his system's performance:
As he didn't mention his standard deviations, just returns, he is revealing some real ignorance here, since
So we have our sophisticated pattern recognition model -- we don’t program it a priori with our preconceived ideas of how the market should work. It’s very much data driven, and it's building its models based on what it sees. But it has the ability to build sophisticated models of how financial data interacts with each other.Warning! Warning! Neural Nets likely! Heck, even my hero Richard Feynman was optimistic about neural nets in the 80's. What's the attraction between smart people and neural nets?
For static problems, neural nets will get you the right model, no matter how nonlinear, in the long run (ie, when a bunch of monkeys finish typing Hamlet). You don't even need a theory, the data leads you there. The problem is that in the nonphysical world "degrees of freedom" are very few, much less than you think. For example, if you were predicting defaults, a simple model that merely had a dummy variable for being in the telecom sector in 2000, and being in Hotels in 1990, would do as well as S&P ratings. That's clearly a stupid model, but for only two parameters, you get a lot of statistical power!
Neural nets explain everything and nothing. Their finite sample properties are lousy, especially in the real world where data are not iid, they are correlated in very nonstationary ways over time and industry. There is no transparency that allows one to understand the interactions and thus rectify errors, and the "fit-test-validate" approach is really merely a test of fit and subsample stability, since inevitably one mines the data until the validation performance is similar to the test performance (do this 100 times and 5 will be 'significant' no matter what). I think of it as a fancy way to run a regression with too many variables, their squares and cubes, and the product of all these terms.
Then there's this interesting note on his system's performance:
But our system works, we’ve been trading with real cash for 2.5 years. We make 80-90% annual gains. We plan to launch this year a hedge fund using our technique.80% returns on what? For a long short fund, the denominator is arbitrary. You can be long 100 dollars and short 100 dollars. If you make 10 dollars profit, is that a 10% return? No, you don't need 100 dollars capital, perhaps only 10 dollars capital is needed by your broker. But is it then 100% capital? No, no fund manager would risk that much with their money. The largest annual standard deviations I have seen targeted are in the 30% range, but most are for modest 5-12% annualized standard deviations.
As he didn't mention his standard deviations, just returns, he is revealing some real ignorance here, since
- Mentioning returns only is arbitrary, you might just as well say they were positive. Not understanding the way to measure performance (risk/reward, Sharpe, etc.) means he probably does not have his pattern recognition algorithms maximizing the right thing.
- The 90% return implies a simple return/stdev ratio of at least 3 (given a top standard deviation of 30%), and probably much higher (presumably standard dev is much lower than 30%). That's possible, but highly improbable, sort of like a long only manager saying he hopes to beat the S&P by at least 10% annually. When someone shows me a backtest, above a certain point their performance actually hurts their attractiveness because it suggests more about overfitting and bias than finding a great opportunity.
HedgeFundGuy - am 2005-03-31 04:47 - Rubrik: Finance