Tutorial Descripton


Data Over-Mining and How to Avoid It
by Halbert White


Abstract
Modern computing and advances in learning methods have created an unprecedented capability to rapidly create and evaluate asset-trading systems. But just as a single learning model can over-fit a given set of data, creation and evaluation of a large number of trading systems can over-mine the data. The result is one or more systems that look great on paper but that can fail dismally in practice. In this tutorial we describe Data Mining Reality Check (DMRC), a new statistical method that can be used to separate "fool's gold" trading systems from the real thing. We illustrate DMRC by performing and evaluating a massive search for calendar effects in the US equity markets.

Biography
Halbert White is Professor of Economics at the University of California, San Diego, and Senior Partner and Chief Scientist with Bates White & Ballentine, LLC, a nationwide US economics consulting firm. His research interests include artificial neural networks, finance, econometrics, and forecasting; he is the author or co-author of over one hundred articles and numerous books in these areas He received his A.B. in economics from Princeton University and his Ph.D. in economics from MIT.



Back to Tutorial