JOURNAL OF COMPUTERS (JCP)
ISSN : 1796-203X
Volume : 2    Issue : 1    Date : February 2007

Noisy K Best-Paths for Approximate Dynamic Programming with Application to Portfolio Optimization
Nicolas Chapados and Yoshua Bengio
Page(s): 12-19
Full Text:
PDF (797 KB)


Abstract
We describe a general method to transform a non-Markovian sequential decision problem into a
supervised learning problem using a K-bestpaths algorithm. We consider an application in financial
portfolio management where we can train a controller to directly optimize a Sharpe Ratio (or other
risk-averse non-additive) utility function. We illustrate the approach by demonstrating experimental
results using a kernel-based controller architecture that would not normally be considered in traditional
reinforcement learning or approximate dynamic programming.We further show that using a non-additive
criterion (incremental Sharpe Ratio) yields a noisy K-best-paths extraction problem, that can give
substantially improved performance.