Learning from Delayed Rewards

Christopher John Cornish Hellaby Watkins submitted his University of Cambridge PhD thesis 'Learning from Delayed Rewards' in May 1989. Watkins's own account states that the thesis described a range of reinforcement-learning algorithms, including one-step Q-learning, and sketched a proof of its convergence.