Learning from Delayed Rewards
Christopher John Cornish Hellaby Watkins submitted his University of Cambridge PhD thesis 'Learning from Delayed Rewards' in May 1989. Watkins's own account states that the thesis described a range of reinforcement-learning algorithms, including one-step Q-learning, and sketched a proof of its convergence.
Historical classification
This verified record belongs to LINEAiGE's artificial-intelligence history layer beginning in 1956.
Why it matters
A documented research milestone for Q-learning in the reinforcement-learning literature, represented here strictly as the May 1989 thesis-submission milestone.
People, organizations & technologies
PEOPLE
Christopher John Cornish Hellaby Watkins
ORGANIZATIONS
University of Cambridge
TECHNOLOGIES
Q-learning, Reinforcement learning, Markov decision processes, Dynamic programming
Evidence & verification
STATE
verified
CONFIDENCE
high
LAST VERIFIED
2026-09-12T07:04:11+08:00
Watkins's own thesis page identifies 'Learning from Delayed Rewards' as his Cambridge 1989 PhD thesis and states that it describes reinforcement-learning algorithms including Q-learning. His first-person historical account states that he submitted the thesis in May 1989. The University of Cambridge repository independently identifies Watkins as author and the item as a thesis. May 1989 is therefore recorded only as the thesis-submission milestone; it is not asserted as a universal invention, release, or system-birth date. University of Cambridge is recorded as the thesis institution, not as inferred development geography. No ancestry, influence, causality, or relationship edge is inferred.
Sources
PhD Thesis: Learning from Delayed Rewards
Christopher J. C. H. Watkins
Reinforcement Learning: some history
Christopher J. C. H. Watkins
Learning from Delayed Rewards
University of Cambridge Apollo Repository
Trace the chronology
Record identity
LINEAiGE IDq-learning-1989