LINEAiGE
AI HISTORY

Learning from Delayed Rewards

Christopher John Cornish Hellaby Watkins submitted his University of Cambridge PhD thesis 'Learning from Delayed Rewards' in May 1989. Watkins's own account states that the thesis described a range of reinforcement-learning algorithms, including one-step Q-learning, and sketched a proof of its convergence.

May 1989AI HISTORYVERIFIED

Historical classification

This verified record belongs to LINEAiGE's artificial-intelligence history layer beginning in 1956.

Why it matters

A documented research milestone for Q-learning in the reinforcement-learning literature, represented here strictly as the May 1989 thesis-submission milestone.

People, organizations & technologies

PEOPLE
Christopher John Cornish Hellaby Watkins

ORGANIZATIONS
University of Cambridge

TECHNOLOGIES
Q-learning, Reinforcement learning, Markov decision processes, Dynamic programming

Evidence & verification

STATE
verified

CONFIDENCE
high

LAST VERIFIED
2026-09-12T07:04:11+08:00

Watkins's own thesis page identifies 'Learning from Delayed Rewards' as his Cambridge 1989 PhD thesis and states that it describes reinforcement-learning algorithms including Q-learning. His first-person historical account states that he submitted the thesis in May 1989. The University of Cambridge repository independently identifies Watkins as author and the item as a thesis. May 1989 is therefore recorded only as the thesis-submission milestone; it is not asserted as a universal invention, release, or system-birth date. University of Cambridge is recorded as the thesis institution, not as inferred development geography. No ancestry, influence, causality, or relationship edge is inferred.

Sources

SOURCE 1 · PRIMARY · CLAIM-EVIDENCE

PhD Thesis: Learning from Delayed Rewards

Christopher J. C. H. Watkins

SOURCE 2 · PRIMARY · CLAIM-EVIDENCE

Reinforcement Learning: some history

Christopher J. C. H. Watkins

SOURCE 3 · PRIMARY · CORROBORATION

Learning from Delayed Rewards

University of Cambridge Apollo Repository

Trace the chronology

Record identity

LINEAiGE ID
q-learning-1989