All We Know and All We Forget