The Kli Yakar (riding on the coat tails of the Abarbanel) summarizes seven different answers given by the Rishonim that you can look up if interested, but I wanted to take a step back and make one a small observation about the problem itself. The entire premis that there is some "question" here that needs to be resolved is one that a historian would dismiss without a second thought. The problem is only a problem if you assume that the concept of reward / punishment in Tanach is consistant with the concept of reward / punishment accepted by the Rishonim, presumably based on Chazal. Otherwise, the answer is simple: In the good ol' Biblical days, there was no idea of afterlife. Reward and punishment happened to you in this world. Later, the Rabbis, either motivated by what they realized were irresolvable philosophical / theological problems with the Biblical account or by external influences on their thinking, came up with this idea of reward / punishment in an afterlife. Biblical Judaism and Rabbinic Judaism are separated by a difference of hundreds of years (not to mention differences in location and surrounding culture) -- is it any wonder that there exists theological differences between the two?
Baruch shekivanti, it took less that 10 seconds to find the following except ( link) from a very prominent Conservative Rabbi, one with a PhD after his name to boot, so his scholarly credentials trump mine:
The biblical references are all to divine recompense and retribution in this world, in terms of material prosperity and suffering here on earth. But a remarkable shift of emphasis took place, it is generally held, at the time of the Maccabees, when righteous men and women were being slaughtered because of their loyalty to their faith.Why not embrace this approach? After all, what could be better than not only solving a kashe that plagued the Rishonim, but even more than that -- showing that the kashe never existed in the first place? As to why the Rishonim didn't give that answer if it's so good -- maybe the Rishonim simply did not have the same historical sense of things that we do; perhaps they were more tolerant of anachronisms and the like; perhaps they saw history as a blur rather than a set of discrete time periods, each with its own flavor and character.
In the face of such direct contradiction to the notion of reward and punishment in the here and now, faith could only be maintained by affirming that recompense and retribution were to be the fate of humans in the Hereafter, in the World to Come, as it is called by the Rabbis. In the Rabbinic literature, while this‑worldly formulations are not unknown it is in the World to Come that the doctrine is made to receive its chief application.
I think this argument is mistaken. I think the Rishonim avoided saying halachos or hashkafos were historically conditioned not because they viewed history as one flat space into which everything melds as one, but rather because they viewed the totality of Torah as one flat space into which everything blurs and melds as one. I think the Rishonim held that the truths of Torah, at least and certainly with respect to ikkarei emunah, are akin to logical or mathematical axioms. 2+2=4 whether you live today or lived 2000 years ago or will live 200 years hence. If the concept of reward and punishment entails belief in an olam ha'ba, the necessity of that belief would have been true in the days of Tanach as much as in the days of the Tanaim and Amoraim.
This example is illustrative of a point I've written about here before, but is worth repeating because it au courant even in some Orthodox circles these days to take the opposing position. The tools of the historian are different than the tools of the halachist. Even where the questions they examine happen to overlap, the framework in which those questions are posed is radically different, the methodology of study is different, and the answers and the assumptions that underlie them are also often radically different.
A Brisker who might answer up a kashe of Tosfos with a gavra/chefzta split will undoubtedly defend his Brisker methodology as being rooted in the Rishonim themselves. It's just taking the use of structural categories, ideas the Rishonim use in many places, to the next level. The historian has no such similar precedent to call on. He cannot argue that his approach is an extension of an already existing methodology of Torah study that has been passed on as part of the mesorah. I think this is the reason the "yeshiva" world views with such suspicion the conclusions of the academic world in matters of hashkafa and halacha. It has less to do with the conclusions themselves than with the methodology, with the paradigm.
I'm not an either / or guy, so let me throw in a caveat. That is not to say we should avoid *all* critical or historical inquiry into Torah. We can be tocho achal and toss away the klipah. My point is that before doing so, one must first recognize the difference between the wheat and the chaff and know what to toss and what to keep. An answer to a halachic or hashkafic question that rings "true" to the historian may carry very little weight in the world of Torah, where other Truths reign supreme. It's those Truths alone that count.