Tosfos (114a) writes that the matzah and maror must be on the table while haggadah is recited to fulfill the halacha of "lechem oni - lechem sh'onim alav devarim harbei", bread over which things are recited. One can understand this halacha in two ways: 1) it is a din in sippur yetziyas mitzrayim that it be recited using the "props" of matzah and maror; 2) it is a din in the shem matzah that its status is dependent on the haggadah being recited over it. (By way of analogy, if you hold kiddush b'makom seudah is a din in seudas shabbos, the words of kiddush serve of define the meal as a shabbos meal. Here, the words of haggadah serve to define the matzah as "lechem oni").
Rashi (daf 36) writes that lechem oni refers to the fact that "gomreim alav es ha'hallel v'omrim alav haggadah". It is interesting that Rashi puts hallel before haggadah, reversing the chronological order. I would suggest that Rashi is based on a diyuk in the words "devarim HARBEI". The chiddush, which Rashi highlights by placing it first, is that not only haggadah, but even hallal is recited over matzah. In any case, one can understand Rashi in one of two ways as well: 1) hallel is part of sippur yetziyas mitzrayim (which the Rambam makes explicit in Sefer HaMitzvos) and therefore must be done using the "prop" of matzah; 2) for matzah to be defined as "lechem oni" it must have hallel recited over it. Based on this, it would seem that even after the seudah is over, one should have a piece of matzah on the table for the duration of Hallel of one wishes to be yotzei this chiddush of Rashi.