nidrei tzedaka - shibud mamon or not?

The gemara in Bava Kama36b tells the story of an individual owed ½ a zuz in damages who decides, in light of it being such a small amount, to donate the money to tzedaka in the presence of R’ Yosef. At some later point the individual changed his mind and asked for the money back from R’ Yosef. R’ Yosef said the gift was irrevocable - ‘kvar zachu bah ani’im’, the poor have already taken possession of the gift, because even if the poor are not present, ‘anan yad ani’im anan’, we, the court, serve as the agents of the poor and already hold the funds. The Geonim and Rishonim struggle mightily with a number of issues in the sugya, one of which relates to our previous discussion of nidrei tzedaka. Why does the gemara need to invoke the principle that R’ Yosef or the court acts as the agent of the poor to accept ownership on their behalf in order for the gift to be irrevocable - was the person who pledged the money not bound anyway to give it to the poor by virtue of his pledge, nidrei gavoha, or by virtue of amiraso l’gavoha k’mesiraso l’hedyot? The simplest answer is that nidrei tzedaka constitute a promise on the part of the giver to fulfill a pledge, but do not constitute a shibud or kinyan on the funds (Ba’al HaMaor, R’ Hai Gaon, cited by the RI”F). However, many Rishonim disagree. R’ Chananel (cited in Tosfos) writes that the individual who pledged the funds did not wish to retract, but simply wanted to borrow the funds temporarily for his own use; R’ Yosef, however, held that one collected by the gabai, the funds cannot be lent out. It is clear from R’ Chananel (and the Ramban in Milchamos) that the pledge to charity (the neder) did indeed create a shibud mammon, meaning the poor had ownership rights by virtue of the neder – the only reason the gemara invokes zechiya on the poor's behalf is to prevent the giver from even having temporary use of the money. The RI"F (and Tosfos explaining why amiraso l’gavoha does not apply here) also writes that although tzedaka creates shibudei mammon, here the money in question had not yet been paid to the individual making the pledge. Kinyanei hekdesh cannot apply to something that is not in one’s reshus or that is not yet ba l’olam. The Rambam cites this opinion in the name of the Geonim, but rejects it l’halacha. R’ Chaim Brisker (Hil Mechira ch 22) asks: even though the funds themselves are not in the hand of the individual making the pledge, there exists an obligation, a shibud, for those funds to be paid. According to many Rishonim, a shibud is treated like any other asset and theoretically can be bought and sold. If so, why is the obligation, the shibud, not considered something which can be transferred through amira l’gavoha or nidrei tzedaka to the reshus of hekdesh? I think I might leave this one hanging, or take a look at the R’ Chaim!