The Maharatz Chiyus has a long series of essays on the sevaros the Avudraham advances to explain why there are brachos on certain mitzvos but not others. The Avudraham quotes from other Rishonim that there is no bracha on the mitzvah of tzedaka because it remains in the hands of the recipient, not the giver, whether to accept the tzedakah and bring about the kiyum mitzvah. The potential to decline tzedaka distinguishes it from all other mitzvos.
The Maharatz Chiyus points out that this sevara depends on a recent sugya in daf yomi (Kesubos 40). The gemara questions why a rapist cannot marry an anusa if there is an issuer involved in the marriage – why not say aseh doche lo ta’aseh? The gemara answers that classically aseh doche lo ta’aseh applies where there is an unavoidable need to perform the aseh; in this case, if the anusa declines to marry her attacker there is no mitzvah on him to marry her. Rashi explains that we therefore encourage the women to decline marriage. However, the Rashba learns that the woman does not actually have to voice her decision to decline – the fact that the mitzvah potentially can be voided if she declines categorically distinguishes the mitzvah from other mitzvos.
The Avudraham’s approach parallels the Rashba – the potential of the ani to decline tzedaka categorically distinguishes the mitzvah of tzedaka. Whether any particular ani chooses to do so is irrevelant. According to Rashi, potential or categorical distinctions carry no weight – we look only at whether there is an actual mitzvah or not.