This question rather requires answering by ethnologists. From archaeological view, this phenomenon can easiest be explained as the popular adaptation of earlier ‘”water cults”. From prehistory, we know numerous sacrifices from swamps, floods, streams, the sea, or at springs. In most cases, valuable objects were sacrificed to gods, up to the most valuable of all: human life. The use of throwing coins into water we know form the Roman Era; it was used to please flood and spring gods.
Nowadays, hardly anybody will know, when he has just thrown a coin with his right hand over the right shoulder, with his back to the fountain of the Roman Fontana di Trevi, why he or she did that. In the best case, the answer will be something like ‘it will bring luck’ or ‘I will return here one day’. A connaiseeur of Roman culture would at least know to add that one single coin would lead to a certain return to Rome, two coins that the one throwing would fall in love with someone from Rome and three coins would lead to a marriage to the person in question.
We keep to it: it is about the manipulation of luck in any kind of way and thois can be reached by the sacrifice of a certain material value (obulus), if one does it in the right manner and addresses it to the right divinity. With the introduction of Christianity such presentations were banned towards the realm of superstition, although many “heathen” practices remained in the form of from the Church Authorities harmlessly disguised folk traditions.