This question goes for prehsitory and the Middle Ages as well.
From the emergence of modern man (the homo sapiens sapiens, following the homo sapiens neanderthalensis) we are sure people used spoken language. Probably, the Neaderthal also used something similar, probably more simple than what we know today. By the way, most Neanderthals, if you would put them in a modern suit and plant them in a city, would not even draw attention, that little difference there is between them and us.
So, language is already known by people for a long time. However, one travelled less then nowadays and the distances covered were shorter too. This means, dialects were much more developed, one could easilly hear if someone else was from your own village or not. But then again, the differences were not that large, that a merchant from, say, Northern Holland, could not make himself understandable in for example Denmark. During the Middle Ages, one often used a ‚in between language‘ or lingua franca. For clergymen and nobility, this often was Latin.
The ordinary folks – Iron Age or Middle Ages – will rarely have had to deal with strangers which language they could not understand.