As I understand it, when physicists talk about something behaving both like a particle and a wave, what they mean is that it has momentum like a particle, but its position is determined probabilistically by a wave function.
That's not quite accurate. It would be better to say that it interacts like a particle but propagates like a wave. In particular, while a quantum object is not interacting with anything, we represent it by a wave (a.k.a. wavefunction). The way in which the wavefunction changes over time is well described by the Schroedinger equation. Given a wavefunction, you can probabilistically determine a position and also probabilistically determine a momentum.
However, when the object interacts with a classical measuring device (pretending for the moment that such a thing exists), that interaction occurs at a single point. The position at which it occurs is distributed according to the probability distribution you can extract from the wavefunction that the particle had prior to the interaction. This makes interaction a rather odd event, because immediately before the interaction, the object had a spread-out wavefunction, but afterwards, it has a wavefunction completely localized to a single point. The Schroedinger equation cannot account for that type of change in the wavefunction, so in this model, something else has to be going on. Whatever it is, we call it collapse of the wavefunction.
Admittedly, the presence of some mysterious process that isn't described by a known evolution equation is disconcerting. What it is generally taken to mean in reality is that the model which predicts this wavefunction collapse is insufficient. Many scientists expect that taking into account that the measuring device is a quantum object itself will go a long way toward resolving this problem. If you're interested, much of the work being done in this area falls under quantum decoherence.