Can any number of particles appear anywhere? The probabilistic nature of particles tells us that they can exist at more than one place at once, with various probabilities. Is it correct to assume that then, if I hold an apple in my hand, suddenly the whole apple could appear a mile from me, with an extremely low, but nevertheless non-zero probability?
 A: It is correct but profoundly uninformative and outside the domain of conventional physics.
You are probably aware that, in statistical mechanics, there is a fantastically small probability that the randomized thermal motions of all the air molecules in a room find themselves, at a given moment,  in a small volume in that room, let's say in the shape of an apple, to make it more bizarre and vacuously dramatic. This is in a perfectly classical theory, statistical mechanics, whose foundations are probabilistic. 
Quantum mechanics is also probabilistic, but even more bizarre. Indeed, an electron in most systems would have a wavefunction with infinite tails, extending to a mile. So a measurement a mile away could detect that electron with a negligibly small probability. "Negligible" is established 
code for "no sane physicist worrying about it", although an absolutist philosopher might.
An apple has about $10^{25}$ atoms, interacting in interesting ways to stay together, but, to the extent you want to philosophize about the wavefunction of an apple, yes, you'd crudely multiply the tails of the  $10^{25}$-fold tensor product of these atomic wavefunctions and estimate a negligible probability for the apple to be observed a mile from here, and very suddenly, at that. Non-vanishing, to falsely reassure you. 
Crucially, wavefunction effects of routine macroscopic objects are aggressively pointless, unless these are highly specialized coherent/condensed  macroscopic states, "Schroedinger kittens", etc... I should hope you are not asking about special macroscopic quantum effects of that ilk.
This is at the core of probabilistic physics theories, and the mere flag of "non-vanishing, however infinitesimally small" is an invitation to physicists to shrug and walk away, and leave the matter to philosophers and theologians.
A: The apple is a bound state ( sequentially like an onion) of molecules , atoms, electrons and nuclei, protons and neutrons in the nuclei, quarks antiquarks and gluons within each proton and neutron in each nucleus.
In principle it has a quantum mechanical wave function describing the probability of its existence in your hand, i.e an (x,y,z,t) which is to all intents and purposes 1, as the number of particles composing it is of order the Avogadro number ~$10^{23}$. In the quantum mechanical formalism this is described by a matrix, called density matrix, where the rows and columns are all the ~$10^{23}$ wavefunctions and the off diagonal elements are the quantum mechanical phases between all those wavefunctions. In the case of the apple all the off diagonal elements can be considered to be zero except those that are very close in space to each other, less than nanometer, which means the classical position of the total apple.
The concept of "they can exist at more than one place at once" comes from solutions of the barrier problem , and it is called tunneling.

This particle does have a quantum mechanical probability of being outside the barrier, and the process is important in studying nuclear physics processes. 
Note the small dimensions of nuclear physics ( fermis) , that contrast with the large dimensions of an apple.
In physics zero probability is a number approached as a limit, as Zachos says in his comment, as physics is about observations and measurements.
