How is it made sure that something has been discovered, and not just noise? Is one discovery of something that is predicted considered to be enough (Higgs-particle)? What are the probabilities of a false positive in current experiments, like the dark matter experiments? How do they compare to the probability of a true positive?
This is something that particle physicists are perfectly well aware of. For any given observed effect, there is always a nonzero probability that the observation will be a false positive that was caused by a random fluctuation. The name of the game is taking enough data that this probability is small enough. In general, the more data you take, the less likely it is that statistical fluctuations will persist.
If a group wants to claim the discovery of a new particle, the golden standard is five sigma: the size of the effect must be five times its standard deviation away from what you'd expect if the particle didn't exist. If an experiment satisfies this condition, then there is a chance of 1 in 35 million that if the particle didn't exist, the data would look like do by sheer statistical fluke. If this probability is any bigger, the particle physics community holds off on making the discovery official until the probability crosses this threshold.
No, a single detection is not sufficient to conclude something new has been discovered. Enough events have to be observed to pass a statistical hypothesis test to reject the null hypothesis that what is being observed is indeed noise. As @EmilioPisanty stated in his answer, a "5 sigma" significance means that the observation is such that the chance of a false positive is equal to the area under the probability distribution at greater than 5 standard deviations from the mean, which is a very small probability.
Particle physicists have a very good idea of what the "noise" for a given experiment will look like in the detector, from experience with previous experiments. This "noise" consists, for instance, of signatures from other particle decay channels than those of interest that are caused in the course of the experiment, as well as various sources of electronic noise in the detector. A complex computer simulation of the detector and its response to noise and expected signal signatures, known as a Monte Carlo simulation, is a major part of the effort in any particle physics experiment.
The test of whether a discovery is made is not just a simple p-value test of the final mass or lifetime data of signal and background. Extracting a signal from the raw data is a multistage process, carried out with computer software- elementary particle physics is the original "Big Data" problem and had been since long before the term gained currency in business.
The conditional probabilities are taken into account during the reconstruction of the raw data from the detectors, incorporating both the characteristics of the detectors (probability of detection/false alarm, sources of systematic error) and the branching fractions of the many intermediate decay channels from the original particle to what is observed passing through the detectors. The original particle (Higgs etc.) decays into other particles, which in turn decay into other particles which decay into things like kaons, muons, protons which are long-lived enough to be seen passing through the detector.