Is it really a particle? Forgive the stupid question but when colliding particles together, how does one know that a particle is actually a new form of sub-atomic matter and not simply just some shattered fragment of the larger particle?
I am not quantum theory literate, I do not pretend to understand more than abstract concepts and the math is beyond my comprehension.  But I have never heard anyone explain how they know they have discovered a new particle and not some shattered fragment of something larger.
If I throw a baseball at a glass window, I expect the window to shatter into smaller pieces of glass. Its still glass. Why doesn't this occur at a sub-atomic level too?  Granted particles are too small to observe behavior in this manner but how does one know that some split nucleus of an atom is nothing more than a shattered fragment of the larger whole which in itself might be shattered further still?
 A: On and below the atomic level, matter doesn't behave in the same divisible way as macroscopic matter, but shows signs of indivisibility not visible at larger scales.
This indivisibility is of a quantum nature, and has a basis in the mathematics of group representations. All moving pieces of matter are described by a Hilbert space carrying a unitary representation of the Poincare group, the local symmetry group of the universe. Splitting a junk of such a piece amounts to splitting the representation into several representations. 
But this can be done only up to a point, as there are irreducible representations that cannot be split. These behave like the prime numbers in number theory, which cannot be further factored into smaller whole numbers. 
Therefore we can tell mathematically whether a particle is elementary - it is if it is described by an irreducible unitary representation of the Poincare group. These representations were classified in 1939 by Wigner, and those of them consistent with the observed physical laws (in particular causality) are characterized by mass (nonnegative) and spin (hafintegral). These therefore characterize an elementary particle. (Because of internal symmetry groups there may be additional quantum numbers such as charges that specify an irreducible representation of the full symmetry group.)
The only way an elementary particle can split is by gaining from the interactions with surrounding particles enough energy and momentum (and degrees of freedom) to produce several other irreducible representations - then it ''decays'' into the corresponding particles. But these are created in the decay process, and were not ''part'' of the original particle.
Sometimes one finds in this way new particles, living for a very short time before they decay again. That the particle is new can be deduced from its observed quantum numbers (mass, spin, and charges) - it is new if these differ significantly from those of the known particles.
A: Every particle has certain properties (e.g. mass, spin, charge, etc) and behaviors attributed to it.  In a particle collider, you can measure properties of the particles it detects.  The idea is to determine (statistically) if the properties you observe are consistent with the properties you know (or expect) from each particle.  This isn't easy, and that's a big part of why detectors are so complicated (and expensive), because they have to be able to measure properties very accurately to determine---with confidence---what actually happened.  Additionally, this is why particle experiments require lots of results.  Just because one event does or doesn't seem to match well, that doesn't mean there's anything strange going on.  You need to see lots of example to say, with confidence, that you know what's going on.
Consider an analogy:  Someone has a collection of sports balls (baseballs, basketballs, softballs, footballs, bowling balls, etc), that they are pitching to a batter.  You want to know what ball the batter hits, based on certain pieces of information (lets say you can't actually see what the batter hits).  If you hear a loud crack, you can say it probably wasn't a tennis ball.  If the ball flies out of the park, you can probably say it wasn't a bowling ball.  If the batter hits lots of balls out of the park, but a certain set of balls seem to always land within 20 feet of him, you can say that those are probably not baseballs --- but they are consistent with what you would imagine for basketballs, or footballs or something.  Finally, if unexpected happens -- like he hits a ball, but it gets stuck-to the bat --- you might think there is a ball in there you're not familiar with: a new type of ball.  But you would also want that type of event to happen a number of times so that you're confident it's not a fluke.
A: The area of smashing particles is surrounded by detectors. Whatever is captured by detectors is a particle (I would say "by definition" for simplicity). It causes some reaction inside detector. For instance, it can be a trace of bubbles in Wilson camera. The shape and other parameters of this trace depend of the type of particle. It has been known from numerous experiments that there is only a limited number of trace types, and hence there is only limited number of particles that can be detected directly (i.e. each type of trace is associated with some particle type, like electron or proton. All electrons result in similar shapes of trace when crossing the Wilson camera). So, the first "class" of particles comprise of those instances that can be detected directly. Let's call the "real" particles.
There is also another "class" of particles that cannot be detected directly. Physicists think that they are particles too for two reasons:


*

*Physicists developed a model that explains why this particle decays into certain combination of "real" particles that can be detected directly.

*Quantitative predictions of this model are consistent with experimental data.


Examples of such particles are W and Z bosons, as well as number of so called "resonances" (presumably short-living particles) etc. And also Higgs boson of course.
In fact, collider is a kind of a black box: there are "real" particles on input and output, but we can only guess what happens inside. "Guessing" process is explained in principle by @zhermes.
This, in turn, brings us to another question:
Why some particles cannot be detected directly?
There might be a number of reasons for that:
a) our detectors are not good enough
b) particles are too short-living (which is in fact equivalent to reason (a))
c) for some "fundamental" reasons (usually quantum mechanical arguments are used)
Frankly speaking, I do not know for what reason, for instance, W and Z bozons cannot be detected directly.
And last (but not least): I admit that possibly those non-"real" particles are not in fact particles as per rigorous definition (field configurations that realise irreducible representations of Poincare group in Minkowsky space or something like that). I admit that they might be "shattered fragments" of "real"  particles.
But modern physics is unable to establish that. Because our models were developed based on collider-type experiments (i.e. "real" particles in and out of the black box). And all models are based on "black box" logic - quantization. Do you know what quantization is? it is a set of numbers connecting inputs and ouptuts of the black box. Vicious circle. That's why these models are useless for identifying the properties of the isolated and not moving "real" particles (mass, charge ets.).
A: Scientists did not know for sure whether materials were made of particles of specific sizes, or just chunk of arbitrary size like fragments of glass, until a little over 100 years ago. Chemists had determined through careful measurements that materials had chemical reactions in specific ratios, but that by itself did not prove atoms or molecules existed as we understand them today. It was only later that theoretical calculations showed that formulas based on the idea that matter is always composed of particles of specific sizes made very accurate predictions. Later experiments showed that charge and other properties always come in specific increments that match specific kinds of particles.
A: There is a technical definition of particle in the Standard Model [*]. Using mass, spin and charge we can characterize each one of the elementary particles, either known (e.g. photon) or theorized (e.g. graviton).
If you break your glass window into large enough pieces you obtain pieces of glass, but if you break again those fragments and again and again, there is a limit where the broken elements are not more glass but individual atoms with completely different properties. Your glass window is made of billions and billions of atoms and this is why you can break it into dozens of smaller pieces of glass (each one containing billions and billions of atoms), but one composite particle as the proton is made of only three quarks and you cannot break it into a dozen of smaller protons (because you cannot form groups of three quarks with only three quarks!).
[*] An elementary particle is defined in terms of irreducible representations of the Poincaré group.
