In his popsci book, "Particle at the end of the universe", Sean Carroll says that the LHC, due to it's sheer information gathering capability, necessarily needs to completely discard most of the data that it gathers.
I was wondering how this was done, separating the "interesting" events from the rest.
Carroll says that between hardware and software techniques, only several hundred events per second, out of the many millions produced per second, are kept on record for full analysis.
Is it a pattern matching algorithm that is used to analyse the data, in which case do the people involved, both theorists and experimentalists, use this as a guide to what "odd" events would look like?
I'm sure I am second guessing the combined years of experience of thousands of theorists and experimentalists here. Also I certainly do not have the background to ask the question in specific terms.
But in general, if an exotic event produces a signature that is really new to us, such as dark matter or a member of a fourth generation of particles, how would we be sure we have not missed it, if we don't know what to look for in the first place?
Of course, the answer could be, say due to the energy output of the LHC, we have limits as to what is possible and we do actually know what to expect in any particular range. I just wonder is there a chance that we will miss something as important as the Higgs?