The title says it - I've encountered these terms several times but have never found an explanation anywhere. An example of use is this ATLAS note.
If I may hazard a guess: Data rate is high at the LHC so only a fraction can be analysed in real time (online) while some is stored for later analysis (offline).
Online data analysis is cursory analysis done as the data is collected. It is often used for the purpose of selecting which events to save to disk or tape to be analyzed later (an event "filter"). Given that the current CERN experiments will be taking, in the next run, data at rates exceeding a terabyte per second, this notion is essential.
In fact, the online analysis stream is performed in sever steps, each discarding a high percentage of the event data. The trigger itself can be considered a hardware implementation of the level-0 event selection. CDF and D0 had three levels of online (plus the trigger); the third level also assigned events to data streams, so that for example potential top candidates were one stream.
The offline analysis is done afterward, on the stored data The offline analysis is done on "farms" of computers, which in the days of CDF and D0 were mainly in the Feynmann Center at Fermilab, but today are tens of thousands of CPUs distributed across universities and institutions around the world. The term "offline" is generally reserved for the process of event reconstruction, where the raw data from the detectors is processed to determine what happened in the event, e.g., "19 electrons came out at from the primary vertex with these momenta; there are secondary vertices at this point and that point, a hadron 'jet' of this energy and momentum came from this secondary vertex..." and so forth. This offline reconstruction is in principle done once and for all for each event; in practices, reconstruction is run a couple of times as techniques are refined based on experience with the events.
The results of event reconstruction are saved as what in Fermilab days were known as DSTs (data summary tapes) and are greatly compactified relative to the original data. Nowadays this summary data is kept on disk, distributed around the world. Then there are many projects of offline analysis (using the DSTs) coded or designed by many experimenters, to extract the actual physics from those summarized events.
The term "online" typically refers to event reconstruction, and "offline" to analysis of the reconstructed events. The reason they are called "online" and "offline" is that online reconstruction is performed in real time as data is being collected, and "offline" is performed later using resources (computing banks, storage, etc.) that are not directly tied to data collection.
I'm not sure I'd use the term "online analysis", because it isn't really analysis. "Online processing" is more accurate.
To understand online processing better, it's important to realize that a high energy physics detector is not actually one detector; it is in fact a loose federation of many subdetectors that function somewhat independently. Each subdetector has its own unique processing and storage capability that collects and temporarily holds data from each event. There are online systems (triggers, filters) that examine data from each of the subdetectors and decide whether the event is real and sufficiently interesting to keep. Because there is so much data and so little time, this is done in several stages. The first stage is typically implemented in hardware (for speed); it looks at output from a few of the subdetectors - including, in some cases, detectors that are special-built for just this purpose - and decides whether the output from all the subdetectors should be read out or whether it should be discarded. If the online processing makes it past the first stage, it moves on to the second, where data from more of the subdetectors is processed and a similar keep-or-toss decision is made. As it moves through the stages, more processing is performed, data from different subdetectors is merged and compared, and similar decisions are made. Once it passes the final filter, the final stage of event reconstruction is performed where all the (pre-processed) data from all the subdetectors undergoes final online processing and is merged into a single "event", which is then stored for offline processing.
Similar processing can be (and often is) performed offline using stored "raw" data; this is usually done to incorporate improvements in calibration and to reflect a better understanding of the subdetectors' operation. Again, I'd call this "offline processing" rather than analysis.
Offline analysis is the process where physicists examine the event data -- or, more accurately, sift through millions of events and select the ones that meet certain criteria -- and try to figure out what caused the event. This is where the interesting physics results come from, such as discovery of the Higgs, or precise measurements of certain branching ratios, or limits on supersymmetic parameters, or any of the other myriad results published in physics journals.
To give you another example, from more medium-energy physics, I collaborated on an experiment where the construction of the "data summary" that dmckee describes took something like eight times longer than actually collecting the data. We kept up by analyzing the data in parallel on ten computers, but an hour's worth of data from the morning wouldn't really be ready to dig into until late afternoon. During data collection, our "online analysis" ran the same algorithms as the full analysis algorithm, but on two- or five-minute data segments, so that we could tell promptly whether things were going okay or whether there was some problem that might require us to perhaps stop the experiment and make a repair.
After the data collection was completed we knew things that we hadn't known when the experiment started, and our analysis toolchain was different and smarter as a result. That was the beginning of the real "offline analysis."
