I always use to wonder how the experimental physicists discover new particles every now and then whose dimensions/properties/mass/charge several order of magnitudes below that of anything that is visible/perceptible. So as engineers do, I guess they also set up a extremely complicated equipment and do some pretty complicated experiments and measure some physical quantity, say (pardon me if my example is poor) an voltage or a magnetic field intensity or could be anything depending on the experimental setup. Now they compare this measured voltage variation with that of what is expected theoretically and then go on to prove the hypothesis. ( This is the hardest thing I could imagine).

Now my request is where can i find a set of data (preferable a continuous variation of a physical parameter with respect to another... may be sampled at sufficient sampling frequency) along with the context of experiment (as minimal as possible but sufficient) so that i can carry out some processing of data in my own way so that i can verify the hypothesis or any such a thing. Simply put i need some really cool real world data ( in the form of signal).

Is any such thing available on the internet or where could I find one? Please suggest me something which involves signal processing.

  • $\begingroup$ Wikifying since there is no single correct answer to this question. $\endgroup$ – David Z Jan 29 '11 at 21:35

Update: April, 2016 The CMS collaboration has made about 300 terabytes of LHC data (and the tools and tutorials needed to understand it all) openly available. The data is reconstructed to the particle track level, so it is not actually raw, but there is still a lot to know to make good use of this data.

The raw data from modern particle physics experiments are many terabytes (even petabytes) in size, and quite complicated.

For collider experiments the detectors are compound, layered devices with three or more different technologies used by five or more distinct subsystems, plus ancillary monitoring of detector performance, temperature and humidity conditions in the experimental hall, data provided by the accelerator operating crew on the state of the beam, and on and on and on. There are ten of thousands of individual detector channels and hundreds of "slow" devices (like the thermoments, magnet currents, beam current monitors, etc...). All of this has been pre-filterd by the trigger hardware (and exactly what filtering was applied changes over time).

For neutrino experiments the data are detailed information about the charge detected from photo-tubes (some combination of total charge in a window, peak voltage, peak time, onset time, and/or digitized wave forms) for hundreds or thousands of PMTs. Plus environmental monitoring like that done by the collider people.

In both cases there scads of calibration data, changes in operating conditions throughout the data taking period, and sometimes replacement of re-tuning of sub-systems part way through.

There is typically many tens of thousands of lines of custom computer code for opening and processing the data files. Code written by physicists. Now, particle physicists are a little more professional about coding then some of their peers, but that does not mean state of the art process and beautiful code.

It generally takes many thousands of grad-student and post-doc hours to reduce this to something physics can be extracted from.

There is a reason we call this "Big Science".

That said, you generally can get the data. Eventually. (Each collaboration will hold theirs for a while to insure they get to publish first.)

How do you get it? Just ask.

But you'll have to provide your own storage (and possibly copying hardware); come to where the data are kept; understand that the documentation will be scattered over hundreds or thousands of internal (to the collaboration) documents written as they went along by diverse authors some of whom have English as a second or third language (and may evidence some idiosyncrasies); and that help interpreting all this will be terse as these people have moved on and have other projects keeping them busy. And you may have to convince the people with the data that you have the capacity to manage it.

The availability of partly processed data sets is not something I am as sure about, but you could try asking for that too. The worst that can happen is you get told "No". But even if you can get this, don't image that it is easy to work with.

If I haven't dissuaded you, let me suggest a practical method for getting started. Go to the nearest university that has a nuclear or particle physics group, and ask to help out. Really. There is always a need for lab monkeys, and you will learn as you go along because you can't do the work if they don't teach you stuff.

In the process you'll

  1. Learn how some of the sub-systems work. Get a feel for what kind of raw data they return and how it is processed into less raw data. If you ask people will tell you how the less raw data can be transformed into still more physics-like information and eventually reconstructed into particles.
  2. Make some contacts in the business. Begin able to say "I work with Prof. Smith and Podunk U." is much better than "I'm interested." when it comes to getting access to data.

Dear Rajesh, you want the data announced by experimenters that were used to discover new particles. That's simple. Write the URL


where the seven crosses are random digits - the first one should better be 9 or 0 because the first two are a year. The third and fourth digit is the month, and the last three digits shouldn't be too high. You can play with it and find tens of thousands of papers of the kind that you described.

For example, one of them will be


which happens to be the paper on the observation of the top quark, the most recent elementary particle that was discovered in 1995.

I could also give you tens of thousands of similar papers and solutions to your problem in the field of condensed matter physics, chemistry, biology, geology, or dozens of other disciplines of science. You see the plethora of solutions so perhaps, your question was a little bit too vague? ;-)

  • $\begingroup$ Please suggest me something which involves signal processing. $\endgroup$ – Rajesh Dachiraju Jan 28 '11 at 17:23

At least in particle physics, you will not have access to this low-level information that you want unless you are part of the experimental group which is actually making the measurement. There are many reasons for that, some political other more physical. Let's comment quickly on both.

First of all, it is a lot of work to measure these particles and the collaborations want to use the data themselves before allowing anyone else to look at it. So, they won't show you the raw data before they extract everything they can. "Ok, show me data from old experiments", you may say. Now come another problem, the amount of data is simply overwhelming. It is not something you can record in a DVD and carry home or download from megaupload.

As for the more physical problem. You are specifically asking for very low level, raw data. To transform voltages into particles, there are lots of other assumptions involved. For instance, how the interaction between the particles and the detectors happens, how the signal on the detectors are processed and so on. Every experiment spends years, literally, understanding these interaction to develop softwares which allow the reconstruction of the signal and testing them to know whether the accuracy of this reconstruction is enough to measure whatever you want to measure properly. Well, even if you are given access to the raw data, who can guarantee that you will reconstruct it properly? The people who measured the data don't want any other person drawing wrong conclusions with their work. It's a kind of scientific responsibility. The only information you will really have access today is the high-level, already analysed data that can be found in any paper (see Lubos' answer above).

There are interesting proposals to make more information available publicly. One that you may want to look at is the Recast system. But something similar for all experiments is far from being reality.

  • $\begingroup$ There have been a few re-analysis from low level data. Once priority is assured people get more relaxed about access, though as you say they may demand you convince them you know what you're doing. $\endgroup$ – dmckee --- ex-moderator kitten Jan 28 '11 at 17:49
  • $\begingroup$ @dmckee : Thank you very much for the answers. It is pretty overwhelming to know such details. But my need is not that much. I obviously do not intend to carry out the whole processing which is done much much carefully by big groups and also it is impossible to do and to convince them that i could and assure that i know what i am doing and last but not least to even think of such a thing. All I need is a real world signal preferably from the physics world...so that i could study the mathematical nature of a real world signal...I do not intend to draw any physical conclusions from it. $\endgroup$ – Rajesh Dachiraju Jan 28 '11 at 18:12
  • $\begingroup$ @Rajesh: Define "signal". You could read some papers as LuboŇ° suggests to get an idea of how diverse and abstract that data are. As a particle physicist, when I read a paper on a new discovery I have to muse over the figures and text to figure out what they've done and why they think it is important. Every time. $\endgroup$ – dmckee --- ex-moderator kitten Jan 28 '11 at 18:27
  • $\begingroup$ @dmckee : Signal for example is a voltage variations with time at the end of microphone which captures the acoustic pressure variations in front of it. This signal is sampled and digitized using A/D convertor for storage and reproduction and analysis purposes. I work on processing human generated speech (acoustic) signals. I am curious to have a look and work with some signals from the physics world where i get to study the true nature of real world signals. Even as simple signal as speech signal is very complicated throws so many unexplainable things. $\endgroup$ – Rajesh Dachiraju Jan 28 '11 at 18:37
  • 1
    $\begingroup$ @Rajesh: Each category of detector has it's own signal characteristics. For PMTs its uni-modal, negative going charge pulses, for wire chambers its uni-modal, current pulses on collection wires, but bi-modal on induction wires (where these are used). Germanium and silicon detectors have another set of characteristics, etc. Go to your local university and ask a professor in nuclear or particle physics. They will likely have a few detectors of some kind in their lab. $\endgroup$ – dmckee --- ex-moderator kitten Jan 28 '11 at 18:52

You can download raw (or processed) data from various observational cosmology projects (e.g. COBE and WMAP) here:


As a project, you could use this to re-compute the famous angular power spectrum of the cosmic microwave background radiation.


Not the answer you're looking for? Browse other questions tagged or ask your own question.