Take the 2-minute tour ×
Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It's 100% free, no registration required.

I used the package 'EUREQA', version Formulize, to analyse the monthly smoothed sunspot timeseries from 1750 till 2010.

It gives me a simple formula, with 8 coefficients, that match data with a correlation coefficient of 0.99214404.

In order to obtain a formula only dependent on past values, to ease the projection into some future, I used dS as a one month delayed difference and asked for a formula of the form: S = f(dS, dS^2 , dS^3).

The best formula obtained is:

   SunSpots = 27.36486757 + dS
    + 27.36486757*(sma(dS, 110))
    + 21.73018064*(sma(dS, 73))
    + 10.03456042*(sma(dS, 34))
    + 2.693356275*(sma((dS^2), 131))
    + 2.189454695*(sma((dS^2), 403))
    + (27.36486757*(sma(dS, 110))*(sma(dS, 110)) - 21.73018064)/(sma((dS^2), 131))

sma(var,length) is the simple moving average.

It reveals a cyclic dependence on 34,73,110,131 and 403 months and a cross between 110 and 131 months (responsable for a long cycle).

The correlation coefficient is quite high, and I expected that the formula could give me some values into the short future. In the last months of the timeseries the error became higher and the formula is quickly divergent in the future.

  1. What can cause such a discrepancy?
  2. What kind of physical model could give those components in the formula?

Any ideas ?

enter image description here

share|improve this question
4  
::boggle:: You have at least 16 parameters in your fit there, and have no physical reason for choosing them. That's a red flag to anyone who does fitting of complex data sets. –  dmckee Feb 5 '12 at 5:51
    
There's a fun quote attributed to John von Neumann on this topic: "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk." –  Richard Terrett Feb 5 '12 at 12:19
    
@dmckee: there are only 8 coefficients as reported by formulize, and R^2 Goodness of Fit = 0.33587211. In a model a simple feature associated with one length scale appears as two parameters: length (or area or volume) and time. The data for Maunder, long range, was ommited and it must be important too. The eq. is particularly simple and it was not my choice of any model. Looking at the shape of the cycles it does not call for chaos, at least to me. Overfitting? maybe not because there are almost always one or more daily observations and I used the month avg. –  Helder Velez Feb 5 '12 at 23:16
2  
Whatever the program is reporting each of the length parameters to the running average functions count, too. The "don't keep throwing parameters at it" rule is something that we all have to learn. You should only break it when you can defend each and every parameter. And by "defend" I mean on the basis of physics and not the data set you're fitting too. Is there some reason to think that the sunspot cycle depends on running averages? If so is there reason to think it depends on so many different time scales? If you can't answer yes to both of those questions then you are wasting your time. –  dmckee Feb 6 '12 at 1:51
add comment

3 Answers

You've got about 24 sunspot cycles in your raw data; and you've fitted a model with at least 16 coefficients. (there are eight figures that are given to ca. 10 s.f.; and there are eight figures for number of months being averaged. One might make the case of there being additional coefficients too: 1.0 for dS, and the exponents)

So of course your out-of-sample data is going to be well outside the model: all you've done is curve-fitted without any causal basis, and over-specified the model. Fast, large divergence is exactly what one would expect in the circumstances.

NASA has some helpful thoughts on how to predict sunspot cycles, including references to papers. They report that predicting in-cycle behaviour is "fairly reliable" once the cycle is at least three years in; they use the length and size of one cycle, and the amount of activity at the minimum, to estimate the next cycle. To this, they add data from measurements of changes in Earth's magnetic field, because these changes are caused by solar storms.

I'd start with NASA's modelling, because they have some skin in the game: In their words (from the above link:

"Planning for satellite orbits and space missions often require knowledge of solar activity levels years in advance."

share|improve this answer
    
8 coefficients as reported by the tool. When I injected those parameters in the tool it gives a simple formula to generate all of them (I dont have it now). The formula is underspecified because the Maunder was not included and it is known that are longer periods in the sunspot series. –  Helder Velez Feb 5 '12 at 23:21
1  
Your tool is under-counting. –  dmckee Feb 6 '12 at 1:52
    
@HelderVelez I've edited my answer to clarify the coefficient count; and added some relevant info from NASA on predicting sunspot cycles –  EnergyNumbers Feb 6 '12 at 7:24
add comment

It is proverbial that overfitting, by using an unparsimonious model, leads to poorer out-of-sample predictions than a more parsimonious model with a worse fit. Chatfield puts this less strongly: « Although statistics, like the AIC (Akaike's Information Criterion) and BIC (Bayesian Information Criterion), penalize more complex models, the reader should realize that there is still a danger that fitting many models to the same data may give a spuriously complex model that appears to give a good fit, but which nevertheless gives poor out-of-sample predictions.» The Analysis of Time Series, 6th ed., p. 265.

In the financial world, the mutual funds with the highest returns one year almost always perform more poorly than average the next year.

No one knows why, but this is why you should hire a professional instead of using a software package yourself.

share|improve this answer
add comment

My short answer to

1 - What can cause such a discrepancy ?

Deterministic chaos

2 - What kind of physical model could give that components in the formula?

Deterministic, in chaos theory, means that there exist dynamical equations that govern the system under study, which individually are absolutely predictive. Acting together on the system they induce a behavior that to start with seems chaotic in the everyday sense of random, but after studying the results one can find a generalized predictive behavior that cannot be relied on for predicting the outcome of the next time step in an analytic solution.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.