# Doubts regarding dimension of a system:Definitions and algorithms

I need to do phase reconstruction from time series data. In doing so, I encountered Takens' embedding theorem and Cao's minimum embedding dimension $d$ by nearest neighbor method. In paper "Optimal Embeddings of Chaotic Attractors from Topological Considerations" by Liebert et al., 1991, says that minimum embedding dimension for reconstruction should be less than $2m+1$. This confused me since I am aware of Whitney's embedding dimension which stated $d=2*m$ where $m$ is the fractal dimension. Then there is Kennel's method of false nearest neighbor. Can somebody please explain to me:

1. What are the techniques for calculating embedding dimension?
2. Difference between embedding dimension and correlation dimension?
3. What is the technique of proving that a system has finite dimensionality after the signal passes through a filter?
4. Can somebody tell me what is the formula for embedding dimension
• should it be Cao's method or Kennel's false nearest neighbor method.
-
Whitney embedding applies to fractal dimension? Can you give a reference please? That's really surprising and extremely interesting to me if true. –  WetSavannaAnimal aka Rod Vance Sep 12 '13 at 4:54
Well,if you would pardon my ignorance,I am not completely sure about this. In this link scholarpedia.org/article/Attractor_reconstruction it says 2n+1 whereas in this link it says 2n for Whitneys theorem.So,what is actually the embedding dimension,fractal and correlation dimension? –  SKM Sep 12 '13 at 21:35
Ahh got it! There's no problem. I thought when you said "fractal dimension" you meant the generalized Haussdorff of a fractal set, whereas you're talking about the integer dimension of a manifold in phase space. Naturally I nearly fell off my chair, because the Whitney theorem is proven for differentiable manifolds, so what I was thinking you were saying didn't make sense! I'm not sure I can help with your question though. –  WetSavannaAnimal aka Rod Vance Sep 12 '13 at 21:57
... and now that I understand your question fully I can upvote it; but this isn't my field so that I am as mystified as you are. The answer must to be that the embedding sought has other properties as well as simply a smooth topological embedding, so that the Whitney embedding, whilst a topological embedding, falls short in some way related to the problem, perhaps by being an "optimal embedding" (I of course don't know what that means) as well as a Whitney one. I guess I'm stating the obvious here ... just thinking aloud. –  WetSavannaAnimal aka Rod Vance Sep 13 '13 at 0:42
In general, when a chaotic signal is embedded for reconstruction what is the formula and embedding dimension used? I came across Cao's nearest neighbor method but I still could not find whether to choose 2n or 2n+1. What is a manifold? –  SKM Sep 13 '13 at 5:52

In the following I assume that some basic mathematical requirements are fulfilled, like continuous and differentiable observables.

When you try to embed a system, you use $m$ observables $x_1, …, x_m$ trying to represent the dynamical state of the system. $m$ is called embedding dimension and if it is sufficiently high (and you have used appropriate observables), your representation fully captures the system’s state (e.g., exactly knowing $x(t)$ suffices to predict $x(τ)$ for any $τ>t$) in a continuous manner (e.g., $x(τ)$ can be continously mapped to $x(t)$). In this case, the representation is an embedding. E.g., the right part of this figure is an embedding of a sine wave using position and velocity as observables, and this figure is an embedding of a Lorenz system with $m=3$. Takens, Cao, Liebert and Kennel and others now tried to answer the question of what are appropriate observables and what is a sufficiently high $m$ for a system of which you only have a time series.

The strong Whitney embedding theorem (applied to this problem) basically says that if your system’s dynamics is governed by $d$ differential equations, there exists an embedding with $m=2d$. This is not very helpful, since we neither know $d$ nor how to embed, i.e., which observables to use. The weak Whitney embedding theorem states, that almost every representation with $m≥2d+1$ is an embedding. This is better, but you still need $m$ independent observables ($m$ dependent observables are one of the reasons why it is only “almost all”). Takens now found, that representing the phase space using only one observable at $m$ different times is almost always an embedding if $m≥2d+1$. And this finally is helpful, apart from the fact that we do not know $d$. Also representations with $m<2d$ may be an embedding, e.g., you need $d=2$ to generate a time series $\sin(t) + \sin(αt)$ with $α∈ℝ\backslashℚ$ and the corresponding phase space (a torus) can be embedded with $m=3$.

So there exists a smallest dimension for which there is an embedding and this optimal embedding dimension is $2d+1$ or smaller. We now want to use an embedding that is equal or not much larger than this optimal embedding dimension, since an overly high embedding dimension makes analysis more difficult. We would not need to perform time-series analysis, if we already knew the system sufficiently well to tell its optimal embedding dimension, so we can only estimate it with several techniques that you already named, which all have their advantages and disadvantages. So there cannot be a definite answer to questions 1 and 4.

As for question 2, there are several fractal dimensions that may be used to characterise systems, e.g., the Hausdorff dimension, the box-counting dimension or the correlation dimension. Their values are $d$ (and thus integer) for linear systems, non-integer for most non-linear systems and infinity (or not properly defined) for stochastic systems. Also their values are usually close to each other for one system. The corresponding concepts are so entirely different from the embedding dimension that it’s hard to name differences. At least for the pendulum, it is easy to see, that $d=1$ and $m=2$ is the optimal embedding dimension.

Question 3: For every aspect of your dynamics, there is a filter that may destroy the corresponding information in a time series. E.g., a low-pass filter may destroy the effects high-frequency dynamical noise the same way as it destroys high-frequency observational noise, and thus make a stochastic (i.e., infinite-dimensional) system look finite-dimensional. So, the best you can do is select an appropriate filter that should not destroy anything and argue that it does. Even then, you cannot distinguish between high finite dimensionality and infinite dimensionality. So only, if you find low finite dimensionality, you can make such a statement. There are several characteristics that may be useful and all should be compared to the respective results for appropriate time-series surrogates and should not depend on $m$. Anyway, if you do not want to calculate the dimension, but only show that it is finite, it suffices to show that the system is not stochastic (what I said about filters and surrogates, still applies).

-