I will go a bit into the math, then a bit into the physics. Definition (1) is not wrong, but its interpretation is different from the usual. Maybe it is simpler to think, equivalently, that $$\dot{W} = \eta (t); $$ but a Wiener process is not differentiable in any set of finite length (although the demonstration for this is not trivial), so it must not be interpreted in the same sense as usual.
However, it makes sense, as a formal definition, in its context. Remember the definitions of the Wiener process:
- $ W_0 = 0 $
- $ W(t) - W(s) \sim \mathcal{N}(0,\sigma ^2(t-s)), \forall t,s $
- $ W(t_{j+1}) - W(t_j), j = 0, 1, ..., n-1 $ are independent for any pair of times $ t_0 < t_1 < ... < t_n $ (i.e. independent increments).
- $ W_t $ is continuous in $ t $.
From the second definition you get that $ \langle W(t)^2 \rangle = \sigma ^2 t, \forall t.$ It is also possible to show that $ \langle W(t) W(s) \rangle = \sigma ^2 \min(t,s).$ (I may edit to put the proof).
From the third definition you get that $$ \lim_{h \rightarrow 0} \bigg\langle \frac{W(t+h) - W(t)}{h} \bigg\rangle = 0 .$$ This would be the "derivative" of the Wiener process, which according to (4) is continuous.
Now, consider the correlation $$ \phi_h(s) \equiv \bigg\langle \bigg( \frac{W(t+h) - W(t)}{h} \bigg) \bigg( \frac{W(s+h) - W(s)}{h} \bigg) \bigg\rangle . $$ Even though it looks a bit messy, realize that in the limit of small h, this would be the correlation between derivatives.
Using the previous properties, it is not hard to show that $$ \phi_h (s) = \frac{\sigma ^2}{h^2} [\min(t+h, s+h) + \min(t,s) - \min(t+h,s) - \min(t, s+h)]. $$ If you draw this, time s = t-h, t, t+h versus $ \phi_h (s) $, you will see that it is a triangle of height $\frac{\sigma ^2}{h^2}$. Taking the limit of small h, it becomes a Dirac delta representation, i.e.: $$ \lim_{h \rightarrow 0} \phi_h (s) = 0 $$ and $$ \lim_{h \rightarrow 0} \int \phi_h (s) ds = \sigma ^2 ;$$ and that's your definition of time correlation function, $ \lim_{h \rightarrow 0} \phi_h(s) = \langle \eta(t) \eta(s) \rangle = \sigma ^2 \delta(t-s)$.
Although, this is more like a plausible definition; you know that Dirac's deltas are properly defined as generalized functions and there are people here able to give a thousand more interesting details. Let's go with the physics, and why are this definitions useful:
In the explanation that Einstein gave to Brownian motion, he makes two interesting observations:
- On the one hand, as the Brownian motion depends on so many, chaotic, uncorrelated variables (e.g. impacts from a big number of molecules which move very fast through the fluid), the correct way to probabilistically describe them is through uncorrelated forces (that is, the relation through consecutive "hits" on the Brownian particle is, at least, really hard to describe).
- On the other hand, the mean squared displacement is the observable that best describes the movement of the particle (and not directly its velocity, which may not be observable).
If you take the Langevin equation, where the stochastic force appears, $$ m \frac{dv}{dt} = -\lambda v + \eta(t), $$ one does not aim to write down that force as a real function of time, but rather interprets it as white noise, as Kurt G. and Quillo commented. See that through the Wiener-Khinchin theorem one can write the spectral density of the stochastic force, $$ S(\nu) = \int_{-\infty}^{+ \infty} \langle \eta(t + \tau) \eta(t) \rangle \exp(-i\pi\tau\nu) = \sigma ^2 ,$$
i.e., it is constant for all frequencies, as white noise should be.
White noise is obviously unphysical, but it makes sense if you think that it is constant through a distribution of frequencies much larger than those which you can measure. But if you are asking yourself about the ultimate reality, I think that, if you could "zoom in" indefinitely into the path, you would find the correlations, and hence the velocity would become differentiable, as it is expected for a physical problem.
If this is the situation, why is writing down Langevin's equation useful at all? For doing this problems, one has to deal with the definition of Stochastic integrals (see Itô or Stratonovich), in which you use different chain rules and take mean values over short times. At some point, one has to take a discretization; the important thing is that this discretization is "fine enough" for the correct description of the observables of interest. As we pointed out at the start, the integral (1) is not the typical integral, but rather a stochastic integral. That is, the Wiener process is written down as the integral of a stochastic force because the stochastic term in Langevin's equation can be interpreted as coming from a Wiener process (rather than the other way around).
Thanks to M. Feito, whose PhD thesis introduction I read to get a bit of understanding of this really cool field while studying my degree, and who I have mostly followed here.