# Clean lab data set

In a experiment I have a set of values for position an time (t,x) but due to the lab conditions the set is a little noise (you know, limitations, I cannot have infinite precision, thermal noise, the weather, etc). I have the curve x(t)

An I want to compute the velocity of my object, v(t). I have try with Euler algorithm and similar, also I try to bin the time and do the average of each point, and finally try to divide the time in bins and compute the velocity as the slope of the linear regression of each bin, but there is no results, it keep too noise and I want to make it smoother. There is some method that could make a discrete derivative of a set of points smoother?

Data:

1 -101.004
2 -96.023
3 -92.528
4 -89.689
5 -87.416
6 -84.887
7 -82.42
8 -77.672
9 -72.756
10 -67.63
11 -66.463
12 -63.083
13 -58.414
14 -55.957
15 -52.661
16 -50.451
17 -45.201
18 -41.955
19 -37.84
20 -34.205
21 -31.801
22 -29.501
23 -26.275
24 -21.017
25 -16.692
26 -14.426
27 -10.326
28 -4.4846
29 0.1209
30 4.5369
31 5.5959
32 5.8426
33 9.5106
34 13.6508
35 16.8758
36 20.4132
37 23.1078
38 24.7478
39 30.7428
40 36.7988
41 42.1008
42 45.9488
43 49.2898
44 52.2268
45 57.4548
46 61.2818
47 66.1938
48 69.8588
49 75.3448
50 77.9258
51 81.9008
52 88.8838
53 92.3288
54 94.1398
55 100.33
56 106.314
57 108.626
58 113.567
59 116.61
60 121.329
61 123.042
62 124.853
63 128.689
64 130.937
65 134.637
66 138.912
67 143.487
68 144.322
69 144.789
70 150.862
71 153.53
72 158.406
73 162.615
74 164.523
75 168.438
76 170.319
77 172.629
78 174.404
79 179.129
80 183.013
81 184.123
82 187.282
83 190.343
84 194.989
85 196.346
86 197.099
87 200.433
88 204.428
89 208.306
90 210.948
91 215.774
92 219.786
93 222.868
94 226.474
95 227.578
96 229.032
97 233.418
98 236.884
99 239.687
100 244.572
101 250.665
102 252.372
103 257.418
104 262.03
105 267.597
106 271.767
107 275.266
108 278.497
109 281.835
110 287.747
111 290.53
112 294.188
113 296.783
114 303.146
115 306.081
116 309.827
117 314.725
118 314.88
119 319.088
120 320.973
121 323.842
122 327.633
123 329.559
124 332.203
125 336.16
126 340.731
127 343.707

• Do you mean something like this? Commented Feb 22, 2021 at 13:21
• Also you may want to ask on cross-validated Commented Feb 22, 2021 at 13:22
• @jacob1729 Yes, I already try for higher orders but velocity keep too noise. Thank you Commented Feb 22, 2021 at 13:53
• How about a moving average? Calculate the distance between time 0 and time 20, then point 1 and point 21... the distance gives you a knob, low distance = high locality, high distance = high accuracy Commented Feb 22, 2021 at 15:56

I agree with user253751 comment. The method of "moving average" seems the best in your case.

However I would use a variant less sensitive to the scatter of data. From point 1 to point 20 proceed with linear regression for x(t)=a+bt . The local speed at point 10 is about v=b. Proceed on the same manner with the set of points 2 to 21 for the local speed of point 11. And so on up to the last set of points from 108 to 127 for the local speed at point 118. Draw the graph of v(t) from t=10 to 108. This will give a view of the fluctuations of speed.

One can do it with smaller or larger sets of points (for example 10 points or 30, etc. and compare the ranges of fluctations.

Of course the best method should use a "physical" function x(t)=f(a,b,c,... ; x) in which a,b,c,... are parameters to be adjusted by non-linear regression so that the curve f(t) be as close as possible to the data x(t). Then the first derivative gives a view of the speed as a function of t.

If the phenomena involved are not sufficiently known and if no physical model can be derived, one can try a worse method in chosing a "non-physical" function for example a polynomial : f(t)=a+bt+ct^2+...

RESULTS WITH THE DATA :

Case f(t)= a + b t giving the average speed on the whole range :

Case f(t)= a + b^t + c t^2 giving the general tendency of variation of speed (mean acceleration 2c) :

Example with higher degree polynomiql f(t)= a + b t + c t^2 + d t^3 + f t^4 + g t^5 + p t^6 + q t^7 + r t^8

This is suppoded to give an order of magitude of fluctuations of speed. But questionable due to calculus deviations especially if the number of adjusted parameters is large compare to the number of points and if the scatter of data is large compare to the real fluctuations of speed.

Possibly the method of moving average would give similar shapes of speed variation depending on the size of the moving set of points.

Blue curve : Above result using polynomial fit (degree = 8).

Black curve : "Moving average" method (Moving set of 20 points) .

Before deleting the code (Mathcad) used to compute and plot the last above curve I made this screen copy :

• Great! Both methods looks very agreement. In my opinion "Moving average" is a bit better just because this way I don't miss information about fluctuation and find the same result. Thank you Commented Feb 27, 2021 at 20:47

Linear regression will give you the line of best fit for your data, and calculating the correlation coefficient will tell you how well the data fit that line.

• Yes, but the problem is that I want to know the velocity more locally, and how it vary or keep constant. Maybe I just need more points in between to make it smoother Commented Feb 22, 2021 at 14:00
• @user239504 Well, you can divide the time line into intervals with, say, ten or twenty data points in each and then apply linear regression to each interval. But beware of introducing spurious precision - you need to have an idea of the error bars on your measurements before you can say whether a small variation in apparent velocity is real or just an artefact of your data. Commented Feb 22, 2021 at 14:09

You could try smoothing with splines, for example cubic splines. There are routines for that in Matlab etc.