Questions tagged [data-analysis]

Use for question relating the the techniques of data analysis.

Filter by
Sorted by
Tagged with
6 votes
5 answers
189 views

Is my uncertainty when fitting a curve to data from an oscilloscope too small?

I was trying to measure the capacitance of a (large) capacitor using discharge decay. I saved the data from an oscilloscope while taking measurements, and it recorded about 250,000 points. I noticed ...
0 votes
0 answers
32 views

How to "classify" signals?

I have some signals and I want to classify them into different types. Signals are all sine shape signals, some might have glitches. I know I could classify them base on frequency or amplitude, but is ...
  • 11
0 votes
0 answers
12 views

Real-time non-linear parameter fitting

I have this equation where $s$ and $\delta$ are unknowns and I have many measurements for each one of the other parameters: What would be a simple algorithm to find the $s$ and $\delta$ that fit the ...
  • 33
0 votes
1 answer
48 views

Error propagation question

Lets say I have built a rotating device. Now I want to measure how accurate the rotation is. For this I use a measuring instrument with a resolution of $0.1^\circ$ with an uncertainty of $\pm0.2^\...
  • 11
-1 votes
0 answers
74 views

Data analysis software for physicists students/researchers

During my undergrad in Physics, whenever we needed to perform some sort of data analysis, we were taught to use Excel (for basic linear regressions and simple uncertainty calculations, this was ...
2 votes
0 answers
69 views

Fitting a curve to my data [closed]

I got some data of the I-V characteristics of a solar cell (diode) in the dark and I have been trying to fit the data with the standard equation: $$ I = I_0(e^{qv/nkT} - 1) $$ The below table is my ...
0 votes
1 answer
37 views

How to discriminate the background and signal events in high energy physics (HEP) when they have a similar distribution on a variable?

In high energy physics (HEP), the Punzi method is usually used to optimization cut to discriminate signals and background events in some typical distributions, like PID variable, when Signal and ...
0 votes
0 answers
31 views

Path reconstruction from accelerometer and gyroscope data

I found a dataset here whose columns are attitude.roll, attitude.pitch, attitude.yaw gravity.x, gravity.y, gravity.z rotationRate.x, rotationRate.y, rotationRate.z acceleration.x, acceleration.y, ...
3 votes
2 answers
49 views

Asymmetric bar errors

Generally, when gaussian or normal distribution is assumed, we can calculate $\sigma$, the standard deviation of the population and $s$, the SD of the sample. When we have a non-symmetric probability ...
  • 5,765
0 votes
1 answer
59 views

When finding the refractive index of a glass block should the graph be forced through the origin or not? [closed]

Experiment: To measure the refractive index of a block of glass. A block of glass is placed on a sheet of paper and the outline is drawn. A point of incidence is chosen and the normal at the point of ...
  • 1,273
1 vote
1 answer
78 views

Evaluate action of $f(\frac{d}{dx})$ using the Fourier/Laplace transform

Initially I asked this question on mathoverflow. I however thought physicists may face this sort of problem more than mathematicians (I am an engineer). Due to that, I decided to ask here as well. ...
  • 93
-1 votes
2 answers
93 views

Is there a fundamental limit to the precision of calculations?

For daily applications, it is accepted that there are limitations to the precision of our calculations when we want to compute quantities that are rational multiples of irrational numbers. Then ...
0 votes
0 answers
4 views

Global fit analysis of the Balitsky-Kovchegov equation

Ingredients: Suppose that I have the QCD BK equation $$\partial_y N(r,y) = \int d^2\vec{r}' \mathcal{K}(\vec{r},\vec{r}',|\vec{r}-\vec{r}'|;\{\lambda_i\}) \left[N(r',y) + N(|\vec{r}-\vec{r}'|,y) - N(...
  • 918
0 votes
1 answer
90 views

What are internal and external estimates of error?

I have n measurements for a quantity, and I need to calculate the internal and external estimates of error. I know what standard error is and how it is calculated, but have no clue regarding internal ...
2 votes
4 answers
194 views

Uncertainty formula for division

$$(A \pm a) \times (B \pm b)=AB\left(1 \pm \frac aA \pm \frac bB\right)$$ Here is the formula im struggling with, I'm just trying to apply this formula for division (i.e for speed, $\frac dt$, instead ...
  • 145
1 vote
2 answers
59 views

Free statistics program to evaluate measurement data? [duplicate]

Does anyone know of a good free statistics program where I can, for example, enter measured values and their respective uncertainty and the program will then spit out the variance weighted means for ...
0 votes
1 answer
41 views

Estimate "potential" from data [closed]

I have the following problem: I am given time-seriesdata, which I am told solves the equation of motion which corresponds to a Lagrangian of the form $L=T-V(x)$. Is it possible to infer $V(x)$ from ...
  • 1,404
0 votes
0 answers
31 views

Does the resolution of a timeseries affect the estimate of the power-spectrum?

I have been trying to estimate the power spectrum of a timeseries using fourier transform. I noticed that by downsampling my timeseries I get a different spectrum as shown below Here you can see ...
  • 460
3 votes
1 answer
77 views

Error Estimate For the Mass of the Observable Universe?

Recent interest in what is called "The Hubble Tension" brought to the fore potentially conflicting error estimates for $H_0$ as illustrated in be below diagram. I've been looking for any ...
  • 1,222
0 votes
3 answers
82 views

Calculation of the statistical error in measurements

I am not sure how to determine the statistical error in a measurement. As an example, let's assume that we measure a quantity $X$ $n$-times and get $x_1,\cdots,x_n$. The corrected sample standard ...
  • 91
0 votes
0 answers
24 views

How to interpret ion saturation current and floating potential time series in plasmas?

Let's say I have a Langmuir probe taking measurements of the floating potential and Ion saturation current inside of a plasma, which allows me to generate a time series of these quantities. There are ...
0 votes
0 answers
21 views

Rate Upper Limit Inference With New Background

I am trying to set an upper limit on a process in an experiment. Over a given time period I observe $N_1$ events, which are all likely to be from a background. The background events are a Poisson ...
1 vote
0 answers
24 views

How to obtain exoplanets transmission spectra from JWST's NIRISS data of WASP96?

I was having a look at JWST's NIRISS data of WASP96 (specifically at the x1dints fits file which should be already calibrated). From this, I would like to obtain a result in a fashion similar to the ...
  • 39
1 vote
2 answers
107 views

Minimum $\chi^2$ fit of experimental data in absence of fluctuations of one variable

I know how to make a minimum-chi square fit when I have two variables both affected by statistical error. But how do I perform the fit when one of them has no statistical error, just the uncertainty ...
4 votes
2 answers
698 views

Rules of significant figures, precision, and uncertainty

In the physical sciences (which are physics, chemistry, astronomy, materials science, etc.), we learned that the uncertainty is +/- the smallest unit (which is 1) of the last significant figure if the ...
0 votes
0 answers
35 views

Where can I get a good quality video (preferably slow motion / high frame rate) of Brownian motion particles for tracking their positions?

I am trying to analyse how good the Langevin equation fits actual experimental data by tracking the position of Brownian motion particles from video footage. However, I was unable to get my hands on a ...
0 votes
0 answers
68 views

Statistics/Data Analysis Book Recommendation

I looking for recommendations for textbooks on statistics and data analysis. I am a first-year physics student who just finished a module in data analysis/data science. Even though I did pretty well ...
0 votes
0 answers
36 views

Line Spread Function processing

I have measured signal of the x-ray detector through a slit chamber. I need to calculate MTF from it, somehow. I know, that MTF (|OTF|) is a Fourier Transform of the Line Spread Function. The question ...
3 votes
2 answers
98 views

Error calculation for a relative difference with only one standard deviation (STD)

I have a quantity that is calculated with two measurements, but I only have a tabulated standard deviation for one of the measurements. How would I calculate the error on the calculated quantity? The ...
0 votes
2 answers
74 views

Ambiguous propagation of uncertainty

Suppose I have a voltage versus current graph with various "peaks" in current at given voltages. For example, this is what is seen in a Franck Hertz experiment. I can measure the value of ...
  • 1,695
1 vote
2 answers
63 views

Different fitmodels for different intervals

A number of points were measured and plotted on an XY graph. I want to have the best fit line for this graph. The best fit model for the first part of the graph is described by some function $f_{1}\...
1 vote
2 answers
98 views

How to quantify the uncertainty of the time series average?

Usually, we measure a fixed physical quantity, then we simply average MEAN on the measurement results and calculate the standard deviation STD, then the measurement result can be written as MEAN±STD. ...
  • 11
2 votes
3 answers
90 views

Total standard deviation for a set of measurements

I have a doubt about the calculation of the standard deviation for a set of measurement. To explain better what I mean, I will illustrate what I am doing in my experiment. I am collecting the spectra ...
  • 21
1 vote
1 answer
48 views

What is the difference of two Chi-square formula, and how can I use them properly?

I found two formula for chi-square: $$\chi^2 = \sum_i ^N \frac{(\text{measured}_i - \text{expected}_i)^2}{\text{variance}}$$ and $$\chi^2 = \sum_i ^N \frac{(\text{measured}_i - \text{expected}_i)^2}{\...
  • 193
1 vote
0 answers
49 views

How the Zero-point Energy of the System containing 2 Fermions in 3 Micro-Energy States is 1?

If we distribute 2 Fermions $\mathrm{(A,A)}$ in 3 Micro-Energy States (0,$\epsilon$,$2\epsilon$), the confirmation is given by : $$ \begin{array}{|c|c|c|c|c|} \hline 0 & \varepsilon & 2 \...
  • 113
0 votes
2 answers
98 views

How to compute errors from a fit?

I have a set of data with errors, how do I compute the error for the fit $f(x)=a$? I remember there are formulas for the errors in parameters for fit $f(x)=ax+b$, i.e. $\delta a$ and $\delta b$. Where ...
0 votes
1 answer
36 views

Data analysis of ratios

I am analyzing some observational data which includes the ratio of masses (two masses for the same object, but measured using different methods). In principle, it is largely arbitrary whether one ...
0 votes
0 answers
19 views

How to correct for periodic errors in data set (worm motor)

I have a monochromator produced by Bentham which I am using for my physics experiment. This runs through the wavelengths in steps of what I thought were exactly 1nm. To calibrate the monochromator I ...
0 votes
4 answers
68 views

In a least squares fit, is the resulting error in the fit parameters considered statistical/random error or systematic error?

In a least squares fit, if there is no error estimate on the input data points, is the resulting error in the fit parameters $\sigma_y$ (which is calculated from residuals) considered statistical/...
1 vote
1 answer
329 views

Calculating Coherence Length and spectral width given an interferogram

I'm currently doing a lab in which we use a Michelson-Morley interferometer to analyse different light source. One of the mirrors in the interferometer is moved by a stepper motor. One of the light ...
0 votes
1 answer
45 views

Comparing the variance of the position of particles of a sample of matter with the sample's temperature

As the title suggests i have attempted comparing the variance of the position of particles of a sample of matter with the sample's temperature. Now firstly we need to clarify the basics: method for ...
31 votes
7 answers
3k views

Least squares fit versus averaging

Suppose I want to find the spring constant of a spring from measured $F$ and $x$ data points. There are two basic ways to do this. I could calculate the spring constant for each data point and ...
1 vote
2 answers
50 views

What does the inverse background efficiency represent?

I am reading a paper from the ATLAS experiment on the identification of tau jets from background jets and came across this figure: I am struggling to find what the formula is for the inverse ...
  • 161
3 votes
2 answers
124 views

Determining the appropriate number of significant figures to report in least squares analyses

I'm tasked with fitting a set of data to a line using Excel's $\text{LINEST}$ least-squares fitting routine. The data are $l$ and $m$ values, assumed to follow the relationship: $$l = l_0 + {g\over k}...
  • 969
0 votes
2 answers
96 views

Data science for physics recommendations

As a mathematical statistics student, I was wondering about learning data science application for physics, in general probabilistic modelling for physics. I don't mean that much engineering ...
0 votes
1 answer
30 views

Displacement distribution Molecular Dynamics

I have a file "traj.dat" with the trajectory of $N$ particles for $M$ frames separated by $\Delta M$ steps (with a time-step $\Delta t$). I often compute Mean Square Displacement (MSD) of ...
3 votes
0 answers
100 views

Utility of Topological Data Analysis in Theoretical Physics

I audited a lecture on Topological Data Analysis, and I found it really interesting, primarily because of the connection to algebraic topology. I asked the professor if there are any connections to (...
1 vote
3 answers
98 views

Validity of power law fit (and analytical fit choice)

Context I am carrying out molecular dynamics simulations of trapped ions in a radio-frequency field of amplitude $U_{RF}$ + DC (see details https://arxiv.org/abs/2102.04098). I am studying radio-...
0 votes
0 answers
34 views

Fitting time resolved data with a convolution

I want to fit experimental data of a time dependent Signal. The Signal is the result of a concentration convolved with a pulse. But the question is more in general, so I hope the picture below will ...
  • 317
0 votes
0 answers
22 views

Cross correlation for the fluctuating part of two timeseries. Can it attain values gretaer than unity

I have written a code to estimate the lagged cross correlation between the fluctuating parts of two quantities $(X, Y)$. To estimate the fluctuating parts I am using the following equations: $X^{'}(t) ...
  • 460

1
2 3 4 5
9