We measure the day length in the past by analysing astronomical records. We have good data for the last couple of centuries, but the uncertainty is larger for older time periods. Our estimate for the day length 1.4 billion years ago is rather crude.
To do this analysis, we need good clocks and good astronomical observations. People have been recording astronomical observations for thousands of years, but obviously records of observations made before the invention of the telescope don't have the same precision as more recent records.
We've had reasonably good clocks for several centuries. However, until the early 20th century, the rotation of the Earth itself was actually our best clock, and we set our mechanical clocks from astronomical time measurements. I have some information about that in my answer to How could Tycho Brahe determine positions without accurate clocks?
When we use this data to discover the day length in the past, various patterns arise. There are short-term variations and long-term variations. Some of these variations have simple patterns, which make them easy to predict. But some of them are chaotic and unpredictable, which is why we cannot make accurate predictions of leap seconds, and why our estimates of ancient day lengths are only approximate.
We make mathematical models of all the relevant physics, put those models into computers, and compare the computed results with the observed data. We adjust the computer models until they match the observed data within the limits of precision of those observations. Of course, if the observations are ancient, they may not be very precise, and may contain various types of errors, so we have to be careful.
The most useful astronomical data for determining the Earth's rotation in the past are records of eclipses, especially total solar eclipses. Astronomers of all cultures have recorded these events, going back to ancient times. The precise location of the path of totality on the Earth's surface is quite sensitive to the Earth's rotation angle. We can calculate the times of eclipses to high precision over a span of many thousands of years. If we know that a total eclipse was observed at some location that can help us know the Earth rotation angle on that date, even if the old record of the time of totality isn't very accurate.
Retired NASA astronomer Fred Espenak, aka "Mr Eclipse", has some excellent information on this topic. Please see Delta T and Universal Time. Fred has lots of great info about eclipses and photos on his MrEclipse site.
An important source of ancient astronomical data are the Babylonian Astronomical Diaries, which spanned seven centuries of astronomical and meteorological observations, starting around 650 BC. They formed the basis of the tables prepared by astronomers like Ptolemy and Hipparchus. There's more information about these diaries on Livius.org.
The modern organisation that monitors the Earth's rotation and orientation is The International Earth Rotation and Reference Systems Service (IERS), formerly the International Earth Rotation Service.
Measuring the irregularities of the Earth's rotation
The variability of the earth-rotation vector relative to the body of the planet or in inertial space is caused by the gravitational torque exerted by the Moon, Sun and planets, displacements of matter in different parts of the planet and other excitation mechanisms. The observed oscillations can be interpreted in terms of mantle elasticity, earth flattening, structure and properties of the core-mantle boundary, rheology of the core, underground water, oceanic variability, and atmospheric variability on time scales of weather or climate. The understanding of the coupling between the various layers of our planet is also a key aspect of this research.
The most important process governing the long-term slowing of the Earth's rotation is the tidal interaction between the Moon and the Earth. This transfers angular momentum from the Earth's rotation to the Moon's orbit, increasing the length of the day and the month, and increasing the Moon's mean orbital radius. The precise motion of the Moon is rather complex, but we have extremely good models of the Moon's motion in recent decades thanks to the ongoing Lunar Laser Ranging experiments.