If we had just invented the first clock, and we only had a calendar system, how would we set the time of day for the first time? I've noticed there are extensive answers on this website about the accuracy of atomic clocks and how they reference the time between each other with the average of time between each other, but I realized: that doesn't answer the question of: assuming we do have the first clock in the world (atomic or otherwise!) and we must set the time of day for the first time: how would we do it?
Is there a specific astronomical observation we can collect to set the time of the current day and perhaps the rest of the date as well or is it all based on what we did back when the first clocks started synchronizing between them?
 A: There is significant evidence that our primitive ancestors put a lot of effort into determining the precise moment of sunrise on the day of the spring equinox.  One way to do this is to find the day of the winter solstice (when the sun appears to rise furthest north on the horizon) and the day of the summer solstice (when the sun appears furthest south on the horizon).  The day exactly between these two days is the spring equinox, when the sun rises from due east over the horizon.
You can determine the moment of sunrise precisely by, for example, making a small hole in a stone slab and aligning the hole so the sun will shine through it at the moment of sunrise and make light appear on a wall opposite the slab.  If you're very patient you can make small adjustments to the slab's alignment year after year until it's as precise as you can get it.
The moment of sunrise on the spring equinox is exactly six hours before noon.  So once you've done this measurement you have exact time of day and exact dates for a yearly calendar, as well as a precise indicator of due east.
After you've developed a civilization based on this knowledge, you can derive Newton's law of universal gravitation, which you can use to calculate the periods of orbit and rotation of the earth with great precision.  You can use these calculations to refine your measurement of the time of sunrise and noon on the spring equinox and adjust your clocks and compasses accordingly.
A: Atomic clocks don't set the time of day, they just count off seconds. The earth does not have a steady rotation speed relative to the accuracy of atomic clocks. So the time of day has drifted away from the count of atomic-seconds and the 86400-second definition of a day by about 37 seconds since the system was set at zero.
UT1 is the solar astronomical time of day with small adjustments for irregular motions of the earth, it is basically an angular measure converted to an equivalent time of day. The location of longitude zero was picked arbitrarily. "UT" is the raw solar time without adjustments for wobbles. The real physical measures used are of sidereal days converted to solar days because stars can be observed with higher repeatability and angular precision than the sun.(There is one more sidereal day per year than solar days because of the revolution around the sun being in the same direction as rotation.)
TAI is atomic clock seconds since some arbitrary starting point.
UTC is UT1 shifted a fraction of a second to synchronize with integer TAI seconds. When the difference between UT1 and UTC exceeds 0.9 seconds they add or subtract a leap second from UTC.
Timezones are defined only as UTC plus or minus some number of hours. Local time
There have been 37 net leap seconds since the start of January 1 1970 when UTC and TAI were set at zero. The atomic clock system is older than that but it doesn't matter because it is only acting as a metronome and counter.
GPSS time is very similar to TAI time in that they share the same definition of a second, but it must compensate for the apparent drift caused by relativistic time dilation. This is due to the satellites being further out of earth's gravity well and subject to the accelerative forces of orbital mechanics. Without compensation for this time dilation, GPSS locations would become inaccurate at a rate of about 6 miles(10km) per day. GPSS time is currently about halfway between TAI and UTC,  and it has a different epoch that started in 1980. They must also reset the internal date every 1024 weeks(about 20 years) because of legacy computing aspects of the system's design and the need for backward compatibility.
GMT(Greenwich mean time) has been antiquated for nearly a century and is grossly misused on top of that. GMT is 12 hours behind what was then GCT(Greenwich civil time), and GCT was replaced by UTC half a century ago. GMT dates changed at solar noon in Greenwich(for astronomy reasons). GCT, later UTC, dates change at solar midnight in Greenwich.
A: Ultimately it is just a convention. One such convention based on astronomical observations would be to choose noon - the moment when the sun is highest in the sky in some location - to be "time zero". This has been done historically, see e.g. https://en.wikipedia.org/wiki/Greenwich_Mean_Time.
This is complicated by the fact that the precise motion of the Earth around the Sun (and around its own axis) is complicated, so there isn't exactly 24 hours between noon on one day and noon on the next. However, once you have chosen "time zero" to be some point in history, e.g. based on the noon of a particular day in a particular location or some other reference point, you can of course just use your very precise atomic clock to keep track of time from there.
Edit: you may also be interested in leap seconds which are additional seconds added occasionally to our calendar to keep it in sync with the observed solar time.
https://en.wikipedia.org/wiki/Leap_second
A: Put a stick in the ground. Try to make it vertical I think. Watch the length of the shadow for many days and years. You’ll see that every day the shadow is long then short then long. There is a moment when it is shortest. You could just pick some day and the moment when the shadow is shortest and call that noon. If you want a little more significance to your $t_0$ you could wait for the shortest day of the year and call the shadow short time on that day noon.
If you’re using something other than earths rotation as a clock then you’ll find that that first day is the only day when the short shadow time happens at exactly noon. Otherwise you’ll see it drift around a bit and this is due to instability of either the earths rotation or your clock due to various physical effects.
If you trust earths rotation more you should recalibrate your clock occasionally so that noon does occur at the short shadow time. If you trust your clock more you should consider the difference between the short shadow time and noon on your clock to be a measurement of instability in earths rotation.
Note that earths rotation is kind of a funny clock because it’s not only random noise that makes the rotation not perfectly harmonic, but there are other periodic effects, such as revolution around the sun or various precessions, that cause periodic fluctuations in the earths rotation rate. But, there is also regular drift and noise familiar from other types of clocks.
edit: Note the procedure above allows you to set $t_0$ for your accurate/stable clock but it doesn't set the ticking period $T$. To set the ticking period $T$ to match Earth's rotation you would operator your clock for many days/years periodically tuning its oscillation frequency $T$ to match the rotation of the Earth. But again, note that this is kind of a funny process if your clock is actually more stable than the Earth's rotation because you will get different answers for calibrated clock tick rate depending on how long you wait. This is because the duration of a single day (i.e. noon to noon) varies over the course of the year so a more practical approach would be to let 1 day be the start of your calibration, then 365 days be the end. Make sure your clock ticks "noon" on the last day.  This means your clocks $T$ will be calibrated to the 365-day average of the length of a single day, at least during that 365 day period (because the average day length might vary from year to year).
So the takeaway is that we can use "astronomical" observations of noontime to set both $t_0$ and $T$ for our clock. But, as I've described above, if your clock is more stable than the rotation of the Earth (which it's not actually that hard to do) then you'll have to arbitrarily designate some noon to be $t_0$ and you'll have to arbitrarily designate some duration of days/years to be your calibration time during which you calibrate $T$. After that you take your clock as the stable reference and you interpret discrepancies between your clock and the Earth's rotation to be due to fluctuations in the Earth's rotation rate.
You can confirm that it is Earth drifting and not your clock by making multiple different clocks all with better stability than the Earth's rotation, and noticing that all of these clocks drift less relative to each other than the Earth does relative to any one of them. And it's the best if the different clocks are based off of different types of mechanisms to ensure there isn't some common mode drift to all your purportedly stable clocks that would trick you into thinking your clocks are more stable than they are (when in fact they drift, just at the same rate as all their peers).
