In general, a coordinate system is not associated with any observer at all. Any observer can use any coordinate system, and the coordinate system doesn't have to be "rigid"; it can twist and turn and change in all kinds of ways. Whatever coordinate system we use is chosen based on convenience, not dictated by whatever observers are involved.
As highlighted in the OP, the relationship between a coordinate system and what the observer actually sees is only indirect at best. No matter what coordinate system we use, determining what a given observer sees requires doing some "ray-tracing" — that is, determining which lightlike geodesics connect the object-of-interest's worldline to the observer's worldline, and paying attention to the proper times along each of those two worldlines. And that's if the object is pointlike; if it's an extended object, then we need to consider each little piece of the object separately. Some special situations may admit special shortcuts, but in general, this is what we need to do. These calculations might be easier in some coordinate systems than others, but the important point is that it's the calculations — not the coordinate system — that determines what the observer sees.
When people talk about "the" coordinate system associated with a particular inertial observer in special relativity, they usually mean a coordinate system $t,x,y,z$ that has both of these two properties:
The given observer's worldline is described by $(x,y,z) = (0,0,0)$.
The proper time increment $d\tau$ is given by
$$
d\tau^2=dt^2-\frac{dx^2+dy^2+dz^2}{c^2}
\tag{1}
$$
along any timelike worldline — including arbitrary non-inertial worldlines.
By the way, if the observer is non-inertial, then no coordinate system has both of these properties.
In the coordinate system highlighted above, the given observer's proper time is simply $\tau=t$. However, there are lots of of different coordinate systems in which the given inertial observer is at $(x,y,z)=(0,0,0)$ and has proper time $\tau=t$. An observer is described by a single worldline, whereas a coordinate system can cover all of spacetime. What the coordinate system does along the given observer's worldline doesn't say much about what it does elsewhere. That's why when people talk about "the" coordinate system associated with a given inertial observer, the more restrictive condition (1) is usually implied.
The post
How do frames of reference work in general relativity, and are they described by coordinate systems?
offers more thoughts about this subject from the perspective of general relativity.