The phrase "per unit $x$" is used in physical definitions because it's convenient and familiar (once you've done some physics stuff for a while). Density is the mass per unit volume, the electric field is the electric force per unit charge, the gravitational field is the gravitational force per unit mass, and so on. This doesn't mean you actually have a unit of $x$, it's just because in defining a ratio, having a $1$ (unity) in the denominator makes it easier.
Example: a solid block has a mass of 13kg, and a volume of 2.5L. The density is therefore 13kg/2.5L, but you would more likely say 5.2 kg/L (i.e. 5.2 kg/1 L), or 5.2 kg per unit volume.
For the electric field, we could put everything in terms of force, but it's immediately clear that because this force depends on charge, it's easier to work with force per charge $E=F/q$ at each point in space. Then when we're interested in the electric force on a charge $q_0$, we don't calculate $F$ directly, we calculate $E$, then use $F=qE$. Makes life easier. The fundamental reason is electric force depends linearly on charge, so it's better to divide the dependence on charge out when doing calculations.
Addendum: Here's how it makes calculations easier. The Coulomb force between two charges, $q_1,q_2$, separated by a distance $R$, is defined without reference to an electric field as:
$$
F = k\frac{q_1 q_2}{R}
$$
Therefore if $q_1$ is at position $(0,0,0)$, and $q_2$ is at $(x_0,y_0,z_0)$, then $R = \sqrt{x_0^2+y_0^2+z_0^2}$ and
$$
F = k \frac{q_1 q_2}{\sqrt{x_0^2+y_0^2+z_0^2}}
$$
Whatever $q_2$ has for position and charge, we can always say
$$
E(x,y,z) = k\frac{q_1}{\sqrt{x^2+y^2+z^2}}
$$
Which is called "the electric field of charge $q_1$", or maybe more exactly, the electric field contribution of $q_1$. Now we know that if $q_2$ is located at $(x_0,y_0,z_0)$, we first calculate $E$ at this location, then find the force by multiplying the field by the charge of $q_2$.
$$
\begin{align}
E(x_0,y_0,z_0) = E_0\\
F = q_2 E_0
\end{align}
$$
This is the same equation as above. What effect would $q_1$ have on a different charge in a different location? Simply calculate $E$ and multiply by the charge. If you have a system of 20 charges, sum up the electric field contributions of each, then you get the total electric field; calculate $E$ at some location, and you know what the force would be for a charge $q$.
