The labeling has perfect physical meaning.
Consider we have fixed positive charge which creates electric field $\vec E$. Electron in this field will be attracted to positive charge with force $\vec F = q\vec E$, where $q$ is electric charge (with sign!). From the latter we can see that force and field have opposite directions for electron. In particular, it follows that $\vec E$ points from positive to negative charge.
From the point of view of energy gain, the electron has lowest energy close to positive charge. The energy of the former is related to the electric field potential $\phi$ (integral of force) as $\epsilon = q\phi$ and is the lower the higher $\phi$ is. Coulomb force drags electron towards positive charge, i.e. in direction opposite to electric field $\vec E$. This is the physical illustration of the relation $\vec E = -\mathrm{grad}\phi$ (gradient of some scalar field is the vector pointing towards the direction of increasing field).
The density of current passing through a conductor cross-section can be written as $\vec j = qn\vec v$, where $q$ is charge, $n$ - density of charge carriers, $v$ - speed. The latter is linked to applied electric field as $\vec v = \mu \vec E$. Note that electrons move in the direction opposite to electric field, as was shown above. Here $\mu$ is carrier mobility and is defined as $\mu = q\tau/m^*$, where $\tau$ is an average scattering time and $m^*$ is "effective" mass of the carrier. Using the above one can finally arrive to the differential form of the Ohm's law: $\vec j = (q^2n\tau/m^*)\vec E$, where the quantity inside the brackets is electric conductivity or inverse resistivity.
This allows to answer the current "sign" question.
- Current and electric field directions always coincide, i.e. independent of carriers sign.
- For electrons the direction of current is inverse with respect to
electrons velocity. For holes (positively charges carriers) the
directions of current and velocity coincide.
Voltage drop between points 1 and 2 is the difference of electric field potential in this points, $V_{12} = \phi_1 - \phi_2$. It is positive for $\phi_1>\phi_2$, hence it is opposite in sign to electric field. So here is the final conclusion: current flows from highest potential to lowest.
As you can see, you do not need any conventions, and all can be figured out from mechanics. Hope this clarifies the confusion about signs.
Concerning charges, non-zero electric field means charge imbalance. It also means, there should be current as a reaction to remove this charge imbalance. When you are building energy diagram in this case, keep in mind that for positive charges the higher the field potential, the higher the energy, so that their energy diagram is inverted compared to that of electrons.
UPDATE:
The energy diagram difference for positive and negative charges follow from the energy definition $\epsilon = q\phi$. While an electron has lower energy near positive charge $Q$, a hole (positive charge) will have larger energy near positive charge $Q$, whereas potential $\phi$ of the field created by $Q$ stays the same.
This is explicitly illustrated in p-n diode (see picture). p-doped region has negatively charged ions, and n-doped region has positively charged ions.
Here the energy well for electrons looks as usual: electrons are accumulated in the energy well, since they indeed have lower energy near positively charged ions in the n-region and larger energy in p-doped region near negatively charged ions.
On the other hand, holes are accumulated near the the negatively charged ions in the p-region, where they have lower energy, and not in the n-region near positively charged ions, where the have higher energy. If you look at their energy diagram from electronic point of view, it looks counterintuitive: holes sit at higher energy. However, if you reverse energy axis, it will make perfect sense: holes sit in their energy well.
Image legend: Vertical axis - energy, horizontal axis - distance. White dots - negative ions, black dots - positive ions. Shaded areas show population with carriers.
Image is derived from here.
