How does resistance *really* work? (DC, battery, LED, atoms, electrons) Backstory: I’m a software engineer just getting into electronics and it seems that everything I’ve ever been told about electricity my whole life is a candy-coated lie. I can’t find consistent logical answers to the most basic of questions and it’s driving me mad!
The kindergarten math V = IR makes sense... unless accounting for conservation of energy, matter, and real laws of physics.
I’m old enough now. I just want to know the truth, even if it hurts.
The effect of voltage on current


*

*A resistor, led, and copper wire walk into a bar

*The bar tender serves a 9v battery to share

*The electric field is too weak to serve the led directly

*The copper wire volunteers to help direct the electric field

*The ampacity of the led is too low, so the resistor hops up on the bar in-line with the copper wire so the LED can get a drink

*Ohms Law is a lie, but the coulombs are absolutely intoxicating. All hell breaks loose and the resistor catches the whole bar on fire, burns it to the ground, everybody dies, and I lost five dollars.


Pause.
No, wait, there was no fire, that was just my anger at how every explanation I read of this scenario is in direct contradiction to what I thought I knew about conservation of energy and matter.
Contradictions for which I'd like answers
If charge causes the electrical field, then why does the voltage drop across the resistor? The electrons didn’t just magic themselves away. Isn’t the charge the same?
If charge passing though the resistor causes the atoms to enter a lower energy state, thereby releasing IR photons that heat up the place... then where did the extra coulombs go each second?
How come 2x resistance makes my battery lasts (on the scale of) twice as long but at (on the scale of) 1/4 of the power?
If resistance slows the flow of current, shouldn't ALL of the current still be accounted for somewhere on the system? solved: many of the explanations I was reading made it sound as though resistors lowered the current (...from infinity?) by "burning off" the "extra" current, which made no sense and contradicted the idea that the current supply and current drain were equal (Kirchoff's Law, common sense). Hence, the oversimplification of some of what I was reading confused me greatly.
... either my understanding is way off or there’s a well kept secret that few people are sharing (or my Google fu is busted)
 A: 
If charge causes the electrical field, then why does the voltage drop across the resistor? The electrons didn’t just magic themselves away. Isn’t the charge the same?

Crudely, the electric field accelerates an electron. It flies along until it bumps into something (a molecule in the resistor). This causes the electron to slow down (or even fly back the other direction), and the molecule to vibrate. 
(Actually the free electrons in the material will mostly be flying around in almost entirely random directions, with only a very slight bias in the direction opposite [because they're negatively charged] the electric field)
Then the electron, still in the electric field starts accelerating again.
The vibration of the molecule gets transferred to other nearby molecules in a random way, which we see at the macro scale as heat.

If charge passing though the resistor causes the atoms to enter a lower energy state, thereby releasing IR photons that heat up the place... then where did the extra coulombs go each second?

There's no atoms changing state or photons here, just electrons being accelerated and then transferring their kinetic energy directly to molecules they run into ("interact with" to use more physics-y language).

How come 2x resistance makes my battery lasts (on the scale of) twice as long but at (on the scale of) 1/4 of the power?

With a fixed source voltage, 2x resistance means 2x less current.
This means 1/2 the power consumed by the resistor, not 1/4.

If resistance slows the flow of current, shouldn't ALL of the current still be accounted for somewhere on the system?

Yes, the current flowing out of the resistor is the same as the current flowing into the resistor. The current is the same at every point around the circuit (out of the battery, into the resistor, out of the resistor, into the LED, out of the LED, and back in to the battery)
A: What really happens to the electrons in a solid when an electric field is applied is extremely complicated, and depends heavily on the material in question. What's more, the electrons cease to be "electrons" as the elementary particle in vacuum, they become quasiparticles with non well defined velocity and with other strange properties. I'm afraid there's no simple answer to the original question. It has to be at the level of quantum field theory applied to condensed matter. I do not have such a level of understanding (yet at least).
Nevertheless, I can offer a much different insight than the ones already posted, closer to what really happens in a conductor when an electric field is applied. Let's take a simple material as conductor such as an alkali metal. Its atoms/ions form a crystal. An intuitive way to think about the electrons in that solid, is to assume that all the core electrons, i.e. the ones in filled shells, are not free electrons and we can ignore them completely for electrical conduction matters. Only the single valance electron is a free electron. That yields one free electron per atom. All these free electrons behave roughly as in a cold Fermi gas, that is, they have to satisfy Pauli's exclusion principle and their occupation number obeys Fermi-Dirac statistics. Thus:


*

*Case when $\vec E = \vec 0$. In that case, the energy of the electrons range from 0 up to about the Fermi energy, $E_F$ (if the temperature is at absolute 0, then it is exactly at the Fermi energy). In k-space (momentum space, not real space), the electron's momentum form a Fermi sphere. Note that this is valid for most alkali metals, but for metals like copper and iron, the shape is not quite spherical. The wavefunction of each electron extends to the crystal sample (they are not localized), and they have velocities ranging from 0 up to the Fermi velocity, which is about two order of magnitude slower than light. But they go in all possible directions and thus the mean velocity is null: there is no current, the drift velocity is 0.
- Case when $\vec E \neq \vec 0$. What happens when we apply an electric field? Usually ordinary currents have a magnitude which causes a very, very small perturbation to the energy of whole system. Contrarily to what the Drude model assumes, in reality only the electrons near the Fermi surface of the sphere (or simply surface in general) can "feel" or react to the applied electric field. This is due to Pauli's exclusion principle which implies that no two electrons can share the same state. Thus the free electrons that have an energy much lower than $E_F$ cannot increase their energy, since all the states which have an energy slightly above them are already occupied. Therefore the net result of the applied field is to cause the electrons that were moving in the field's direction with momentum near $p_F$, to interact with the field and have their momentum switched in the other direction, with roughly the same magnitude. The fraction of the free electrons that can react to the electric field is of the order of $v_d/v_F$, or about $10^{-4}/10^6 =10^{-10}$. Hence only about one free electron per ten billions will get influenced by the electric field. Mathematically it is equivalent to a shifting of the Fermi surface against the direction of the electric field, by an extremely small amount (because the E field is such a small perturbation). Note that the drift velocity that arises in that free electron model is the same as in Drude's model, but the physics is quite different and proved to be more correct than Drude's.
Just to clear some misconceptions: when one applies an electric field in a conductor, it "travels" at a fraction of the speed of light, roughly about 20% to 80% of light's speed. The electrons that take part in electrical conduction move at speed about two orders of magnitude slower than light, and they are extremely less numerous than the number of free electrons. This yields a drift velocity that matches the one in Drude's model. Note that the number of electrons that can react to an applied electric field does not match the number of electrons that can absorb heat, or take place in heat conduction.
About the resistance (or resistivity): The resistivity is partly due to the scattering of the few free electrons going against the $\vec E$ field that take part in the electrical current. They are scattered by phonons and "put back" in the energy state where they were before the $\vec E$ field was applied. Note that they interact with phonons (a quasiparticle), and defects (like a missing atom in the crystal lattice), among others. The electrons that participate in electrical conduction do not really bump into atoms as Drude's model claims.
A: In addition to The Photon's excellent answer, the part about charge conservation is formalized in circuit analysis as Kirchoff's laws.
When you look at a circuit as a graph with edges (components) and nodes (mutual connections), the current entering and leaving a node sums to zero (KCL) and the voltage changes as you walk any loop in the graph also sums to zero (KVL).
So, yes, current leaving any part of the circuit is required to come from somewhere else and charge is conserved.
A: 
If charge causes the electrical field, then why does the voltage drop across the resistor? The electrons didn’t just magic themselves away. Isn’t the charge the same?

The voltage (of a charge) changes as it goes across an electric field.   Just like the potential energy of a mass changes as you lift it in the air.  The mass is the same from one place to another, but the potential is different.
The big difference is that gravitational potential near the earth is mostly uniform.  There's very little we can do to change it.  The electrical potential in a wire changes constantly as the charges rearrange.
When a current goes through a resistor, the charges arrange themselves so that there's an electric field inside.  Now when a charge moves through the field, it changes the potential (voltage) from one side to the other.

If charge passing though the resistor causes the atoms to enter a lower energy state, thereby releasing IR photons that heat up the place... then where did the extra coulombs go each second?

Photons are not coulombs.  If you slide a block down a ramp, it heats up without the block changing.  The photons came from the change in potential as it loses height.  In the same way the charges passing through the resistor don't change, but the energy of that charge does change.
