What exactly is a photon? Consider the question, "What is a photon?".  The answers say, "an elementary particle" and not much else. They don't actually answer the question. Moreover, the question is flagged as a duplicate of, "What exactly is a quantum of light?" – the answers there don't tell me what a photon is either. Nor do any of the answers to this question mentioned in the comments. When I search on "photon", I can't find anything useful. Questions such as, "Wave function of a photon" look promising, but bear no fruit. Others say things like, "the photon is an excitation of the photon field." That tells me  nothing. Nor does the tag description, which says:

The photon is the quantum of the electromagnetic four-potential, and therefore the massless bosonic particle associated with the electromagnetic force, commonly also called the 'particle of light'...

I'd say that's less than helpful because it gives the impression that photons are forever popping into existence and flying back and forth exerting force. This same concept is in the photon Wikipedia article too - but it isn't true. As as anna said, "Virtual particles only exist in the mathematics of the model." So, who can tell me what a real photon is, or refer me to some kind of authoritative informative definition that is accepted and trusted by particle physicists? I say all this because I think it's of paramount importance. If we have no clear idea of what a photon actually is, we lack foundation. It's like what kotozna said:

Photons seem to be one of the foundation ideas of quantum mechanics, so I am concerned that without a clear definition or set of concrete examples, the basis for understanding quantum experiments is a little fuzzy.

I second that, only more so. How can we understand pair production if we don't understand what the photon is? Or the electron? Or the electromagnetic field? Or everything else? It all starts with the photon. 
I will give a 400-point bounty to the least-worst answer to the question. One answer will get the bounty, even if I don't like it. And the question is this:  
What exactly is a photon? 
 A: The photon is a construct that was introduced to explain the experimental observations that showed that the electromagnetic field is absorbed and radiated in quanta. Many physicists take this construct as an indication that the electromagnetic field consists of dimensionless point particles, however of this particular fact one cannot be absolutely certain. All experimental observations associated with the electromagnetic field necessarily involve the absorption and/or radiation process.
So when it comes to a strictly ontological answer to the question "What is a photon?" we need to be honest and say that we don't really know. It is like those old questions about the essence of things; question that could never really be answered in a satisfactory way. The way to a better understanding often requires that one becomes comfortable with uncertainty.
A: My entry:
For a free or weakly-interacting electromagnetic field, which has radiation in some region at a definite frequency and energy, there is a minimum nonzero amount of energy that one can add to or take away from the field. That amount is a "photon."
Now the fine print:


*

*Of course, many other answers here are more precise, and I think quite a few are insightful as well, but I took the question to be "explain it like I'm a child, but truthfully," and tried to get as close as possible to this.

*As others have noted, photon is not always used consistently, but for virtually every use I can think of the above statement is true (if you think you have a counterexample, please point it out to me). I say "virtually" because the one exception I can think of is the so-called "virtual photon." However, I think this terminology is wildly overused by non-experts anyway and should be avoided, or at least should be discussed separately.

*"Strong" vs "weak" coupling does have a standard precise definition among physicists, but really the transition between free photons plus matter excitations to strongly coupled excitations like polaritons happens smoothly, and there is at no point a sharp qualitative change.

*Experimentally, the requirement of "a definite frequency" will often be relaxed slightly to "a well-defined frequency," because any real source of light always has some finite spectral broadness. This is one of the issues that sometimes causes a slight difference between the experimentalist and theorist notions of "photon."

*This definition, phrased in terms of the energy of a particular part of a field, is hard at first to square away with the "billiard ball particle-like" picture you might have of photons as discrete objects that fly around and bounce off of things. This is simply because that picture is, in many cases, severely flawed. In some very specific situations (perhaps Compton scattering), you might be able to get away with it. However, it is so often misleading that it would probably be better to jettison it entirely until you understand the conditions under which it is roughly valid, which is a subtle issue worthy of an entirely separate discussion. Most of the time, photons are really not at all like little "billiard balls of light".
A: I agree fully with flippiefanus' answer. First and foremost, a photon is a useful concept introduced to describe phenomena to which we do not have an intuitive approach, and apart from that we do not really know what a photon is. Even though it's true, this is not particularly satisfactory. What I want to add to his answer is why the concept of a photon was introduced.
For a long time there was a dispute if light is a particle or a wave. Newton supported and developed much of the "corpuscular theory of light". His strongest argument was that light travels in straight lines, while waves tend to disperse spatially.
Huygens, on the other hand, argued for light being a wave. The wave theory of light could explain phenomena like diffraction, which the corpuscular theory failed to explain. When Young performed his famous double-slit experiment, which showed interference patterns just like the ones known from sound waves or water waves, the question seemed to be settled once and for all.
[An interesting side note: waves need a medium to propagate in, but light also propagates through vacuum. This led to the postulation of the so-called aether, a medium which was supposed to permeate all space. The properties of this aether, however, were contradictory and no experimental evidence of it was found. This played a role in the development of the theory of relativity, but that's another story]
Then, around 1900, Max Planck was able to describe the spectrum of a black body correctly with what he at first thought was just a mathematical trick: for his calculations he assumed that the energy is radiated in tiny portions, rather than continuously, as you would suppose if light was a wave. The black body spectrum was one of the most important unresolved questions at that time and its explanation was a scientific break-through. In consequence, his method received a lot of attention.
Shortly after, Einstein used Planck's method to explain another unresolved problem in physics: the photoelectric effect. Again, this phenomenon could be described if light is imagined as small packets of energy. But unlike Planck, Einstein considered these packets of energy a physical reality, which were later called photons.
This neologism was by all means justified, because by that time it was already clear that photons had to be something else than just small billiard balls as Newton imagined. Sometimes it exhibits wave-like properties that cannot be explained with classical particles, sometimes it exhibits particle-like properties that cannot be explained with classical waves.
Here is what we know:


*

*Light is emitted in discrete numbers of packets. This means that there are countable entities of light (which we call photons today). This statement is probably the most fundamental to the photon idea. It would also be my "one-sentence-answer" if someone asked me what a photon is.

*They carry physical properties like energy and momentum and can transfer them between physical objects (e.g. when thermal radiation is absorbed and heats up a body).

*Since photons have a momentum, they must also have mass.

*Later it was also shown that photons have other properties such as spin

*They propagate in straight lines without the need of a medium (The aether I mentioned before was proven to not exist).
All of these properties are commonly associated with particles. In the very least they show that a photon is something (in the sense that a collection of physical properties typically qualifies as a thing). But a photon also has properties that are different from classical objects:


*

*When photons propagate they show diffraction, refraction and interference

*The energy and momentum of a photon correspond to wavelength and frequency of light, which govern the interference and diffraction behavior.
The bottom line is that a photon is neither wave nor particle, but a quantum object that has both wave-like and particle-like properties. In general one can say that the particle-like properties are dominant when you look at


*

*small numbers of photons

*their interactions with matter (like the pair production process you mentioned)

*high energies
while the wave-like properties are dominant when you look at


*

*large numbers of photons

*their propagation in space

*low energies
I think this is as far as one can get with classical analogies. A photon is what its properties and its behavior tell us, and everything else is just an incomplete analogy. Personally, I like to imagine photons (as with any visualization this is by no means correct, but it works nicely in many situations and helps to get a grip) as small, hard, discrete particles that move around in space like waves would.
A: This is the elementary particle table used in the standard model of particle physics, you know, the one that is continuously validated at LHC despite hopeful searches for extensions.


The Standard Model of elementary particles (more schematic depiction), with the three generations of matter, gauge bosons in the fourth column, and the Higgs boson in the fifth.

Note the word "particle" and note that whenever a physical process is calculated so as to give numbers to compare with experimental measurements, these particles are treated as point particles. i.e. in these Feynman diagrams for photon scattering:

The incoming photon is real, i.e. on mass shell, 0 mass, energy $h\nu$. The vertex is a point and is the reason that particle physicists keep talking of point particles (until maybe string theory is validated, and we will then be talking of string particles). The concept of a photon as a particle is as realistic as the concept of an electron, and its existence is validated by the fit to the data of the standard model predictions.
So the answer is that the photon is a particle in the standard model of physics which fits the measurements in quantum mechanical dimensions, i.e. dimensions commensurate with $\hbar$.
To navel-gaze over the "meaning of a photon" more than over the "meaning of an electron" in a mathematical model is no longer physics, but metaphysics. i.e. people transfer their belief prejudices on the explanation.
We call an electron a "particle" in our experimental setups because the macroscopic footprint as it goes through the detectors is that of a classical particle. The same is true for the photons measured in the calorimeters of the LHC, their macroscopic "footprint" is a zero-mass particle with energy $h\nu$ and spin one.

This CMS diphoton event is in no doubt whether the footprint is a photon or not. It is a photon of the elementary particle table. It is only at the vertices of the interaction that the quantum mechanical indeterminacy is important. 
You ask:

How can we understand pair production if we don't understand what the photon is

It seems that one has to continually stress that physics theories are modelling data; they are not a metaphysical proposition of how the world began. We have a successful QFT model for particle physics, which describes the behavior of the elementary particles as their footprint is recorded in experiments and successfully predicts new outcomes. That is all. 
We understand the processes as modelled by QFT, the understanding of the nature of the axiomatic particle setup in the table belongs to metaphysics. Assuming the quantum mechanical postulates and assuming the particles in the table, we can model particle interactions. It is similar to asking "why $SU(3)\times SU(2)\times U(1)$" The only answer is because the model on these assumptions describes existing particle data and predicts new setups successfully.
I would like to give the link on a blog post of Motl which helps in understanding how the classical electromagnetic field emerges from a large confluence of photons. It needs the mathematics of quantum field theory.  The electric and magnetic fields are present in the photon wave function, which is a complex number function and is not measurable, except its complex conjugate squared, a real number gives the probability density of finding the photon in $(x,y,z,t)$.
It is the superposition of the innumerable photons wave functions which builds up the classical EM wave. The frequency for the individual photon wave function appears in the complex exponents describing it. It should not be surprising that it is the same frequency for the probability as the frequency in the classical electromagnetic wave that emerges from innumerable same energy photons (same frequency). Both mathematical expressions are based on the structure of the Maxwell equations, the photon a quantized form, the EM the classical equations.
A: The starting point to explain photons from a theoretical point of view should be the Maxwell equations. In covariant form, the equations in vacuum without sources are \begin{align} \partial_\mu F^{\mu\nu}&=0\\ \partial_\mu(\epsilon^{\mu\nu\alpha\beta}F_{\alpha\beta}) &=0 \end{align}
 It is well known that the second equation is automatically verified if $F$ is defined in terms of the potential $A$
$$F_{\mu\nu}=\partial_\mu A_\nu-\partial_\nu A_\mu$$
The Maxwell equations can be obtained from the Lagrangian 
$$\mathcal{L}=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}$$ when the Euler-Lagrange are used variating the potential.
This classic Lagrangian is the basis for the formulation of the quantum field theory. Since the Maxwell equations define a classic field theory, it is natural to look for a QFT description, and not just a QM description. Without entering in a discussion of the meaning of quantization (which would be too mathematical-philosophical and wouldn't illuminate your question), let's assume that the formulation of a QFT can be done, equivalently, via path integral and canonical quantization. I only will discuss the latter. 
In canonical quantization, the potential $A_\mu$ and its conjugate momentum $\Pi^\mu=\frac{\partial \mathcal{L}}{\partial(\partial_0 A_\mu)}$ become field-valued operators that act on some Hilbert space. This operators are forced to satisfy the commutation relation $$[A_\mu(t,x), \Pi_\nu(t, x')]=i \eta_{\mu\nu}\delta(x-x')$$
Because of this relation, the two physical polarizations for $A$ can be expanded in normal modes that are to be interpreted as annihilation and creation operators, $a$ and $a^\dagger$. If the vacuum state (i.e., the state of minimum energy of the theory) is $|0\rangle$, then the states $a^\dagger|0\rangle$ are called 1-photon states. Therefore, the photon is the minimum excitation of the quantum electromagnetic potential.
Everything above considers only free electromaynetic fields. That means that photons propagate forever, they can't be emitted or absorbed. This is clearly in conflict with real life (and it's too boring).
Going back to classical electromagnetism, the Lagrangian for the EM field with a 4-current $J$ that acts as a source is
$$\mathcal{L}=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}-A_\mu J^\mu$$
The most common example is the current created by a charged fermion (for example, an electron or a muon) $$J_\mu = ie\bar{\psi}\partial_\mu\psi$$
But this type of Lagrangians presents a huge drawback: we don't know how to quantize them in an exact way.* Things get messy with  interactions. We only can talk, with some rigor of asymptotic states: states long before or long after any interactions resemble those of the free fields. Therefore, the real photon is the excitation of the quantum electromagnetic potential that in the limit $t\to \pm\infty$ tends to the free photon as defined above.
So yes, in a sense, you are right that we don't know what a photon is. But this [formal] obstacle doesn't prevent us from making predictions, like the case of pair production that worries you. The key point is that we don't know what happens during the interaction, we cannot know it and we don't need to know it. We only need to compare the asymptotic states before and after the interaction. In order to do that, we need to perform some approximation, usually a perturbative expansion (that results in Feynman diagrams, the wrongly-called "virtual particles" and all that). The comparation between in and out states, encoded in the $S$ matrix${}^\dagger$, is enough to predict decay rates, cross sections and branching ratios for any process you can imagine. And those observables are the only ones that we can measure. In conclusion, the things that you can't precisely define are the things that you can't experimentally verify.
This answer is only a sketch, a complete answer would require me to write a book on the topic. If you want to know more, I encourage you to read any book on QFT, like Peskin & Schroeder, Weinberg, Srednicki, etc.

* In an interacting theory, the classical equations of motion are non-linear, and can't be solved using a Fourier expansion that produces creation and annihilation operators. In the path integral formulation, we only know how to solve Gaussian integrals (i.e. free fields). To solve the path integrals for interacting fields we still need approximate methods like perturbative expansions or lattice QFT. According to Peskin & Schroeder: 

No exactly solvable interacting field theories are known in more than two spacetime dimensions, and even there the solvable models involve special symmetries and considerable technical complication.


${}^\dagger$ For more details on this, I refer you to this excellent answer by ACuriousMind to another question of yours.
A: Ontological answer
There is no short answer. 
The photon is exactly what you get when you study all our knowledge about it in form of mathematical theories, most of which having the big Q in their names. And then probably some more we didn't find yet. There are no shortcuts.
Justification, reference
This is by all intents and purposes a cop-out answer. A question that asks "how does a photon behave?", "what do we know about the interactions of a photon with XXX?", etc. would be easy(ish) to answer. But I suggest that the question "what is a photon" (in the sense of "what is it really, all the mathematics aside?") can have no meaningful answer whatsoever, in the same respect as the question "what is a XXX" (where XXX is any particle or field of the Standard Model) has no meaning.
Instead of typing a lot, I suggest the Feynman interview, Richard P Feynman - FUN TO IMAGINE (full); the part relevant to this answer ("true meaning of things") goes from 01:03:00 right to the end (summary: even if we have the theories about particles right, and thus can explain their effects, we still have no way to explain them in everyday/practical terms without the maths, and there won't ever be one, as there are no "mundane" laws underneath it). Also partly, in respect to easy, short, answers a portion starting at 17:20 (summary: it is hard to describe anything completely at a particular "resolution", it goes ever deeper; as I said, only partially related, but quite insightful still).
EDIT: summaries added.
A: It takes many pages to really answer the question what a photon is, and different experts give different answers. This can be seen from an interesting collection of articles explaining different current views:
The Nature of Light: What Is a Photon?
Optics and Photonics News, October 2003
My theoretical physics FAQ contains an entry with the title ''What is a photon?''. Here is a short excerpt; but to answer the question in some depth needs read the FAQ entry itself. From the beginning:

According to quantum electrodynamics, the most accurately verified
theory in physics, a photon is a single-particle excitation of the
free quantum electromagnetic field. More formally, it is a state of
the free electromagnetic field which is an eigenstate of the photon
number operator with eigenvalue 1.
The pure states of the free quantum electromagnetic field are elements
of a Fock space constructed from 1-photon states. A general n-photon
state vector is an arbitrary linear combinations  of tensor products
of n 1-photon state vectors; and a general pure  state of the free
quantum electromagnetic field is a sum of n-photon  state vectors, one
for each n. If only the 0-photon term contributes,  we have the dark
state, usually called the vacuum; if only the  1-photon term
contributes, we have a single photon.
A single photon has the same degrees of freedom as a classical vacuum
radiation field. Its shape is characterized by an arbitrary nonzero
real 4-potential A(x) satisfying the free Maxwell equations, which in
the Lorentz gauge take the form $\nabla \cdot A(x) = 0$, expressing the zero mass and the
transversality of photons. Thus for every such A there is a
corresponding pure photon state |A>. Here A(x) is not a field
operator but a photon amplitude; photons whose amplitude differ by an
x-independent phase factor are  the same.

And from near the end:

The talk about photons is usually done inconsistently; almost
everything said in the literature about photons should be taken  with
a grain of salt.  There are even people like the Nobel prize winner
Willis E. Lamb  (the discoverer of the Lamb shift) who maintain that
photons don't  exist. See towards the end of
http://web.archive.org/web/20040203032630/www.aro.army.mil/phys/proceed.htm
The reference mentioned there at the end appeared as     W.E Lamb,
Jr.,     Anti-Photon,     Applied Physics B 60 (1995), 77-84. This,
together with the other reference mentioned by Lamb, is reprinted  in
W.E Lamb, Jr.,     The interpretation of quantum mechanics,     Rinton
Press, Princeton 2001.
I think the most apt interpretation of an 'observed' photon as used in
practice (in contrast to the photon formally defined as above) is  as
a low intensity coherent state, cut arbitrarily into time slices
carrying an energy of $h\nu = \hbar\omega$, the energy of a photon at
frequency nu and angular frequency omega.  Such a state consists
mostly of the vacuum (which is not directly  observable hence can
usually be neglected), and the contributions of  the multiphoton
states are negligible compared to the single photon  contribution.
With such a notion of photon, most of the actual experiments done make
sense, though it does not explain the quantum randomness of the
detection process (which comes from the quantized electrons in the
detector).

See also the slides of my lectures here and here
A: To me, the other elemntary particles are equally mysterious. This is because of their non-intuitive nature.
Naive ontology of the world
We humans grow up in a world with objects. These objects have mass and volume, they have discreet boundaries. Our brains are wont to consider the objects as separate things, and each thing can, for instance, be picked up and otherwise inspected by the senses. Things are accordingly denoted by ordinary nouns.
Then there are fluids, which are almost like things but not quite. They do have mass and volume, and they can be interacted with through the senses (although to realize that air is not vacuum by noting air resistance is a non-trivial leap of insight), but they are not separate. They merge with each other and can be arbitrarily divided, unlike things, so you can never put your hands around exactly what a fluid is. Accordingly, we know them by uncountable nouns, and understand that ultimately it makes sense to think of parts of a single universal fluid. The universal fluid becomes the only thing, since other subdivisions are moot, except in the special case of droplets or quantities sequestered in vessels, where the thingness is being artificially enforced (ie. you can pretend the coffee and the milk are separate things so long as they are in their separate cups, but the moment you let them come in contact this pretense falls apart).
Everything besides these we think of as phenomena. For instance, fire is something that happens: It is not a thing you can pick up and manipulate (you can only pick up the fuel, and the fire clings weirdly to it), nor is it something that you can trap in containers and subdivide or combine (though the fuel may be either a thing or a fluid, and it can be operated on, with fire sometimes coming along for the ride). Likewise for sound, light, temperature and similar concepts.
Of course we now know fire is just plasma, ie. a fluid that "spoils" into something else very quickly, and that it is in fact possible to trap it with the right exotic vessel. Thus the imagination may be coerced into accepting fire as a fluid, although in everyday life it still appears as a phenomenon, thus it is not a truly intuitive thing.
The theory of the atom
When the atom theory came about, I'm certain the Greeks at some point expected the "atoms" to have some mass, shape and size, ie. to be things. Thus, a neat conceptual trick is employed: To the careless eye, sand seems like a fluid, since quantities of it appear to freely merge and split (note that sand is uncountable). But on closer inspection, the sand is just a bunch of tiny objects, which themselves are clearly things.
The realization then came that the world would make sense if everything was a kind of sand made up of many tiny particles, which are themselves things. This is nice because it unites fluids and things: The fluid is only an apparent class, and deep down they're all things - which is nice. Things are very intuitive for us humans, and it can be easier to reason about a pile of millions of tiny things than a single bit of fluid because of how weird fluids are. Luckily, it turned out that both fluids and things were made up of particles, and these particles do seem like bona fide things.
Here the caveats begin. Molecules are not quite Newtonian solids. They behave almost like them: For instance, they can have mass and volume. Almost all of them can be broken apart, but if you consider the very rigid rule of breaking up a molecule vs. breaking a rock, it starts looking funny already. They do have a boundary and bump off of each other... But watch out that you don't bump them too hard, or they weirdly merge together (unlike rocks). But the worst part is the boundary, which is only a fake boundary: The Van der Waals radius is not a binary "can/cannot pass" delimiter, but is a consequence of a continuous force equation. It isn't really that much harder to be slightly inside the molecule than slightly outside. Compare being slightly inside of a rock -- impossible.
As an aside, I think it's interesting that the Greeks came up with a theory of the atom rather than a theory of the fluid, where all solid objects are in fact fluids in some temporary state of rigidity. The various theories of the elements come to mind, but they don't make the right physical observations: One could observe discrete pieces of iron can be melted and seemlessly combined, and then conclude that surely it must be possible to melt any thing, therefore there are only apparent things, and everything is essentially a fluid. Perhaps it is because this theory of fluids makes the world more confusing, not less.
Subatomic particles
Molecules, as it turned out, where simply small aggregations of things -- surely when you break up a thing the result must be smaller things? We soon found out about atoms, and then the parts of the atom. This is where we stop, since to my knowledge, none of the elemntary particles are known to be divisible into further constituents. Photons are one such elemntary particle.
The pretense of a thing may be maintained for atoms and molecules through devices like the Van der Waals volume. For elementary particles, this pretense is hopeless. As has been famously shown time and time again, not only do elementary particles not have volume, they blatantly don't have volume: If they did, physics just clearly doesn't work, and you get things like "surfaces" of electrons spinning faster than the speed of light. The observation was then made that the world would make sense if only these particles were points.
Of course, nobody actually knows what a point mass is. Nobody has ever seen such a thing (well, except that which we wish to christen a point mass in the first place). Its implications seem bizarre: For instance, its density is infinite, and in theory the entire universe could be squeezed to a single point. Had particles been things, such insanities would be comfortably precluded: Rocks cannot be squeezed arbitrarily, not even with infinite force.
Not being squeezable, by the way, is another property of things. Even soft things like sponges turn out to just have pockets of air in them. Once the holes are all squeezed out, a thing cannot be compressed further: Liquids politely pay lip service to this principle, although you can tell (eg. by the water in a syringe experiment) that their hearts really aren't in it, and gases just couldn't care less -- another way in which fluids are strange and unlike things. Or at least, to a naive observer without access to the extreme energies required by our modern physical experiments.
The subatomic level is where intuition completely breaks down. You can create analogies, such as to strings and pots of water, but you can never really imagine what a particle is like in terms of what objects from everyday life. The Universe has played a very cruel trick on us, in that it is one way, but it is such that at the macro level at which we necessarily started to understand it, it is entirely in an other way, with no semblence of the one way to be seen. We are then doomed to grow up expecting and getting used to the other way, only to take Physics 201 in college and find out everything we know is an illusion and no intuition is possible for the true nature of the world. Intuition, indeed, is an comprehension based on experience: Who can experience the subatomic? At best we may experience experimental apparatus.
The top-down approach to understanding the Universe fails, and it fails precisely at the subatomic level.
The bottom up version
One can debate the true meaning of intuition, but I think some sanity can be restored by instead starting over and setting everything right. We can forget all the naive baggage about things and fluids, wipe the slate clean, and start with the fundamental truth that in the world, there are particles. Particles have momentum, they are points, they interact with each other and the vacuum in certain ways described by quantum mechanics. They are elementary, and not made of any smaller units. Photons, then, are one such particle, with specific properties described elsewhere (I won't repeat them, since you explicitly said in your question that these descriptions are not what you want).
When truly large numbers of the particles act together, at the macro level some bizarre phenomena come about, such as "volumes" and "state transitions". You can't really get an intuition for these bizarrities from our knowledge of particles. But logically, ie. if you follow the math, you know it is a simple and straightforward consequence, albeit non-intuitive.
Unfortunately, this bottom up intuition is not very useful. All of our daily lives concern macro phenomena. A lot of the interesting things in the universe (basically, all the disciplines other than subatomic physics) are macro scale. One expects that after learning physics, the world will become easier to understand -- but learning the bottom up intuition only makes everything harder. I suspect even subatomic physics is not made much easier, since all the real work is done with math, not intuition.
So, in conclusion, the question cannot be answered satisfactorily. There are two ways of understanding a question like "what is an X":


*

*"Tell me the salient properties of X": For the photon, there is no shortage of various texts, and even on this site there are perfectly serviceable answers that you have found lacking.

*"Help me intuitively understand X": As I said, no intuition is possible without tearing down all the intuition you have built up over your life. If you do tear it down, what intuition may be gotten is unsatisfying, and only serves to give you a headache.


But that said, a photon is an elementary particle. It behaves as if a point. It has momentum, and moves at light speed (implying that it cannot stop). It has an associated electromagnetic wave. The energy carried by this wave is quantized. The photon can interact with other molecules, for instance by being absorbed and emitted; with enough energy you can create them "from scratch", and they appear to always carry packets of energy with them.
One can wonder, if all particles are merely some form or arrangement of discrete bits of energy, which when aggregated in a certain way, leads to an appearance or seeming of point masses (or should I say "masses"?) and particles, and if individual Planck units of energy are really the basis for everything else in the universe, and perhaps the photon is very close to what these "energies" look like "on their own". Perhaps this is closer to what you were asking, but at this point I'm firmly into navelgazing territory, so I'll stop here.
A: 
Who can tell me what a real photon is? Or refer me to some kind of authoritative informative definition that is accepted and trusted by particle physicists? I say all this because I think it's of paramount importance. If we have no clear idea of what a photon actually is, we lack foundation...
How can we understand pair production if we don't understand what the photon is? Or the electron? Or the electromagnetic field? Or everything else? It all starts with the photon.

I think this is really a philosophical problem, not a physics problem. No answer will satisfy you, because you are asking a question which it is impossible to answer with finality : What is the essence of a thing?
Exactly the same problem exists with every concept of human thought, not only in science (What is energy? What is time? What is colour? What is consciousness?...) but also in the humanities (What is love? What is beauty? What is happiness?...). In each case the more we try to define something, the more elusive it becomes, the less we seem to really understand what the essence of it is. And when we think we have a grasp of it, some new property emerges to throw our understanding into disarray again.
I agree with AnoE (perhaps because I am a disciple of Richard Feynman) that things can only be understood as the sum of their properties, their inter-relations with other things.
In life, it is not necessary to know what love is in order to experience it, or to know what justice is in order to act justly or recognise injustice. The only definition we can give is to summarise our experience of a thing into one or more idealised "models" which isolate the features we consider to be "essential".
In the same way, it is not necessary to have an ultimate definition of a photon as a solid foundation before we can study light or develop powerful theories like QED. A working definition or model is adequate, one which allows us to identify and agree on the common experience and properties which we are investigating.
The history of science shows that the concepts we use are gradually refined over decades or centuries, in particular the question of "What is light?" This lack of ultimate definition has not prevented us from developing elaborate theories like QED and General Relativity which allow us to predict with astonishing accuracy and expand our understanding of how the universe works.
"Photon" and "electron" and "magnetic field" are only our models of things we find in the universe, to help us predict and find relationships between things. As Elias puts it, these models are, of necessity, approximate concepts. They are not what really exists. It is inevitable that they will change as we refine our approximations to try to fit new properties, new observations, into the framework of our understanding, our theories.
A: I am annoyed by the definitions of photon as described in the question. It is not that they are wrong, but because I was mislead by them almost as if they were preventing me to understand what a photon is. Below is what I think now. That is of course no new physics, and every interpretation is subjective. I will go through this by introducing few antithesis.
1. Photons are not discrete
The terms like 'particle', 'quantum of light' or 'unit of energy exchange' lead to believe that photons are something discrete and sudden. Second quantization supplements this idea. For example, in second quantization, the Hamiltonian of a single state (say a particular standing wave in a cavity) can be written as
$$ H = \hbar \omega  (a^\dagger a + 1/2)$$
This is also the Hamiltonian for a harmonic oscillator. Consequently, we can easily then write the 'wave function' of this state as $\Psi(q)$ and Hamiltonian with classical kinetic energy like $p^2$ and potential energy like $q^2$ terms. We can write this wave function as a linear combination,
$$ \Psi(q,t) = \sum_n c_n(t) \psi_n(q), $$
and we realize that the dynamics of photons are not that different from dynamics of electrons. In middle of quantum dynamics (between measurements that is), there can be any kind of wave packet described by $\Psi(q,t)$ or the linear combination coefficients $c_n(t)$. Therefore, the number of photons are not discrete, and they are not exchanged instantaneously in discrete quantities.
Instead, all that is, is the field, and it is subject to typical quantum wave evolution. This field couples to matter.
2. Quantization is not unique
Let's discuss the two transverse modes of a propagating photon (there are actually two more, longitudinal and energy-like, but that is out of scope). It is often said that a photon has angular momentum of $\pm \hbar$, which corresponds to circularly polarized light particles. This leads to a spinor-like representation for a photon.
$$ \left[ \begin{array}{c} \Psi_L(q) \\ \Psi_R(q) \end{array} \right] $$
However, in some applications, it better to analyze only linearly polarized photons ($\Psi_{x,y}(q) = \frac{1}{2} (\Psi_L(q) \pm i \Psi_R(q))$). Now, it is easy to see, that just like electron spin, one has chosen just a preferred frame of reference and there is nothing extremely special about the choice of these discrete coordinates. (Of course, there is something special in choice of coordinates: the physical intuition to describe a problem well.) But in fact, I think that even the transverseness of a polarization is a choice of reference.
3. Wave function collapse creates the apparent discreteness
Say a dye molecule gets excited in our eye-receptor, and it subsequently changes its form, and a nerve impulse is transmitted. Such a process resembles a quantum measurement since it involves so many uncontrolled degrees of freedom in high temperatures, and a phenomenon called decoherence happens. Thus, if the photon wave function was previously $(\frac{1}{\sqrt{2}}|1> + e^{i\theta} \frac{1}{\sqrt{2}}|0>$, the effective wave function (integrating out the macroscopic degrees of freedoms) is in a discrete state with probabilities given by their amplitudes. That is why photons can be seen and heard as clicks. With a grain of salt, it is the collapse of the wave function which makes the sound :)
4. Far-field and near-field photons are different
It is often said that a photon has definite energy and momentum which must be conserved (i.e. it follows the dispersion relation $E=\hbar k$ and one photon which hits the detector has always this (E,k). But for example, there are photonic crystals, where photon energies have band gaps and photons appear to have masses (non-linear dispersion relations). Again, one can quantize the Maxwell equations in a photonic crystal by some choice of states, and assign particles to these states. One can speak of photons here as well, and even say that they have mass since their equations of motion behave as they have mass.
However, since the measurement is usually done in far field, where the photons are asymptotically free, one measures photons like $E=\hbar k$.
5. Modes are not unique
Now picture more modes than just one before. The wave function is now $\Psi(q_1,q_2 \ldots q_N)$. Now imagine creating a linear combination of these modes $q'_i = \sum_j A_{ij} q_j$ to localize them as much as possible. In fact, let's localize to the extent that one mode $q'$ will correspond to a particular location in space. Now you have a 'wave unction' of a photon, which gives a probability amplitude of the photon field at different positions of space.
$$\Psi(r_1, r_2, \ldots r_N)$$
By limiting ourselves to N coordinates which describe a photon roughly around positions ($r_1 \ldots r_N$), we have effectively imposed an energy cutoff to our equations and everything is fine.
Now imagine extending this process to a continuum limit (far from trivial) and switching on the light-matter interaction, and we have encountered the problem of renormalization and all really hard and hardcore stuff.
Given all that, one wants for practical reasons and for physical intuitions sake to go back to the second quantization and talk about one photon in mode 15. In other words, second quantization and the talk about particles as an excitations of harmonic oscillators is all just instruments created by and for the physical intuition. But if one wants to understand what a photon is, one needs to go under the hood.
A: I am going to start my answer by referring to a different one: Which is more fundamental, fields or particles
The photon is really just a special case of what is outlined in this answer. Quoting DanielSank:

Consider a violin string which has a set of vibrational modes. If you want to specify the state of the string, you enumerate the modes and specify the amplitude of each one, eg with a Fourier series
$$\text{string displacement}(x) = \sum_{\text{mode }n=0}^{\infty}c_n \,\,\text{[shape of mode }n](x).$$
The vibrational modes are like the quantum eigenstates, and the amplitudes $c_n$ are like the number of particles in each state. With that analogy, the first quantization notation, where you index over the particles and specify each one's state, is like indexing over units of amplitude and specifying each one's mode. That's obviously backwards. In particular, you now see why particles are indistinguishable. If a particle is just a unit of excitation of a quantum state, then just like units of amplitude of a vibrating string, it doesn't make any sense to say that the particle has identity. All units of excitation are the same because they're just mathematical constructs to keep track of how excited a particular mode is.
A better way to specify a quantum state is to list each possible state and say how excited it is.

A photon is exactly that: a unit of excitation (1) of a mode of the electromagnetic field.

The main problem with the photon is that people try to over trivialise it. This has roots in history. In the early days of quantum mechanics particles and in particular photons were invoked to explain the "particle features of light". In the modern view of quantum field theory this picture gets replaced by what DanielSank describes in the linked question.
As such a photon is complicated. It is not a priori a wavepacket or a small pointlike particle. The field theory unifies both these pictures. Real photon wavefields then are superpositions of these fundamental excitations and they can display both field and particle behaviour. The answer to the OPs following question...

How can we understand pair production if we don't understand what the photon is? Or the electron? Or the electromagnetic field? Or everything else?

...lies therein. If you want to know what happens to the real physical objects, you are moving away from photons. Single photon states in nature are rare if not non-existent.

So what is a photon fundamentally?
Quoting from the question:

[...] "the photon is an excitation of the photon field". That tells me nothing.

It tells a lot, the mathematical formalism is very clear and many people have tried to explain it in answers here and elsewhere.

[...] because it gives the impression that photons are forever popping into existence and flying back and forth exerting force. This concept is there in the photon Wikipedia article too. It isn't true. As as anna said virtual particles only exist in the mathematics of the model. So, who can tell me what a real photon is? [...]

The problem here is really the relation between the mathematical formalism and "reality". A "real" photon is not a thing, the photon is a mathematical construct (that was described above) and we use it (successfully) to describe experimental outcomes.

 (1) courtesy of DanielSank. 
A: The word photon is one of the most confusing and misused words in physics. Probably much more than other words in physics, it is being used with several different meanings and one can only try to find which one is meant based on the source and context of the message.
The photon that spectroscopy experimenter uses to explain how spectra are connected to the atoms and molecules is a different concept from the photon quantum optics experimenters talk about when explaining their experiments. Those are different from the photon that the high energy experimenters talk about and there are still other photons the high energy theorists talk about. There are probably even more variants (and countless personal modifications) in use.
The term was introduced by G. N. Lewis in 1926 for the concept of "atom of light":

[...] one might have been tempted to adopt the hypothesis that we are dealing here with a new type of atom, an identifiable entity, uncreatable and indestructible, which acts as the carrier of radiant energy and, after absorption, persists as an essential constituent of the absorbing atom until it is later sent out again bearing a new amount of energy [...]–"The origin of the word "photon""


I therefore take the liberty of proposing for this hypothetical new atom, which is not light but plays an essential part in every process of radiation, the name photon.–"The Conservation of Photons" (1926-12-18)

As far as I know, this original meaning of the word photon is not used anymore, because all the modern variants allow for creation and destruction of photons.
The photon the experimenter in visible-UV spectroscopy usually talks about is an object that has definite frequency $\nu$ and definite energy $h\nu$; its size and position are unknown, perhaps undefined; yet it can be absorbed and emitted by a molecule.
The photon the experimenter in quantum optics (detection correlation studies) usually talks about is a purposely mysterious "quantum object" that is more complicated: it has no definite frequency, has somewhat defined position and size, but can span whole experimental apparatus and only looks like a localized particle when it gets detected in a light detector.
The photon the high energy experimenter talks about is a small particle that is not possible to see in photos of the particle tracks and their scattering events, but makes it easy to explain the curvature of tracks of matter particles with common point of origin within the framework of energy and momentum conservation (e. g. appearance of pair of oppositely charged particles, or the Compton scattering). This photon has usually definite momentum and energy (hence also definite frequency), and fairly definite position, since it participates in fairly localized scattering events.
Theorists use the word photon with several meanings as well. The common denominator is the mathematics used to describe electromagnetic field and its interaction with matter. Certain special quantum states of EM field - so-called Fock states - behave mathematically in a way that allows one to use the language of "photons as countable things with definite energy". More precisely, there are states of the EM field that can be specified by stating an infinite set of non-negative whole numbers. When one of these numbers change by one, this is described by a figure of speech as "creation of photon" or "destruction of photon". This way of describing state allows one to easily calculate the total energy of the system and its frequency distribution. However, this kind of photon cannot be localized except to the whole system.
In the general case, the state of the EM field is not of such a special kind, and the number of photons itself is not definite. This means the primary object of the mathematical theory of EM field is not a set of point particles with definite number of members, but a continuous EM field. Photons are merely a figure of speech useful when the field is of a special kind.
Theorists still talk about photons a lot though, partially because:


*

*it is quite entrenched in the curriculum and textbooks for historical and inertia reasons;

*experimenters use it to describe their experiments;

*partially because it makes a good impression on people reading popular accounts of physics; it is hard to talk interestingly about $\psi$ function or the Fock space, but it is easy to talk about "particles of light";

*partially because of how the Feynman diagram method is taught.
(In the Feynman diagram, a wavy line in spacetime is often introduced as representing a photon. But these diagrams are a calculational aid for perturbation theory for complicated field equations; the wavy line in the Feynman diagram does not necessarily represent actual point particle moving through spacetime. The diagram, together with the photon it refers to, is just a useful graphical representation of certain complicated integrals.)

Note on the necessity of the concept of photon

Many famous experiments once regarded as evidence for photons were later explained qualitatively or semi-quantitatively based solely based on the theory of waves (classical EM theory of light, sometimes with Schroedinger's equation added). These are for example the photoelectric effect, Compton scattering, black-body radiation and perhaps others.

There always was a minority group of physicists who avoided the concept of photon altogether for this kind of phenomena and preferred the idea that the possibilities of EM theory are not exhausted. Check out these papers for non-photon approaches to physics:

R. Kidd, J. Ardini, A. Anton, Evolution of the modern photon, Am. J. Phys. 57, 27 (1989)
http://www.optica.machorro.net/Lecturas/ModernPhoton_AJP000027.pdf


C. V. Raman, A classical derivation of the Compton
effect. Indian Journal of Physics, 3, 357-369. (1928)
http://dspace.rri.res.in/jspui/bitstream/2289/2125/1/1928%20IJP%20V3%20p357-369.pdf


Trevor W. Marshall, Emilio Santos: The myth of the photon, Arxiv (1997)
https://arxiv.org/abs/quant-ph/9711046v1


Timothy H. Boyer, Derivation of the Blackbody Radiation Spectrum without Quantum Assumptions, Phys. Rev. 182, 1374 (1969)
https://dx.doi.org/10.1103/PhysRev.182.1374

A: When Max Planck was working to understand the problem of black-body radiation, he only had success when he assumed that electromagnetic energy could only be emitted in quantitized form. In other words, he assumed there was a minimal unit of light that could be emitted. In assuming this, he of course found $E = h v$ where $h$ is Planck's constant. 
In 1905, Einstein took this seriously and assumed that light exists as these fundamental units (photons) with their energy given by $hv$ where $v$ is the frequency of the radiation. This photon explained many an experimental result, and gave light its wave-particle duality.
Many things have a minimum "chunk": the Planck length, below which distance becomes meaningless, the quark, gluon, and other fundmental units of matter, Planck time (supposed to be the minimum measurement of time, it is the time required for light to travel in a vacuum the distance of one Planck length).
So, what is a photon?
It is the minimum "unit" of light, the fundamental piece. I would say the atom of light, but that doesn't quite convey the right image. (The "quark" of light?)
It's also important to remember that we can't "see" a photon, just as (well, even more so) than we can't "see" a quark. What we know about them is from experiments and calculations, and as such, there isn't really a physical picture of them, as is true for most of quantum mechanics.
A: With any concept in physics there is a dichotomy between model and physical system. In practice we forget about this dichotomy and act as if model and physical system are one and the same. Many answers to the question "what is a photon?" will reflect this identification of model and physical system, i.e., a photon is an ideal point particles; a photon is a field quanta; a photon is a line in a Feynamn diagrams, etc. These definitions of a photon are deeply rooted in models. Our predilection for identifying model and physical system is rooted in a false assumption that the intuition and imagery we develop for understanding the model applies equally well to the physical system. We convince ourselves that the little white speck whizzing through 3-space at the speed of light in our head is a photon, when in reality it is imagery associated with a model.
In light of this, there are two essential ways to answer the question "what is a photon?" The first way is to refer to a model and say "the photon is concept X in model Y." Many users have taken this route. The second way is to refer to an experiment and say "the photon is the thing that is responsible for this data value." I tend to prefer this route when answering the question "what is a ___?" because it avoids the assumption that model and physical system are identical. Applied to the photon I would say "a photon is a packet of electromagnetic radiation that satisfies $E=h\nu$ and it is the smallest packet of electromagnetic radiation."
If you are dissatisfied with both types of definition then you are out of luck. Our models will always remain in our heads and the physical world they so accurately describe will always remain just out of reach.
A: Radiation is emitted when an electron decelerates and absorbed when it accelerates according to the well known Larmor formula.This radiation is a continuous electromagnetic field. Inside the atom, electrons change orbit and accelerate fast in the process resulting in emitting and absorbing radiation so fast that it appears as lines of spectra. But other than the spectral lines, atoms and molecules emit at so many other frequencies due to the various oscillation motion in them and the corresponding acceleration/deceleration accompanying that.These background frequencies would naturally be of lower frequency than that of the line spectra for the same system. Cherenkov radiation is perhaps the nearest to a continuous spectra. 
That is why radiation from all matter is composed of sharp spectral lines on a background of continuous radiation. The photon is the unit of radiation energy exchanged between bound(not free) electrons. It is like the currency of money.. and like currency, photons are not all of the same denomination/energy. The formula E=nhf gives n as the number of photons exchanged of frequency f resulting in energy E. But since f is variable and not even discrete, E is not discrete, but the photon is. 
The photon is also described as a packet of energy. This is correct, but only means the minimum (n=1)energy of a certain color that can be exchanged in a certain interaction between atoms. Normally the energy of waves is directly proportional to the square of their amplitude and nothing to do with frequency. To reconcile this with the definition of the photon very much needed for nuclear interaction, the number of photons n is introduced to compensate for the original formula E=hf. Note that while a photon is a packet of energy, the amount of energy in the packet can vary. A single blue photon has a lot more energy than a red photon for example. Gamma rays have the highest energy content due to their higher frequency.
If you read a book on photonics, you'll find that the word photon crops up in nearly every line of it. This shows how important is the concept of a photon- despite its unusual and a bit confusing definition and use.  
A: As far as I understand we agree:
We use the word Photon when we somehow observe, that a material object changes energy content, momentum and spin and there is no other material object in direct proximity to interact.
We know that there is magnetic force and electrical force and that these forces can be unified to the electromagnetic force and the spacial and timely development of those forces allows electromagnetic waves very similar to other waves that can be observed directly like water waves.
From water and air we know that certain excitations can exist like plain or circula waves. Special waveforms are vortices or more general solitons.
We know that solitons can exist as quantized excitations like in superfluids. So there is a certain probability that such solitons can be solutions of the Maxwell Equations. That would allow to imagine, what a Photon indeed "is".
While writing I had the idea to google for "photon soliton". An found this open paper as a starting point for me: Photons as solitons and there: direct access
P.S. please do not downgrade this answer as it is work in progress and I will react on any comment to be more clearly. And I can not comment to others answer just now. Thanks
A: This is the big "zen" question of physics for centuries, thanks for asking it. 
Other answers are good/acceptable, this one (reputationally risky, but sincere/detailed) takes in some ways radically different angle/approach. Other answers look toward the past, this one will attempt to do the near impossible of anticipating the future in a visionary yet =---still scientifically grounded way. In other words the questionable parts of it can be considered hypothetical i.e. hypotheses-under-consideration but all carefully backed by current/ solid (in some cases very recent) research findings.
The photon story has a dramatic "blind men and elephant aspect" that crosses many centuries.[1] The wave versus particle nature of light was even debated in Newtons time in the 17th century now close to ~4 centuries ago and light units or particles were named "corpuscles" in contrast to the Huygens wave theory.[2] Newtons theory held sway for something like a century over the latter "partly because of Newtons great prestige" even though Huygens was formulated nearly/ initially the same time. This shows an example of the "reverse" effect of human reputation on scientific thinking in the era.
The recent Lacour-Ott experiments are breakthrough and show "a locally deterministic, detector-based model of quantum measurement".[3][4] This is a startling finding that has not been widely considered yet. It proves that a complete quantum mechanical formalism can arise in the analysis of mere classical systems. So this calls into serious question the nearly-century long assertions that quantum mechanics is inherently different than classical mechanics, now seen as not merely a belief system but a virtually a dogma of the field. There are many other recent developments that put chinks/dents in this long armor and seem to force a reevaulation/reconsideration[7] (but this will surely be a lengthy process and it is only beginning).
The new theories are being compared to Bohmian mechanics but have distinctly different and new aspects, and should not be knee-jerk dismissed as refuted. One of the most comprehensive surveys so far is by Bush.[5] its is newly supported by experiments![6]
So how is this possibly conceptually/theoretically? One striking new realization is that Borns probabilistic law in quantum mechanics can arise in classical systems. See e.g.
Qiaochu Yuan, “Finite noncommutative probability, the Born rule, and wave function collapse” [8] and other much more detailed analysis of detectors comes from Khrennikov and “PCST”, “Prequantum Classical Statistical Field Theory,” what is also known as roughly semi-classical theories.[9][10]
[9] Talks about a detector discarding energy where the incoming energy does not match the detector energy threshold (p9) and the detector "eating a portion of energy" (p10). Lets call this a dissipative detector. another similar concept in measurement is the detector dead time[10p5] where "the detector cannot interact with an incoming pulse". 
It appears these concepts are similar to a very sophisticated/comprehensive study of Bells theorem where the signalling system may have so-called "abort" events and which finds that stricter versions of Bell inequalities are not violated by current experiments.[11][13]
These are similar to the "sampling loophole"[12] which is not necessarily the same as so-called efficiency loopholes, because the former may still persist even as detector efficiency is measured at 100%!
Lets explore the concept of a dissipative detector more carefully and how it would look theoretically. Consider the following sketch. A spherical single wavefront travels through space. Now imagine it passing through the detector. the detector may be in the dead time region and will not detect the wavefront. Or, it may detect it. This is the probabilistic nature of light. It appears that possibly dead time cannot be reduced to zero as a physical law related/similar to the Heisenberg uncertainty principle.
Another way of saying all this is that perfect detectors do not exist. The only detectors we have are made of atoms, aka particles. The mystery of the photon is then finally unravelled. A photon is a (probabilistic) interaction between a wavefront and a measuring device, namely a atom or other particle. The interaction can only be referenced a posteriori and not a priori. In other words even a detector made of a single atom will have this dead time and energy-dissipating property. So there we also have some interpretation for so-called virtual particles.
Others may question "a spherical single wavefront" travelling through space. Exactly that picture is now supported by a new model of spacetime outlined comprehensively by Tenev/Horstemeyer, the "spacetime fabric".[14] they don't seem to consider the EM stress tensor much but one obvious generalization of their work is that EM waves are s-waves in the spacetime fabric.
A fairly straightforward experiment demonstrating these ideas is the HBT effect. Imagine a line of detectors all at the exact same distance from a "single-photon" source as a straightforward way of increasing wavefront detection sensitivity. The idea of a "single-photon" source may be better visualized as a "single-wavefront" source. As the wavefront passes through the detectors, each detector may or may not click. If any click, there was a wavefront. If none do click, the wavefront may have passed but they may have all been in their "unresponsive" deadtime period. The overall combined array will detect the wavefront with greater accuracy.
This effect is already observed but not interpreted under this point of view. It's called photon (anti)bunching in the literature. Many other effects are currently misinterpreted under the fog/haze of our currently cloudy theory. It will take a long time to rework it all. But such re-workings are not unheard of in the history of science, although they tend to be about once-a-century events and literally lead to/require e.g. textbooks to be rewritten (but not all at once!).[17] They cannot be timed exactly (analogously to earthquakes) and are even difficult to recognize in the middle but some signs (collected refs, e.g. also [18], many others not cited due to space/ format limitations, following quote etc) are currently present and we seem to be overdue for one.

“I wish that the people who were developing quantum mechanics at the beginning of last century had access to these experiments,” Milewski said, “because then the whole history of quantum mechanics might be different.”[7]

[1] blind men and elephant / wikipedia
[2] Corpuscular theory of light / wikipedia
[3] A Locally Deterministic, Detector-Based Model of Quantum Measurement / La Cour
[4] Quantum computer emulated by a classical system / physorg
[5] Pilot-Wave Hydrodynamics / Bush
[6] new support for alternative quantum view / Wolchover
[7] Have We Been Interpreting Quantum Mechanics Wrong This Whole Time? / Wolchover
[8] Finite noncommutative probability, the Born rule, and wave function collapse Qiaochu Yuan
[9] Born's rule from measurements of classical signals by threshold detectors which are properly calibrated / Khrennikov
[10] Prequantum Classical Statistical Field Theory: Simulation of Probabilities of Photon Detection with the Aid of Classical Brownian Motion / Khrennikov
[11] Robust Bell inequalities from communication complexity / La Plante
[12] Loopholes in Bell test experiments / wikipedia
[13] Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres / Hensen et al
[14] The Mechanics of Spacetime - A Solid Mechanics Perspective on the Theory of General Relativity / Tenev, Horstemeyer
[15] EM stress energy tensor / wikipedia
[16] HBT effect / wikipedia
[17] Paradigm shift / wikipedia
[18] EmQM13: Emergent Quantum Mechanics 2013 conference / proceedings 
A: A friend asked me this in college, and this is more or less what I told him.
Experimenters were figuring out the behavior of electric & magnetic phenomena in the late 18th and early 19th centuries, and by about the mid-19th century it was all coming together. James Clerk Maxwell put the "finishing touches" on the equations that described (classical) electrical and magnetic phenomena. 
One of those equations, (Faraday's law), describes how a changing magnetic field can induce an electric current, while another equation, (Ampère's circuital law) describes how an electric current can induce a magnetic field.
So think of electrons, they have electric charge, if we "jiggle" one just right, we can create a changing electric field, which induces a changing magnetic field, which induces a changing electric field, and so on... These little ripples of the electric and magnetic fields, inducing one another, that's what photons are.
At some point I read a fascinating account of Maxwell, about how once he had worked out the equations to describe electromagnetism, he observed they could be used to derive a wave equation. Wave equations have a constant that describes the propagation speed of the waves and his derived wave equation had $\frac{1}{\sqrt{\mu\epsilon}}$ for that constant (with $\mu$ being the permeability and $\epsilon$ the permittivity). 
If I recall right, these values could be measured by experiments with electricity. Something like sending a known amount of current through two parallel wires, they'll generate a magnetic force that pushes the wires apart (if the current is in the same direction, and pull them together if it's in opposite directions, I think?). Measuring the resulting force can tell you the value (I suppose of only one of them). 
So these values had been measured, and plugging them in Maxwell got something close to the speed of light, which experimenters had been measuring with increasing accuracy around that time (notably in 1849 and 1862). And this was the first time someone (Maxwell) could realize that light was some kind of electromagnetic phenomenon. [Looking it up I see that actually, Wilhelm Eduard Weber and Rudolf Kohlrausch in 1855 had noticed the units of $\mu$ and $\epsilon$ could produce a velocity, and they measured them experimentally and came up with a number very close to the speed of light, but didn't make that final leap of logic, which Maxwell did in 1861.] (From the wikipedia article History of Maxwell's equations)
I'm not an expert, but my impression is that Maxwell also noted that his equations seemed incomplete, because they suggested that the speed of light remained constant regardless of the speed of the observer or the emitter. It's common for people to think that Einstein's work on special relativity was resolving the famous null result of Michelson & Morley's interferometer experiments looking for a luminiferous aether, but Einstein was actually addressing this invariance indicated by Maxwell's equations, if I understand right.
(Note, it's been a long time since I've recounted most of this, and I just have a B.Sc. in math-physics, and haven't used any of this knowledge in a long time, so it's possible I'm getting some details wrong, but I think the general gist is pretty close.)
A: Before even attempting to ask what a photon is "exactly" , we have to ask: do photons exist?
You can go a long way believing they do not. Atoms, molecules and crystals have discrete states that determine the quantum nature of matter, so emit and absorb quanta of energy while the entity itself can be continuous, much like wine is a continuous entity that is only quantised to 70 cl because of the bottles it is sold in. Quantum mechanics uses the classical EM field. The wavy lines in Feynman diagrams, often loosely called photons, are just a graphical notation for terms in a perturbation expansion.
Yet a problem remains: how come photons are absorbed in very local reactions? How can an extended classical electromagnetic wave be absorbed by a single atom? To me a sensible interpretation of these phenomena is that the EM field describes the probability that an absorption / emission takes place.
For this reason I am now convinced that discrete photons exist and that the wave equation underlying the Maxwell equations is a relativistic quantum wave equation describing massless quantum particles just like the Schrödinger, Dirac and Klein-Gordon equations describe massive quantum particles. The electromagnetic wave equation in my interpretation is a massless Klein-Gordon equation describing the quantum particles known as photons.
This does not answer the question what a photon exactly is. It does propose an answer to the preceding question whether photons exist.
A: The easiest way to understand a photon is that it is a particle which obeys quantum mechanics rather than classical mechanics. In quantum mechanics, particles do not have a position, but we can calculate the probability for where the particle will be found (or in the case of a photon, where it will be annihilated). As Dirac put it

“In the general case we cannot speak of an observable having a value for a particular state, but we can … speak of the probability of its having a specified value for the state, meaning the probability of this specified value being obtained when one makes a measurement of the observable.”

The calculation of quantum probabilities is necessarily different from that of classical probabilities, because quantum mechanics describes truly indeterminate processes whereas in standard probability theory results are determined by unknowns. For deep and subtle reasons in the mathematical foundations of quantum mechanics (not usually covered in standard text books) the probability interpretation requires that the calculation follows the laws of wave mechanics, creating the illusion that particles have wave properties.
I have shown the mathematical argument in The Mathematics of Gravity and Quanta
A: Besides a photon is a plane wave that in a 1-Dimension model goes like this $ e^{i*(kx - wt)}$. So a photon is not localised but it is in a lot of places at once along the whole universe. At least until you do some measurement, you know. Quantum particles move like waves but when measured they are like marbles.
About this mathematics I would say that such a formulation is used for example when you want to get the spectrum of some model. For instance in quantum walk theory, in quantum optics
A: Pertinent Einstein quotes:


*

*
All the fifty years of conscious brooding have brought me no closer to answer the question, “What are light quanta?” Of course today every rascal thinks he knows the answer, but he is deluding himself.


*
For the rest of my life I will reflect on what light is.

Is there anything more to the photon than can be accounted for by the changes in the properties of its emitter and absorber alone? (Answer re-expressed 6-11-16)
After slightly more negative than positive responses, and also a helpful correction, overall it’s time to stop digging myself into a hole.
Nevertheless, checking recent literature, may I finish by drawing the attention of the critics to a recent paper in Quantum Stud.: Math. Found. (2016) 3:147–160 (copy attached Rashkovskiy (2016)) which also argues for the photon’s non-existence, but via the semi-classical approach. Although it goes into the matter in far greater depth and detail than I could achieve, I believe it is consistent with the view that, ultimately, the photon's characteristics might best be sought in the properties of the emitter and absorber, rather than being attributed to a fictitious photon… or does it make any difference at all, whether or not we regard it as 'real'?
As comments indicated that part of my argument is flawed, references to “space-time co-incident points” and  “Minkowski-space “contact interaction” ” have been removed 
Nevertheless, references in Arthur Neumaier’s contribution above include instances of eminent physicists arguing the non-existence of photons:


*

*P C W Davies: “In his provocatively titled paper “Particles do not Exist,” Paul Davies advances several profound difficulties for any conventional particle conception of the photon”

*The Nobellist Willis Lamb, was said to “maintain that photons don't exist.”
I offer an elementary argument, by relying on Feynman’s lectures (Vol 1, end of section 17-2 on page 17-4), 

"In our diagram of space-time, therefore, we would have a
  representation something like this: at 45° there are two lines
  (actually, in four dimensions these will be "cones," called light
  cones) and points on these lines are all at zero interval from the
  origin. Where light goes from a given point is always separated from
  it by a zero interval, as we see from Eq. (17.5). Incidentally, we
  have just proved that if light travels with speed c in one system,
  it travels with speed c in another, for if the interval is the same
  in both systems, i.e., zero in one and zero in the other, then to
  state that the propagation speed of light is invariant is the same as
  saying that the interval is zero."

in which he states that points on the light-cone are zero distance from the origin and from each other. Of course, he means zero 4-distance in space-time, as a result of the Minkowski metric requiring a subtraction between the 3-space interval and c times the time interval. 
So one can argue that photons have no independent existence in space-time: they “appear” only as a loss, by emission from a source, and a gain by absorption [rest of original sentence deleted].
In this view, photons are, in a sense (as others have mentioned in more sophisticated replies) simply convenient fictions of the maths that account for what happens in the observed energy transfer between emitter and absorber, across a 3-space and time gap that we bridge using the mathematics describing wave motions. This suggests that the photon's characteristics (e.g. spin) should be sought in the properties of the emitter and absorber, rather than being attributed to the fictitious photon…
