"In a photon's frame of reference travel is instantaneous, so presumably a photon cannot exist in its own frame of reference."
I am not sure what this means. We as observers cannot boost into such a reference frame but that has no bearing on the existence of a photon.
"I am not aware that a "beam" of photons interacts via gravity despite having theoretical relativistic mass, is that correct?"
Photons do not have mass. This idea has popped up several times in a couple days. Mass = 0 in all frames of reference, related to the fact that its speed is the same in all frames of reference. And light does interact with Gravity via General Relativity. We know this for a fact, light paths are bent along curves when passing near strong gravitational sources, also red/blue shifted due to gravity. This is not predicted by Newton's law of gravity but Newton is only a weak approximation to Einstein's GR theory.
"So in what sense do photons have a physical existence or are they merely a theoretical shorthand for a disturbance in the E-M field?"
One could argue that ALL matter and energy are theoretical shorthand of a sort. In the classical sense light is in fact a disturbance in the electromagnetic field. However a "photon" is a particle. In the early days of QM we adopted a dual description of matter, i.e. particle-wave duality. This was done to describe certain effects that could not be predicted using a particle model. The photon comes into the scene when the same duality was applied to light. It is a particle dual to the "wave" you describe or mention in your question. Plank first introduced the idea to describe Black Body radiation (classical theory failed). He assume that the states were quantized and was able to predict his observed spectrum. However, people didn't give an credible meaning to that. Einstein was the one who assumed that all light had a particle nature in free space and use it to predict the photo-electric effect. Compton also used the particle model to predict the behavior of photon-electron collisions. Once the formalism of QM and QFT were in place the photon had a meaning as a coherent state (or eigenstate) of the field operator equation.
In short, no the photon is not just shorthand for the disturbance in the EM field as that is a classical effect. Its original meaning was in describing the dual nature of light, like the DeBrogile wave was used to describe the dual nature of matter.