Is the concept of a field necessary to electrodynamics? I've read (in Griffith's text) that it is "possible, though cumbersome" to dispense with the field concept in electrodynamics entirely and instead use an action-at-a-distance theory. 
What exactly is meant here? Since fields physically exist, why is this permissible?
Edit: In light of the answers below, I'm curious about the apparent contradiction here - if fields physically exist, then how can one dispense with the field concept?
 A: The reason one might expect such a description to be permissible is that there are a lot of formal similarities with the equations of Newtonian gravity, which is an action-at-a-distance theory where it's possible to dispense with the inter-particle force concept and instead use a field theory. This is permissible even though the gravitational field doesn't physically exist, because the two procedures necessarily yield the same values when you try to compute an object's acceleration.
When they realized E&M fields store and transmit energy and momentum, people quickly began to accept them as physical realities. This sparked a lot of interest in looking at Newton's law of universal gravity in a new light. The goal was to either to modify it to bring it more in line with the format of Maxwell's laws, or at least isolate any fundamental reason explaining why you can't. A young German physicist came up with a solution to this problem and an improved theory of gravitation in 1915. 
A: Even though that's not what we're usually taught, it is possible, in principle, to formulate the theory of electric charges without the use of EM fields, as Feynman and Wheeler tried to do in the 40's. The only thing is that it is not really practical because action at distance does not mean instantaneous action at distance, and one has to keep the delay in the interactions in some way.
The reason why we can do without EM fields is because one never really measure directly the field, or light, but always the motion (or change of motion) of a charged particle due to the force exerted by another charged particle (that might be billions of light-years away).
Of course, it's a nightmare to try to think (and do calculation) that way, and using EM fields is much more practical. But this point of view also asks some interesting questions: for example, if there were only one star in the universe, would it emits light, since no other electron far away would be able to feel the force induced by this light...
A: By definition fields are unobservable and all the electromagnetic phenomena that people attributes to fields can be attributed to particles; that is the underlying reason why one can develop an action-at-a-distance electrodynamics without fields.
I would stress that the name action-at-a-distance is a misnomer that has generated much trouble and discussions in the past. Currently we prefer the term direct-particle action for such formulations.
In such formulations the electromagnetic fields aren't real and appear only as a mathematical device for describing part of the direct interaction between particles. It is possible to start from the Hamiltonian for a group of particles with direct interaction $V$ and obtain an analog of the field theoretic Hamiltonian plus correction terms. For instance for low velocities the interaction is reduced to a Coulomb term which can be written as
$$ V = \frac{1}{2} \int \boldsymbol{E}_\mathrm{L}^2 \mathrm{d}V  - \frac{1}{2} \sum_i \frac{e_i^2}{4\pi\epsilon_0 |\boldsymbol{r}_i -\boldsymbol{r}_i|}$$
or in a compact form as
$$V = V_\mathrm{field} + V_\mathrm{countertem}.$$
I agree with Griffith on that the Feynman and Wheeler formulation of electrodynamics is "cumbersome", but it is not worse than the traditional field formulation of electrodynamics with all its internal inconsistencies and unphysical parameters.
A: If I am not mistaken, action at a distance is a manifestation of the existence of a field.
In one respect, the answer to your question is yes, i.e., you can have a mathematical framework wherein you can describe electrodynamics without making use of the notion of vector fields. However, this is not in line with reality, where such fields do really exist and describing them correctly adds to a clearer comprehension of what is actually going on. 
A: 
What exactly is meant here? Since fields physically exist, why is this permissible?

What is meant is that if fields satisfy some simple boundary condition, for example, if they are all retarded, we can write down the equations of motion of the charged particles directly in terms of particle coordinates, velocities and accelerations. Whether fields "physically exist" is not easy to decide, since they can be mathematically eliminated from the description. Much safer and sufficient is to assume that fields are convenient mathematical objects and to use them as such without any ontological assumptions.
