A gravitational field can be thought of as a Potential field. It involves vector calculus, so if you haven't read up on div(ergence), grad(ient) and curl, it's going to sound like gobbledy-gook :)
Put as simplistically as I can, imagine the gravitational potential energy of an object at various places above a planet. If you had a function that took a single point in space (say, latitude, longitude, and altitude) and returned the potential energy for it, you'd have something that looked like $f(lat, long, alt) = \text{potential energy}$. $f$ would be called a "potential field".
Notice that the potential field returns just a scalar (that is, just a single number and not a vector). Suppose we wanted to know in which direction an object would move if we released it from a given point. In this case, we'd take the "gradient" of the potential field, which would return a vector telling us in which direction, and by how much, an object would accelerate.
It turns out that a lot of different forces in nature act like this. Liquids that have no viscosity (that is, really "thin" fluids) can be described using a potential field. So can the electrical fields from point charges, like the classical model of electrons and protons.
Because the math is the same for all these different forces, it suggests that they are conceptually the same as well; if we can understand one, we can get a good handle on the others. Imagine something like a bathtub draining water out from the drain. Anything floating on top of the water surface will accelerate towards the drain. Objects further out from the drain will accelerate slower than objects closer to the drain. Just like how gravity works!
However, this conceptual model fails on two counts: first, the bathtub water will also start spinning as the it "circles the drain". Objects caught in this "vortex" will start to orbit the drain. You'd think this would match gravity, but it actually doesn't. The spin is caused by friction between water molecules, and it acts to remove energy from the system. Eventually objects caught in the bathtub vortex would get pulled in towards the drain. But in space, there's no friction, and planets orbiting a star don't get slowly pulled in to it. This spinning is called the "curl" of the field. Gravity has no "curl". Neither do electrical fields. Fluids do, which is what gives fluids their tendency to "swirl". Note how objects in a potential field can still orbit, even though the field itself has no curl.
Second, this model would suggest an obvious question: if gravity is like a bathtub slowly draining out, what is getting sucked (presumably space itself?) and where is it going? But this is simply a case where our simple model breaks down. Mathematically gravity is just creating a potential field, the gradient of which produces acceleration; there's no need for the potential field to balance out or anything like that. In fact, if nothing is moving, the potential field won't change either, even though it can still accelerate objects caught in it.
The above explanation is highly simplistic, but it should help the metaphor make sense without using too many convenient math lies :) Sorry for the length.