In an abstract sense, a "dimension" is just a component of a state vector. For example, one might talk about a 10-dimensional phase-space consisting of 3 components for position, 3 for linear momentum, 3 for angular momentum, and 1 for energy. Or one might have an "event" vector which includes an additional dimension representing time.
There are good reasons to believe that there is no 4th spatial dimension completely analogous to the 3 spatial dimensions that we are familiar with: if there were any way to move perpendicularly to space, then this would be happening all the time as a result of interacting with any object that was already moving in that direction. For example, consider that a 4-body system will never stay within a plane once disrupted because it is an unstable equilibrium. Perhaps such a 4th dimension exists, but it would have to have either a different topology, or there would have to be some sort of restorative force which keeps us confined to our hyperplane. The latter case is illustrated by a pool table -- there is a third dimension perpendicular to the table but the balls are glued to the table because of gravity and the counteracting force is provided by the table itself. There is an excellent book called Flatland which addresses these issues in an intuitive and accessible way.