A planet with a square orbit? To understand how gravity influence objects, time and space, I have been thinking of how a planets shape would change the orbits of its moons.
More specifically: can I design a planet whose moon move in a square orbit?
Below is a diagram of my first intuitive try. For simplicity I imagine a two-dimensional shape extruded with the moon moving around it.
1. Is there in theory a shape that would create a square orbit for objects moving around it?
2. If yes, what is that shape?

 A: The 1st thing to look at would be the velocity. Non-zero velocity would imply a smooth orbit; therefore your moon would have to slow down to complete stop at the corner before accelerating into an orthogonal direction.
The 2nd thing to look at would be the acceleration. As the moon approaches, say, upper right corner the acceleration ought to be downward to slow it from the strictly vertical velocity before the corner to the complete stop. And right after the upper right corner the acceleration has to be horizontal to accelerate the moon strictly horizontally.
Moreover, acceleration cannot be zero at the corner: Newtonian's law are 2nd degree, so if an object has zero velocity and acceleration it's not going anywhere. So the moon's acceleration is discontinuous at the corner, from non-zero vertical to non-zero horizontal.
The next question is, can acceleration caused by Newtonian gravity be discontinuous? Well, if your Planet X has finite density then certainly no. Moreover, if your arrangement doesn't include infinite density singularities placed at the corners of the moon's orbit the answer is still no, the reason's being that Laplace potential would have a smooth local solution outside singularities.
A: A sharp corner indicates infinite curvature, so the ratio of the acceleration to the velocity would have to be infinite, which implies that the velocity is zero. Besides the instability of that requirement, that means that as it's approaching a corner, there must be an acceleration away from that corner (that is, the speed is decreasing, so the acceleration vector is opposite the velocity vector). However, that would mean that the net force is tangential to the its displacement vector relative to the planet, but gravity is always in the same direction as the displacement vector.
I don't think there's any way to do it without bodies outside the orbit supplying a force. But if the bodies are outside the orbit, then the force would presumably increase as the object gets away from the center of the orbit. This conflicts with the need for the force pushing the object away from the corner to go to zero as it reaches the corner.
And of course any planet-sized object with the shape you depict would quickly collapse under its own gravity; the force on the corners would be tremendous.
A: Just throwing ideas out here but what about this:

If your planet looks a shell of sufficiently high mass then we have that inside the shell there is no gravity and outside there is high gravity. In the region without gravity the path of the moon will be straight which would allow it to form a nice square. If we punch a couple holes in your planet then the moon could occasionally peek outside where the high gravity causes its path to redirect inside.
A: Let us consider what forces are needed for a square orbit. As Newton pointed out, as long as there are no forces it will move in a straight line... so there must be no gravity along the sides. Then suddenly the moon turns 90 degrees, which implies a lot of force accelerating it. So there must be an enormous force just near the corners, and not along the sides.
This is awkward to achieve with real gravity. The gravitational force is a central force: each particle with mass exerts a $GMm/r^2$ force directed towards it, and you cannot shield from this by putting over masses in front: all the contributions from different mass particles sum together. So you cannot have just gravity bending the trajectory 90 degrees at the corner, since the gravitation from there will also affect the trajectory along the edge.
A general thing is that a complex shape of a planet produces a gravitational field that can be expressed using spherical harmonics. These tend to decay fast with distance if they are high frequence/sharp ("higher order"): weird planet shapes only affect very nearby orbits.
A four planet trick
If you allow for four fixed pointlike "planets" in a square I think one can prove there exists a nearly square orbit. Think of the moon approaching one of them, with the impact parameter $b$ (how far from the straight collision course it starts) a free variable. If $b$ is too large, the trajectory will just bend slightly and sweep past. If $b$ is too small you get more than a 90 degree turn. By continuity there is some $b$ that gives an exact 90 degree turn. That means, by conservation of energy, that it will move with exactly the same speed as it started when it gets far away from the planet. So, we can arrange that it does the same with the next, next and next planets and return to the starting point. The result is an orbit that is like a smoothed square. But it is not so much an orbit around a planet.

There is a subtlety above: the influence of the other three mass points will be felt at all places, so the gravitational bending is not going to be the perfect 2-body encounter I am assuming. Inside the Hill sphere of the planet its gravity dominates over all others and 2-body dynamics is a good approximation. However, proving the existence of the closed orbit requires more. Fortunately this is a continuous situation: if we color the starting points of the moon by how close they get to the second planet at closest approach, there will be some point that reaches the right distance to do a perfect 90 degree turn. Near that point, if we instead color by distance to the third planet, there will be an optimal point that causes three near-90 degree turns. Using the same method for the last planet and the starting point I think one can convince oneself that such an orbit must exist. Things are slightly trickier, since we should also do the same for how velocities change: we want to find a fixed point of the mapping $f:(x_{start},v_{start})\rightarrow (x_{end},v_{end})$ created by the planets so that $f(x,v)=f(x,v)$ (technically, finding a fixed point of the Poincare map). Doing this analytically is likely a nightmare, but one can use software optimization methods.
My plot above was made by starting with a region of initial values selected by hand, finding the one trajectory that got closest to its initial condition (in terms of position and velocity), zooming in on that to find even better starting values, and so on.
A: Unfortunately such an orbit is not possible. Consider Newton's second law: $\vec F = m \mathbf{\vec a}$ .  Along the side of the square, the acceleration $\mathbf{\vec a}$ is a vector pointing in the direction of the orbit, but the force must be pointing somewhere in the direction of the planet, so you can't have an equality.
A: If you just want a body to have a stable-ish square path relative to a parent system's barycenter, this can easily be done with a retrograde orbit. Illustration of concept:

I drew this as a planet in a binary star system, but it doesn't necessarily have to be— Be warned that the body the green planet or moon is orbiting does need to be big enough to have a mass and sphere of influence sufficient to capture the green planet at an appropriate orbital distance and period, though.
Run a Fourier transform or something like that to get the orbital parameters you'll need, and then solve for the mass of each body.
In fact, you may even be able to create significantly more complicated stable-ish orbits than a simple square:
Homer Simpson's orbit: A fun application of the Discrete Fourier Transform

(Not everything will be possible due to the constraints on mass and SOI/Hill Sphere. But there's potential.)
So… To frame it as in the question: You can't really have a single shape that will produce a square orbit for a moon around it, but you can probably have a simple orbital system where a moon can be added such that it will take on a squarish trajectory.
UPDATE: The radius and period required may not be very stable, or on the edge of stability. See the comments by @rob.
A: In reality you can't have a spiky planet as you have drawn. The peaks will collapse under the gravity of the planet. And there's no known way that a "four planet trick" as described by Anders Sandberg could remain stable in reality. A planet's shape can have only very minor influence on it's satellites orbit. The bulge at the earth's equator (which is caused by the centrifugal effect of the earth's rotation) does however influence the precession of orbits of some artificial satellites. This is used to advantage in the "sun synchronous" orbit, where an artifical satellite's polar orbit precesses once a year.
There are however five "Lagrange points" which orbit together with a planet around its star, where the gravity of the planet and star have equal influence. This enables an object to orbit with the planet while being distant from it. Three of these, L1 L2 L3 are in line with the planet and star. Artificial satellites have been put into such orbits, but require station keeping as they are unstable. The other two, L4 and L5, lead and lag the planet by 60 degrees and objects can remain stable in these. Thus there are two groups of asteroids, the "greeks" and the "trojans" leading and lagging Jupiter in its orbit around the sun. These asteroids "librate" about the Lagrange points.
Then there are the "Hildas", a group of asteroids which orbit the sun 3 times in the time that Jupiter orbits twice. The orbits of these asteroids are really elliptical. But from the point of view of an observer at Jupiter, they appear to cover a triangular path, once for every 2 orbits that Jupiter makes of the sun. The orbits remain synchronous as the asteroids interact with the three Sun-Jupiter lagrange points L3 L4 L5 which are mutually 120 degrees apart.
In principle an artificial satellite in a 2:3 resonance could be put in place between Earth and the moon. It would appear to an observer on the moon to follow a triangular path.
A: You can definitely have a system with any shape orbit. Try designing your square orbit solar system as a planet orbiting a binary star. Who cares about stability for now. Most non elliptical orbits probably become unstable by now, but that does not mean all sorts of orbits do not have there day. All things are possible. Circular orbits are just common because they are stable. The other shapes would be destroyed easier.
