Let us consider what forces are needed for a square orbit. As Newton pointed out, as long as there are no forces it will move in a straight line... so there must be no gravity along the sides. Then suddenly the moon turns 90 degrees, which implies a lot of force accelerating it. So there must be an enormous force just near the corners, and not along the sides.
This is awkward to achieve with real gravity. The gravitational force is a central force: each particle with mass exerts a $GMm/r^2$ force directed towards it, and you cannot shield from this by putting over masses in front: all the contributions from different mass particles sum together. So you cannot have just gravity bending the trajectory 90 degrees at the corner, since the gravitation from there will also affect the trajectory along the edge.
A general thing is that a complex shape of a planet produces a gravitational field that can be expressed using spherical harmonics. These tend to decay fast with distance if they are high frequence/sharp ("higher order"): weird planet shapes only affect very nearby orbits.
A four planet trick
If you allow for four fixed pointlike "planets" in a square I think one can prove there exists a nearly square orbit. Think of the moon approaching one of them, with the impact parameter $b$ (how far from the straight collision course it starts) a free variable. If $b$ is too large, the trajectory will just bend slightly and sweep past. If $b$ is too small you get more than a 90 degree turn. By continuity there is some $b$ that gives an exact 90 degree turn. That means, by conservation of energy, that it will move with exactly the same speed as it started when it gets far away from the planet. So, we can arrange that it does the same with the next, next and next planets and return to the starting point. The result is an orbit that is like a smoothed square. But it is not so much an orbit around a planet.

There is a subtlety above: the influence of the other three mass points will be felt at all places, so the gravitational bending is not going to be the perfect 2-body encounter I am assuming. Inside the Hill sphere of the planet its gravity dominates over all others and 2-body dynamics is a good approximation. However, proving the existence of the closed orbit requires more. Fortunately this is a continuous situation: if we color the starting points of the moon by how close they get to the second planet at closest approach, there will be some point that reaches the right distance to do a perfect 90 degree turn. Near that point, if we instead color by distance to the third planet, there will be an optimal point that causes three near-90 degree turns. Using the same method for the last planet and the starting point I think one can convince oneself that such an orbit must exist. Things are slightly trickier, since we should also do the same for how velocities change: we want to find a fixed point of the mapping $f:(x_{start},v_{start})\rightarrow (x_{end},v_{end})$ created by the planets so that $f(x,v)=f(x,v)$ (technically, finding a fixed point of the Poincare map). Doing this analytically is likely a nightmare, but one can use software optimization methods.
My plot above was made by starting with a region of initial values selected by hand, finding the one trajectory that got closest to its initial condition (in terms of position and velocity), zooming in on that to find even better starting values, and so on.