The short, but very true answer, is that this doesn't happen. What you describe is a discontinuity. There's gravity at one moment, and no gravity the next. To the best of our understanding, the universe continuous (or at the very least is well modeled at the macroscopic level as continuous... quantum mechanics is a more nuanced topic). Thus, any discussion of a discontinuity like you describe is nothing more than a mathematical oddity. It is outside of the domain of what the laws of physics were supposed to model.
Now I can think of two examples which might help your intuition. One is a real life effect, and the other is well thought out fiction.
We do have examples of fields "collapsing." When a field collapses, it goes from being very strong to very weak, or even disappearing all together. This is very similar to the situation you describe with switching gravity off with one key distinction: the effect occurs over time. It takes time to collapse a field. During that time, there is an opportunity for work to be done, which accounts for the conservation that you intuitively believe is missing.
When we stop an electric motor, an interesting effect happens, known as back EMF. When the motor is running, it has a strong magnetic field, generated by electromagnets. This field is responsible for transmitting the energy used to keep the motor spinning. However, what happens when we turn the circuit off. When you do, the magnetic field begins to collapse, and a funny thing happens. Because this collapsing field is a changing magnet field, it induces a current flow inside the wires of the electromagnet. If you anthropomorphize a bit, the electromagnet "wants" to keep the field going, so it generates current in the opposite direction to avoid a discontinuity (the real reason involves more of Maxwell's equations and less desire on the part of the circuit, but this is sufficient for the story).
However, this current has to go somewhere, and you just switched the circuit off. That wire no longer has anywhere to send the current. It looks like we're about to have a discontinuity, just like in your example. What really happens is that the electromagnet generates enough voltage to generate a spark across the contacts of your switch, completing the circuit. This spark persists until the field is done collapsing, and it dissipates all of the energy from the magnetic field.
Back EMF is a big deal for circuit designers. Needless to say, we really dislike having arcing electricity where arcs don't belong. The wear our switches out and do nasty things we didn't expect. The solution is that we often put a high-power low-resistance resistor in parallel with the electromagnet. This is a little inefficient, because we lose power to the resistor during operation, but when I suddenly open a switch, turning off the motor, it now has a very easy low-resistance path to dissipate all of that energy without it having to go through my switch!
So that's what real field collapse looks like, but you were specifically thinking of gravity. For that, I'd like to bring up a fictional example: Portal.
In the game Portal, you have a gun that can connect to places in space together, so you can drop objects through your portal and accomplish amazing things. In portal, the rules of physics are simple: gravity always points down and velocity is conserved through the portal. It's easy to show these rules are inconsistent with our universe, because you can build things like perpetual motion machines with these rules.
However, the game was popular enough that some physicists decided it'd be fun to work out the mechanics of the portal gun. Was it actually possible to have a portal gun without violating the known laws of physics? The answer turned out to be yes. If you were very careful with the laws of gravitation, you could prove that there was a continuous gravity field which fit the laws of physics and permitted portals. If you really get down to it, all this woudl be is wormhole physics.
The rules of Portal permitted perpetual motion. The rules of our world did not. The place where they diverged was that Portal assumed gravity was always "down," while the real life physics recognized that gravity would operate through the portal, pulling you through it as you get close. They worked out the equations and proved that you could define a gravitational field where there was a single value for the "gravitational potential energy" of any object. It actually worked out!
The catch: switching the portal on was difficult. Your hypothetical experiment involved switching a gravitational field off; their hypothetical experiment involved switching one on. What they found is that, to be consistent with the conservation of energy, that portal gun needed to emit a huge amount of energy into the universe to account for all of the changes in potential energy caused by stitching space together into knots. That gun had to have quite the power supply. There's a reason Aperture Science went through so many volunteers!
Turning a portal off had the same issue you describe. For their hypothetical portal gun to work, when you turned a portal off, any potential energy that was lost as gravity collapsed had to go somewhere. Presumably that gun had to absorb it... which might explain why suddenly funding for Black Mesa and Aperture science become highly unpopular in the game. That gun had to sink amounts of energy on a planetary scale!
So, in the real world, we can't just flip off gravity. It simply doesn't happen. However, there are fields we can flip off, such as electromagnetic ones. When we flip them off, the energy does go somewhere; energy is always conserved. We can also see that the mathematics can work for collapsing gravity fields as well, although such work is entirely hypothetical.
(Note to anyone working on non-hypothetical gravity field collapsing devices. Valve made a game for you to play. You really need to go play Portal 2 before turning your non-hypothetical device on. Pretty please?)