Is there some formula I can apply to the weight of a gun, along with the force of thrust it creates, and calculate if it will be able to lift itself off the ground or not? Without actually handling the gun, of course.
Given the pieces mentioned in your question: the weight of the gun and force it creates, the answer is a trivial yes. If the force is greater than the weight, it could lift.
The problem is in calculating that force. A rocket is designed to produce force continuously for a period, and given the fuel and some other parameters makes it possible to calculate the average thrust over that period. The gun produces force only for a short period of time. As an example, it might generate a force in excess of its weight for only a couple hundredths of a second because the forces diminish as the gas in the firing chamber expands.