Methodology: how to deal with non-rigorous derivations This is a reformulation of https://physics.stackexchange.com/questions/238371/why-are-non-rigorous-derivations-in-classical-mechanics-and-electromagnetism-tau
which got closed as off-topic because it was formulated in an opinion based way. I think the underlying question is important though and I will try to reformulate it in a more general way.
In physics we are often faced with arguments that lack mathematical rigor. Although this is often unsatisfying, in many situations it is necessary to progress with the physics. On the other hand apart from being unsatisfying one can run into trouble by being not rigorous enough, i.e. the missing rigor might have obscured the actual physics.
The question is: How do experienced physicists deal with such situations? Is there any general advice on how to tell when mathematical rigor is necessary or does every situation have to be assessed individually?
Examples:


*

*A classic example is the use of Fourier transform methods in solving differential equations (in particular in electromagnetism and Newtonian gravity). Technically in 3D the Fourier transform of $\frac{1}{r}$ does not exist, nevertheless it is often taken to be proportional to $k^2$ (which is obtained by ignoring an infinite term in the integration, also known as regularizing the integral). This works fine for most cases, on the other hand there are derivations where one has to be very careful since otherwise factually wrong results are obtained (e.g. the inversion of the Poisson equation for a disc in Newtonian gravity is one example).

*In the closed thread given above is an example of a derivation that is rather intuitive but hand-wavy to some extend. How useful is it?

*An example I had in a second year maths course on differential equations was concerning the uniqueness of solutions to Laplace's equation for certain boundary condition. A commonly used theorem is the uniqueness of solutions under Dirichlet and von-Neumann boundary conditions. But working through exercises I found an example of an equation that had 2 distinct series solutions under such conditions. My supervisor confirmed this and could not explain the phenomenon. (I don't have the exact question anymore and I am also not asking to solve this particular problem. This is just for illustration of my question.)


In an answer I would be looking for an answer to the highlighted question with possible reference to any of the examples if necessary/possible.
 A: Let me try to answer without expressing any opinion.

How do experienced physicists deal with such situations?

This depends on the field of physcis and the physicist in question. Mathematical physicists for instance work with physics as a mathematical theory. They want to build up physics from axiomatic principles and they will require rigor everywhere and everytime (note however that mathematics is not rigorous itself - see for instance the essay "On proof and progress in mathematics" by Bill Thurston). 
For them, your examples are good reasons to be rigorous. All of the problems that you point out could lead to some physics that is not explained otherwise.
Most phenomenological theoretical physicists however want to explain physical phenomena and correctly predict the behaviour of physical systems. This means that they don't care about mathematical rigour. It is assumed that everything that works can be made rigorous if one wants to (but one doesn't). If they believe their ansatz is right but the result is patently wrong, only then will they have a closer look at the mathematics - after all, what counts is the experiment.
For them, your examples are good reasons not to be rigorous. All of the problems that you point out don't lead to interesting physics since the results already explain the physics they are interested in.
All other physicists will be somewhere in between.

Is there any general advice on how to tell when mathematical rigor is necessary or does every situation have to be assessed individually?

First question: From the behaviour of physicists, can we get general adivce? No, because everything goes as illustrated above. 
Second question: Is general advice given by physicists? When I took classes and encountered the problem, any advice given was generally "if it ain't broken, don't fix it". The question what "broken" means was then the opinion of the author. I have seen a lot of disagreement by physicists and mathematicians about the question and I have witnessed instances when a mathematician could prove that a result obtained by non-rigorous methods that was previously thought to be correct was false and more often, I have not witnessed this happening.
Third question: Could general advice be given at all? Is there a way to do this "right"? 
Yes, mine. Jokes aside, this question is much more complicated than you might think: One of the main question you have to answer before answering the question is whether you expect that physics should obey a mathematical theory. In other words, you have to answer the question how well mathematics could describe physics at all. Then you have to answer the questions whether "physics" tries to explain the phenomena around us or whether it tries to illuminate the mathematical structure of the universe (if that exists). Based on these purely philosophical questions can you start to give general advice. These are questions to which the mathematical physicist has a different answer than the phenomenologist.
