What is the difference between a physical theory and mathematical theory? Is there same approach in physical theories like primitive notions,axioms and theorems as in maths ? If it is then is it an only method or approach to study different fields of knowledge?
 A: 
What is the difference between a physical theory and mathematical theory

A mathematical theory is self contained. It starts with axioms, develops theorems and a statement can be proven correct or false   with mathematical tools.
A physical theory is a subset of  a mathematical theory, the subset is picked by additional "axioms" which are called laws or postulates, and are a distillate of physical measurements. A physical theory can only be falsified by disagreement with measurements/data; it cannot be proven to be correct.
Example is the theory of quantum mechanics which depends on a number of postulates. These, from the mathematical theory of differential equations, choose that subset  which describes experiments at the quantum mechanical level, and , very important, are predictive.
A: In mathematical physics, theories while perhaps highly abstract (e.g. formulating a gauge theory entirely in fibre bundle language) usually are laid out in a manner different to mathematics.
However, that said, there are axiomatic formulations of various theories in physics. For example, in quantum field theory, one can assume the Wightman axioms:


*

*There exists a physical Hilbert space $\mathscr H$ in which a unitary representation of the Poincaré spinor group acts, and there exists a state $|0\rangle$ invariant under translations.

*The spectrum of the energy-momentum operator $P$ is concentrated in $V^+$, that is, the upper part of the light cone.

*The quantum fields are operator-valued generalised functions over the Schwarz space with some domain $D$ (which contains $|0\rangle$), which is dense in $\mathscr H$. 

*$U(a,\Lambda)\phi_i(x)U(a,\Lambda)^{-1} = V_{ij}(\Lambda^{-1})\phi_j(\Lambda x+ a)$ where $U$ is a representation of the Poincaré spinor group and $V$ is a matrix representation of $SL(2,\mathbb C).$

*Any two field components either commute or anti-commute under space-like separations and the set of combinations of the form $\phi_{i_1}(f_1) \dots \phi_{i_n}(f_n) |0\rangle$ is dense in $\mathscr H$.


One can take this approach to quantum field theory; for example it has been done for massive $\phi^3$ theory as well as the Thirring model to name a few. However, this approach is not necessarily forced on us as it is in mathematics. 
If your intent is to compute scattering cross-sections for example to hand over to an experimentalist, you will be quite content doing things in the manner of say, Peskin and Schroeder's quantum field theory. However, if you'd like to prove a general statement within a theory, you may resort to an axiomatic approach so it is clear what is assumed.
This was done in the case of the famous no-go theorem by Coleman and Mandula in All Possible Symmetries of the S-matrix and in fact the axioms were an important part of the paper, because it was clear from them that the theorem did not restrict super-symmetry.
A: 
Is there same approach in physical theories like primitive notions, axioms and theorems as in maths?

No, it is not. In mathematics all is derived from axioms. 


An axiom or postulate is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments. ...used in deduction to build a mathematical theory. Axiom from Wikipedia


The point is that the deduction process


... is the process of reasoning from one or more statements (premises) to reach a logically certain conclusion. Deductive reasoning from Wikipedia


Comprehensively in mathematics all postulates (axioms) have to be proofed and until there is no inconsistency the theory will be accepted. Even the change of some fundamental properties is possible.
So the Euclidean axiomatic system was only as long right as long a non-Euclidean geometry starts dealing with non-flat surfaces. The clue is that non-flat surfaces are the "normal" surfaces (our earth is a sphere) and flat surfaces are only an approximation of non-flat surfaces for "small" distances.
What about physics?


*

*The approach is a little bit more vague. Mostly we observe chaotic processes and derive from statistics some laws. Going in detail it was observed that all happens on the atomic and subatomic level. For example heat transfer could be described by radiation or more in detail by photon emission and photon absorption.

*Instead of axioms or postulates physics mostly uses observations. Until such observations could be repeated in the range of some accuracy there is no need in change of the derived law. But the history of science shows that physical laws are changing time to time.

*Indeed there are some theories that are deduced from phenomena and physical properties. Einsteins General relativity theory is such a thing. Plancks law about the Black body radiation, announcing the quantization of radiation, is such an astonishing thing. Or if you need an example for contradicting theories, take Newtons Particle theory of light against Huygens Wave theory of light.



If it is then is it an only method or approach to study different fields of knowledge?

So no, axioms are not the mostly used approach in physics. Mostly observations are used. But an axiomatic Ansatz is possible.
A: 
Is there same approach in physical theories like primitive notions, axioms and theorems as in maths ? 

Yes; necessarily, since these (along with "definitions") are constitutents of a theory in general.

What is the difference between a physical theory and mathematical theory?

Physical theories explicitly and specificly involve "empirical determinations"


*

*as primitive, self-evident notion (the gathering of "observational data"; such as, especially, "determinations of space-time coincidences of material points", and therefore also the distinction and recognition of "identity" of any such "material point" or "participant"), or

*by way of definitions, as "physical quantities" whose values are to be determined (measured), trial by trial, by applying the defined measurement operator to the available observational data.
In contrast, mathematical theories would refer to (Boolean, or numerical) values abstractly, or as primitive notions; perhaps going as far as considering simulations of empirically determined values.
But gathering actual observational data, and evaluating it to obtain results of actual measurement, is a methodology left to empirical science, carried out by experimentalists.
p.s.
Note the distinction between theories and models, especially pertaining to physics:
Any theory, i.e. any framework of primitive or self-evident notions, of axioms and definitions, and of their logical consequences or theorems is manifestly either consistent, or inconsistent (and therefore invalid) by itself; without involving any reference to actually gathered observational data, or to actually (through application of the defined measurement operator to actually gathered observational data) measured values.
In other words: a theory could not be considered falsified (and consequently be discarded) due to any empirical results; neither due to those specific results which would have been obtained through application of this very theory in the first place (and which arguably ought to be discarded, too, if the very theory were being discarded, based on which these results had been obtained), nor due to results obtained by applying any other theories (and measurement operators of their definition, to observational data of their framing). 
What can be empirically tested, and possibly be found false, are instead (physical) models, i.e. collections of specific values of the defined physical quantities, summarizing all values which had been obtained already (from preceding trials) as well as specific expectations/predictions which values (or at least: range of values) might be found in subsequent trials (incl. subsequent evaluation of observational data that had been already "in stock").
Again: the completely ordinary case that some particular value of a defined physical quantity has been obtained by evaluating the observational data of "the latest" trial, and that therefore plenty of models have been found falsified (namely all those which had predicted some different value, in this trial), while some other models ("standard models") still remain corroborated, has no consequence at all on the validity, comprehensibility, applicability of the very theory which provided the definition (as measurement operator) of the physical quantity under consideration. A theory prescribes the range of values of a quantity; a model specific values, trial by trial.
However: given observational data may generally be evaluated in several different "ways" (i.e. by applying differently defined measurement operators; possibly even varying the prescription of what constitutes "one trial worth" of observational data). Especially if the available observational data is not suitable for obtaining values of some particular quantity (i.e. if the trials under consideration were not "eigenstates" of the corresponding measurement operator), then it may be possible and even advisable to seek or define some suitable other quantities (and given the need or opportunity, entire theories for this purpose), in order to extract some values from the available data after all.
