Field lines in case of charges go from +ve to -ve but incase of magnet, they dont start or stop anywhere. They form closed loops. Is this consequence of the fact that single poles dont exist or something else is going on here?
Yes. In an alternative universe where north and south charges can exist independently just as positive and negative charges can exist then magnetic field lines could start and end at these charges. In such a universe the alternative maxwells equations become symmetric, moving electric charge generates magnetic field and moving magnetic charge generates electric field. However, in our universe these magnetic charges do not exist, and as such magnetic field lines close up.
In this answer I explained that macroscopic magnetic and electric fields are created by the alignment of the magnetic dipole moments of subatomic particles respectively a charge separation.
... incase of magnet, they (the field lines) don't start or stop anywhere. They form closed loops. Is this consequence of the fact that single poles dont exist or something else is going on here?
In a closer view the magnetic field of a permanent magnet is the sum of the aligned magnetic dipole moments of the involved electrons and protons. They form a common magnetic field. Nature in our surrounding is made of two kinds of electric charges and of closed magnetic loops from subatomic particles. Nature simply do not provide magnetic monopoles. Only understanding what’s going on inside electrons and perhaps improving the quark model it maybe would be possible to solve the question about magnetic monopoles.