Will two perpendicular orbits settle into a disc? Scenario:


*

*one "fixed" object (like the sun...) of mass X

*two "planets" (P1 and P2) of mass Y

*P1's orbit is perpendicular to P2's orbit, and the sun is the center of both orbits

*P1 and P2 will never collide


My question is: Will this setting (P1 and P2 orbits describe a perfect circle in moment 0) ever be permanently "disc"-like (as we observe in galaxies, and not transitory)?
Is there a way to simulate this? Like a program or maybe something like wolfram-alpha?
 A: Note that circular orbits never exist in nature, and that the scenario that you described would be very rare.  The reason why most solar systems have planets with coplanar orbits is because these planets were formed from an accretion disk, which as the name implies, is a very thin (but dense) cloud of dust and debris orbiting a star.  That being said, it does stand to reason that the gravitational interactions between the planets would gradually (i.e. millions of years) pull them into coplanar orbits.  I would test this hypothesis using the Universe Sandbox, a downloadable planetary physics simulation.  If the $10 cost is off-putting, it should not be too hard to code your own simulation.
Note:  For further information, see Two orbiting planets in perpendicular planes
A: I think that a disc of orbiting material is something that arises naturally when you have enough objects to do statistics with, but not necessarily when you have only a handful of masses as in your example.
I happen to be playing with an orbit simulator lately, so I put two Jupiter-mass objects in perpendicular circular orbits 1 AU from a solar-mass central body. I had the planets start 2 AU apart; twice per orbit (1/4 and 3/4 of the way around) they approach to √2 AU apart. After just a few dozen orbits, this regular attraction has changed their orbital periods by about 5%, so that they "catch up" to each other and approach within 0.2 AU before moving apart again. In the figure below you can see that after each close approach, both orbits become more eccentric. You can also see that after only a few hundred orbits the orbital planes of the two planets are no longer quite perpendicular.

(You can see the density of plotted points change after year 220; this was a change in how often I wrote the state to disk, but not in the internal step size.)
If we let the simulation keep running, however, it changes dramatically. After another few hundred years the ellipticity of the orbits starts to vary continuously, rather than in jumps. The orbits swap which is "horizontal" and which is "vertical" every few thousand years, but not periodically. Occasionally there's a close encounter which makes the orbit eccentricity change quickly. After about 25000 years, such a close encounter ejects one of the planets from the system.

This is the kind of thing that people mean when they say "three-body systems are chaotic." My initial condition did have broken symmetry at the part-per-thousand level, but I don't think that was an important detail.
The relative masses matter. If I run the simulation with the same initial positions and velocities but with earth-mass planets, the close approaches happen every thousand years instead of every few dozen years, and after a quarter-million years the eccentricities have only grown to about $10^{-5}$. However I'm quite confident that system is chaotic as well.
