What's the difference between a perfect fluid and an ideal gas? This is how I understand it at the moment:

*

*A perfect fluid is a collection of non-interacting particles, which are as a whole characterised by energy and pressure.

*An ideal gas is also a collection of non-interacting particles, but here the ideal gas law holds. There, we have pressure, volume and temperature (let's assume a fixed number of particles for both cases), s.t. by applying the ideal gas law, again two parameters remain.

Furthermore, the stress-energy tensor of a perfect fluid can be seen e.g. here on Wikipedia, but I haven't found the stress-energy tensor of an ideal gas.
 A: 
An ideal fluid is a fluid that is incompressible and no internal resistance to flow (zero viscosity). In addition ideal fluid particles undergo no rotation about their center of mass (irrotational). An ideal fluid can flow in a circular pattern, but the individual fluid particles are irrotational. Real fluids exhibit all of these properties to some degree, but we shall often model fluids as ideal in order to approximate the behavior of real fluids. When we do so, one must be extremely cautious in applying results associated with ideal fluids to non-ideal fluids.

Italics mine.
The ideal gas law:

The ideal gas law states that $PV = NkT$, where P is the absolute pressure of a gas, V is the volume it occupies, N is the number of atoms and molecules in the gas, and T is its absolute temperature.

It is compressible because the volume depends on the pressure and temperature.
Of course real gasses and real fluids have deviations in their behavior,but I think the main difference leading to different mathematical models is compressibility.
A: A perfect, or ideal fluid is also incompressible, but an ideal gas is not. Pressure and volume changes on an ideal gas can cause changes in its density.
A perfect fluid is described by an irrotational velocity vector field $\bf v$, so that $$\nabla \times \bf{v} =0$$ and this is not necessarily true for
ideal gases.
The molecules of an ideal gas interact with each other via elastic collisions, and the motion of these molecules is random. The behavior of ideal fluids cannot be described by the ideal gas equation.
While perfect fluids do not have interactions between their molecules, these molecules do not move randomly, but are held together to a fixed volume.
Hence, the behavior of ideal gases and ideal fluids will be different.
Also, in the context of the stress-energy tensor mentioned in your question, perhaps an ideal gas is taken as a simple example of a perfect fluid. An ideal gas can be modelled by a stress-energy tensor which is one described by the energy density and pressure components only. Example of such modelling.
A: Contrarily to the existing answers, and to the citation in one of them, the terminology is not uniform. According to the answer to a previous similar question and to Landau&Lifshitz textbook on Mechanics of Fluids, a perfect fluid is a fluid described by the Euler equation, continuity equation, without viscosity and thermal conduction. It may be compressible. Basically, according to this point of view, the perfect character of a fluid pertains to its dynamics.
The ideal gas was introduced as a prototype system in thermodynamics, and statistical mechanics of equilibrium. Therefore rheological properties are not directly related to this model, although in some cases (for instance within the kinetic theory of gases) some model for the transport coefficients is added to the ideal gas approximation.
