Lets start with the definition of free energy. Free energy is the maximum work you can get out of a system subject to thermodynamic equilibration under some constraints. These constraints could be constant $(T, P)$, $(T,V)$ etc. The constant part can be confusing but can be explained.
Before I do so, let me explain the difference between a thermodynamic potential, free energy, and work. Thermodynamic potentials are extensive state functions like, internal energy $U$, Helmholtz potential $A$, Gibbs potential $G$, Enthalpy $H$ etc. These potentials are state functions and depend on the system state (nothing to be held constant here) and turn out to be Legendre transforms of each other. Strictly speaking, free energy is the difference in the value of the potential in the non-equilibrium state and the equilibrium state. Therefore, free energy is a function of two states: the initial non-equlibrium state and final equilibrium state. Often this subtle detail is ignored and one tends to call the Gibbs function as the Gibbs free energy and so on. Well this is wrong but happens very often.
Anyways next difference is between free energy and work. Free energy is the maximum work one can obtain if the system goes to equilibrium in a reversible manner subject to constraints. Actual work in any system will be lower because of irreversible processes (friction, chemical reaction etc).
OK let's now derive Helmholtz free energy from internal energy. Consider that I have a rigid box with a partition connected to a shaft (to do work). There is the same gas on either side of the box but one side $A$ is at higher pressure and other side $B$ at lower pressure as shown in Figure below. 
The combined system is in mechanical non-equilibrium. If you let the piston go, it will provide work to the outside world while mechanically equilibrating the stuff inside the box. If you were to do this allowing external heat exchange such that you hold $T$ fixed and the total $V$ is fixed of the box, lets calculate how much work will you get. Lets write the balance equations for energy and entropy.
\begin{align*}
&[E]:\quad dU = -\delta W+\delta Q \\
&[S]:\quad dS= \frac{\delta Q}{T}+ \delta S_{gen}
\end{align*}
The $\delta S_{gen}$ is the entropy generated due to friction, chemical reaction, (if stuff was reacting), etc. So now lets substitute for $\delta Q$ from the second to the first equation and re-arrange.
\begin{align*}
& \delta W = -(dU-TdS+T\delta S_{gen})
\end{align*}
The maximum work you can get from this process is when everything inside is reversible, i.e., $\delta S_{gen}=0$
\begin{align*}
&\delta W_{max} = -(dU-TdS)=-(dU-d(TS))=-dA \;(\text{constant }T)\\
&W_{max}= -\int_{initial}^{final}{dA}= A_{init}-A_{final}
\end{align*}
So, if you process was at constant $T,V$ and reversible, you will have the Helmholtz free energy (difference in Helmholtz potential) as work.
Definition of Helmholtz potential $A$: The Helmholtz potential $A$ has been defined as $U-TS$ purely out of convenience to not have to repeatedly keep saying work is $-(dU-d(TS))$ at constant $T,V$. It has been defined as a useful grouping of variables.
The "constant" confusion: Many times you will find processes that happen in a constant $T, V$ setting where this definition is directly useful. But even otherwise one must realize that the free energy is only a measure of work from a reversible path so any reversible path that starts at some $T,V$ and ends at same $T,V$ will give the same result regardless of what you did in the middle, as long as you are reversible! Therefore do we have to be constant $T,V$ always? No, but we have to start and end at same $T,V$. If we do, you will see that the work (net for whatever you did) is the change in the Helmholtz potential or Helmholtz free energy.
Non-reversible equilibration Let us say you don't take the work out form this system. You don't move the piston, but break the barrier, allowing completely irreversible free expansion between both sides. In such a case if you go the equation above you will see,
\begin{align*}
&\delta W=0\\
& T\delta S_{gen} = -dA >0
\end{align*}
This means irreversibility during equilibration destroys Helmholtz free energy or potential to do work. So irreversible or not once the system has reached equilibrium it sits at minimum Helmholtz free energy. If you managed to take it out as work, good for you, otherwise you have generated a bunch of entropy and made nature happy :).
Ideal gas: Nowhere in my discussion I have assumed ideal gas/non-ideal gas so everything above applies to ideal gases.
One last thing In a similar manner you can show that for $T,P$ constant the free energy is Gibbs. In $S,V$ constrained systems the internal energy $U$ itself is the free energy (I mean differences of $U$, $G$). Try working it out, and let me know if you can't get it. These definitions are useful. You often see $G$ used in chemistry a lot since many chemical reactions were done in an open beaker/petri-dish where the atmosphere would keep $T, P$ constant.