In the Casimir effect, after performing the regularization, it is found that the zero point energy between two conducting plates in a distance $L$ from eachother is (in the 1D case), $$E=-\frac{\hbar c \pi}{24 L}.$$ From this result, it is argued that the force felt between the two plates containig the quantized modes is $$F=-\frac{\partial E}{\partial L}\propto-\frac{1}{L^2},$$ which means that this force is attractive, and thus, the Casimir effect emerges. However, it is not clear to me, as from my understanding, the force of a conservative system is calculated with respect to the gradient of the potential energy, and not the total energy (also, in the situation of the Casimir effect, the system is modeled as an infinite well potential, so that there is zero potential energy between these plates).
Then, how can this derivative with respect to the total energy be interpreted as an atractive force between the plates?