The energy of light wave is not simply it's frequency component - it is both its frequency and amplitude.
But as with any wave, amplitude is a different quality than frequency, and the two are not completely interchangeable in their effects (even though a wave with high-amplitude and low-frequency may carry the same or more energy as a wave with low-amplitude and high-frequency) because of the effects of resonance (which is somewhat related to inertia).
Consider when your car bumps over a pothole - even a relatively shallow pothole might break wheels and almost knock your fillings out. That's a high-frequency, low-amplitude shock.
And yet you may drive up and down a mountainside comfortably (even though the amplitude of that movement involves orders of magnitude more energy being borne by the car through its wheels and suspension, it is so diffuse over time that it is insufficient to disrupt the physical integrity of the car or your body, which simply rides the wave rather than being shattered by it).
There is no way to substitute amplitude to achieve the same effect that frequency has - excessive amplitude would compromise the superstructure of the car rather than shocking it's components or passengers.
The same is true of atoms, that at too low a frequency of light, they may be temporarily perturbed by such light but it is insufficient to affect their integrity and cause (for example) the photoelectric effect, and at too high an amplitude, the solidity of the superstructure which is composed of the atoms would be compromised before the individual atoms were.