In addition to the greater variance of the human ear compared to the human eye, I think there is at least one other reason sones (or whatever) aren't SI units and it boils down to this: while all candles of the same color and brightness look the same, not all strings of the same pitch and loudness sound the same.
While loudness corresponds to brightness and pitch corresponds to color, sound has a third quality, let's call it "timbre", that doesn't really have an analogue in light. The closest, for our purposes, is color mixing but here the differences between optic perception and acoustic perception cause a serious divergence.
For optic perception, only three colors of light get their own receptor, and even these have significant overlap (so much so that calling them "red", "green" and "blue" is more than a little disingenuous). As a result, color mixing is simply how most colors are perceived. For acoustic perception, every (or near enough) pitch of sound gets its own receptor, none of this "oh, look, the green and red cones were both set off, but there's more red than green, must be orange" nonsense. No; each pitch gets its own hair cell.
So what happens if more than one hair cell is set off by one note? This is (our definition of) timbre. The lowest-perceived or "fundamental" pitch is the nominal pitch, but the higher "harmonic" pitches tell you what kind of sound it is. This is what distinguishes pianos from harps, flutes from clarinets, plain vowels from nasal vowels from rhotic vowels, and content purrs from disgruntled growls.
Pitch and loudness interact analogously to the way color and brightness interact (the threshold of hearing, ie, the quietest sound a human can perceive, depends on pitch), but timbre also interacts with both. A sound that carries multiple distinct pitches will sound louder than a sound with only one pitch at the same amplitude; the "fuzzier" and "noisier" the note, the louder it sounds, up to a point, although "pure" tones can also be quite piercing. And of course "noisier" sounds can't really be said to even have a pitch.
This is where the difference between color-mixing and timbre really shines (pun very much intended). Because all colors except the reddest of reds and the bluest of blues are perceived as mixed colors, even if they were originally pure, color mixing can be smoothed away to give a fairly consistent luminosity function, even for white light, the "noisiest" of light. (Heck, the candela used to be defined by the brightness of a certain black body, which by definition emits light that is as mixed as possible.)
But pure tones are perceptually distinct from mixed sounds, and timbre is, if anything, more exaggerated than frequency-mixing. A proper "sonorosity function" would have to take into account pitch, loudness and timbre and all the messiness that comes with it; the fact that "noises" don't actually have a pitch, that many pitches played at once tend to be louder than if each were played separately, that timbre can't really be properly ordered, even in the way color can in two or three (or more) dimensions.