Interesting, I was just studying the Fourier decomposition of vowels for half an hour yesterday.
First, you must distinguish vowels and consonants. Words like "bee" (which is how we spell the letter "B") of course aren't uniform sounds. They start with a consonant, in this case one created by lips. Depending on the way how they're created, consonants are divided to many groups. See a table of consonants here.
In general, consonants are types of noise because they don't have a well-defined basic frequency. It means that the Fourier series for a consonant is composed pretty much of all sufficiently high frequencies. The color of the noise – whether higher frequencies tend to be more strongly represented than the lower one etc. – determines the type of the consonant.
There are consonants with a throat sound added, like L,M,N, which may be Fourier decomposed similarly to the vowels, and noise-based consonants such as B,D,G,V,Z which are the sound-equipped cousins of P,T,K,F,S, and so on, and so on.
The most monochromatic sounds are the vowels. They can be sung so they have a well-defined base frequency. Whether one gets U,O,A,E,I – or, in English, OO, AW, AH, EH, EE etc. – depends on how the mouth is opened. This modifies the shape of the resonance cavity and therefore the preferred additional frequencies that are excited by the action of the base frequency coming from the throat.
The presence of higher harmonics is essential for the difference between vowels. I recommend you to look at a page about it, for example this one:
http://hyperphysics.phy-astr.gsu.edu/hbase/sound/vowel.html
The vowel U (OO in English) has the lowest representation of the higher harmonics. It is the closest one to the harmonic sound and it's achieved by changing one's mouth into a passive tube through which the sound penetrates. On the other hand, A (AH) and I (EE) have a huge, important contribution of higher harmonics and when I say higher, I don't mean the 2nd or 3rd. The 20th harmonic etc. etc. are very important.
In fact, it's more accurate to talk about the absolute frequencies. The vowel A (AH) has lots of those higher harmonics that are close to 1,000 Hz (cycles per second) which are already suppressed in U (OO) and partly suppressed in O (AW). As you continue to go towards E (EH) and I (EE), the contribution from frequencies close to 3,000 Hz starts to increase.
These frequencies are calculable from the size of the mouth opened in the right way, from the length of the resonant cavity that becomes comparable to the wavelength of the sound waves that become important.
In principle, all vowels may be emulated by the Fourier expansion using just the base frequency (pressure goes like $\sin \omega t$) plus higher harmonics but the very harmonics including $\sin 20\omega t$ are still very important for the character of the vowel.
Phonetics is the portion of linguistics that studies how language sounds; the experts partially learn some physics although Fourier series aren't their primary tool. Still, to understand phonetics, one has to accept various basic things. I found out that native English speakers misunderstand phonetics because they don't really decompose the language into "pure sounds". Your representation of "B" as a sound analogous to "A" may be an extreme example because you may have meant "B" in the sense of "bee" which is clearly composed of two sounds, the consonant "B" and the vowel "EE".
But even when it comes to vowels only, English (and French and some other languages) is deliberately obscuring the reality as it pronounces many vowels in a variable way. For example, "my" is pronounced as "mai" where the vowel gradually changes from "AH" at the beginning to "EE" at the end. Some native English speakers don't even realize that this "Y" in "MY" isn't a single uniform vowel. There are many other examples, of course.