Note: Emilio Pisanty wrote an answer that is probably a better fit for the question and site, but I'm leaving this answer around because I feel it can contribute to an understanding of how this works in practice.
For one thing, you'd need to be able to differentiate between the signals inside the frequency band.
As an example, I'm going to use a Morse code (CW) transmitter. The same principles, however, apply to pretty much all EM transmitters, of which radio forms a subclass; morse code just happens to be a simple example because it is designed as on/full-amplitude and off/zero-amplitude transmissions only.
Naiively, abruptly, turning a signal on or off, like for example in a crude Morse transmitter, creates sidebands to the transmitted frequency. These sidebands are going to interfere with anything else in the vicinity of the intended transmit frequency, and absent deliberate filtering can extend surprisingly far. The extreme case of this would probably be an unfiltered spark-gap transmitter, which was used before we figured out how to generate a continuously oscillating wave, and with which pretty much everyone was interfering with everyone else on radio. (In fact, that's where the term CW or Continuous Wave comes from in radio applications. Having figured out how to maintain a continous, stable oscillation, we were then able to implement more complex modulations like AM, followed by suppressed-carrier modes, each building on the previous either in terms of spectrum efficiency or signal fidelity, sometimes both.)
This "splatter" can be worked around or managed by various tricks, notably "shaping" of the transmitted or modulating signal. For example, PSK31 (a narrow-band digital transmission mode fairly commonly used in amateur radio) deliberately shapes the modulating signal such that phase reversals occur at near zero amplitude, which limits the splatter to what is considered acceptable bandwidths even with filter bandwidths much larger than the signal bandwidth. (PSK31 signals are just a shade over 31 Hz wide, but are often transmitted using plain SSB transmitters with a filter bandwidth of slightly less than 3 kHz; -6 dB at 2.7 kHz is a relatively common filter bandwidth for SSB.) Morse code transmitters may employ a capacitor across the transmitter keying apparatus to help lengthen the time to get to full amplitude; the less sharp signal causes less pronounced sidebands, leading to less interference than would otherwise be the case.
Obviously, the more time that the modulating signal needs to go from zero to full ampltiude (assuming a binary modulation), the less will be the maximum usable transmission speed of the system, because you spend more and more time in your "wait" state. On the other hand, you can group multiple such signals in a narrower range of transmitted frequencies because they are less likely to interfere with each other. The end result is the same because you are basically trading rise time per signal against number of signals within a given bandwidth.
To derive the maximum symbol rate given an acceptable modulating signal bandwidth and a given signal-to-noise ratio, you can turn to the Shannon-Hartley theorem, which establishes the theoretical upper bound. The modulating signal bandwidth is more important than the transmitted signal bandwidth because you clearly don't gain much in terms of symbol rate by moving from having an analog signal modulating a SSB transmitter, to having the same analog signal modulating a wideband FM transmitter, with the same signal-to-noise ratio after demodulation.