For a rolling tyre the resistance (it's not really friction) is due to viscous losses in the rubber as the tyre is deformed.
If you take a piece of rubber and put work in to stretch it then get work out as it relaxes, the work you get back is less than the work you put in. The balance goes into heating the rubber. If the piece of rubber is part of a tyre, this loss of energy means you have to put energy in to keep your vehicle moving at a constant speed i.e. you have to apply a force. That force is what you feel as rolling resistance. If you touch your car tyres after you've driven the car you'll find they are warm because they've been heated by the losses within them.
The more the tyre deforms, the more energy is lost, and therefore the greater the resistance. So if you use a smooth tyre and pump it to a very hard pressure the rolling resistance will be low. If you use a knobbly tyre and run it at a low pressure the resistance will be high.
You ask if there's a way to describe this mathematically, and yes there is assuming you know the material properties of the rubber you're using. However you'd need to use a finite element analysis.