You can solve this by picking a simple, specific example and then generalizing it as necessary. Let’s start with a simple case, where the tempo is such that there are 120 quarter notes per minute. The frequency of quarter notes is then
$$ \frac{\text{120 quarter notes}}{\text{1 minute}} \cdot \frac{\text{1 minute}}{\text{60 seconds}} \cdot \frac{\text{1 second}}{\text{1000 milliseconds}} = \frac{\text{0.002 quarter notes}}{\text{millisecond}} . $$
This is the number of quarter notes that occur each millisecond. For a more useful number, we can invert this fraction to see that the number of milliseconds taken by one quarter note is
$$ \frac{1}{\frac{\text{0.002 quarter notes}}{\text{millisecond}}} = \frac{\text{1 millisecond}}{\text{0.002 quarter notes}} = \frac{\text{500 milliseconds}}{\text{1 quarter note}} . $$
You can do fancier things involving other notes, tempos, key signatures, etc. by first finding out how many of your note occur in one minute (120 quarter notes in this example), and then following this same process.