Does the color of an object have anything to do with its refractive index, and if so, what is the relation? I tried googling this and searching through the questions on stack exchange, but I haven't been able to find anything addressing this exact question.
The color of an object is completely determined by the refractive index and the geometry of the object. The reason why it is not obvious is that one has to takes the imaginary part of the refractive index into account. The real part determines the wavelength of light in matter and the imaginary part describes absorption of light in matter.
When you ask what color has an object the physics question you ask is what light is reflected or transmitted by the object to your eye. Transmission and reflection are well described by the fresnel equations. They only depend on the the geometry of the problem, the refractive index of the object and the wavelength of the light. For the object to have color the refractive index has to change with wavelength, which we call dispersion.
There are now certain possibilities for the refractive index:
Refractive index is purely real in the wavelength region of interest => no absorption (e.g. diamond, glass, air): Light is mainly transmitted. Few percent are typically reflected. Dispersion is only visible in certain geometries for example prism or rainbow.
Refractive index has a low imaginary part (e.g. Rust, coal...): As in 1. few percent are reflected but in contrast to 1. the light transmitted into the substance is absorbed. Dispersion of the imaginary part determines the color. Rust has an absorption peak around 400 nm (blue) and therefore looks reddish.
Refractive index has a high imaginary part (e.g. metals): Most light is reflected. Higher absorption of a color now means that it looks more like that color because higher imaginary part means higher reflectivity. For example copper has a high absorption in the red and low one in blue resulting in a reddish color.