An object is white when it reflects more or less any color – any frequency – and it reflects it in a random direction: the light gets scattered.
For this reason, the photons arriving from a specific point of the white object (e.g. a point on the paper) are photons that arrived there from random directions in the space and they have random colors. Because the white object mixes the directions, it mixes the colors, too, and the mixture of all colors (including red, green, blue) looks like white light.
A mirror is an object that reflects the photons in a specific direction – the direction that only changes the sign of the angle between the photon and the (tangent to the) surface of the object.
An ideal mirror would reflect all colors perfectly so it would appear colorless. However, there's some scattering from metals such as silver, platinum, and iron. This is mostly independent of the frequency so the metals appear color-neutral i.e. grey.
Copper, gold, and others tend to scatter especially longer [thanks for the fix] (redder) wavelengths, so they appear redder – orange or yellow (shiny yellow i.e. golden) etc. Incidentally, individual gold atoms not bound in the macroscopic body would appear greenish, not yellow.