A statistical test would require a large number of rolls of the dice. For a simpler statistical example, to test whether a coin is fair by tossing a coin $N$ times would result in approximately $\frac{N}{2}$ heads but the standard deviation of that count would be about $sqrt(N)$ so to get to a level of significance where the standard deviation is only 0.01 on the 0.50 value that you are trying to measure would require $N = (0.01)^{-2} = 10000$ tosses of a coin.
Another test you could try would be to try to accurately measure where the center of mass of the die is. You could do this by seeing if the die can approximately balanced on a knife edge which is bisecting one of the die's faces. Do this for 3 of the orthogonal faces of the die and if they all are balanced, then the center of mass is in the geometric center of the die. However, I don't know how to determine what error in this measurement would be achievable and how much an off-center center of mass would affect the fairness of the die.