How do I know the proton isn't made of 3 anti-down quarks? I have a proton, how do I know that it is made  of 2 up quarks and 1 down quark or if it is made of 3 anti-down quarks, each with different color charges?
This question is also applicable to the antiproton, is it made up of 2 antiup and 1 antidown or of 3 down quakrs? 
 A: The Standard Model which has been decided upon after a thorough experimental observation of the interactions of particles at the micro level, i.e. the space and energy dimensions where quantum mechanics reigns, has as a main pillar the quark model.

The quark model started with the above observation: that if the particles were plotted in two dimensions using their  spins and their quantum numbers ( isotopic spin, strangeness, etc) a beautiful symmetry emerged, which also ordered the particles according to their masses.This symmetry was  found to be mathematically represented by the representations of the special unitary group , SU(3). The 3 means that there are 3 basic units which can be permuted to fill up the points of the representations. These were whimsically named "quarks".
This plot is the baryon spin 1/2 plot of which the proton is a member. There are a number of other representations where the data from hadronic resonances fit well, and even a prediction was made that the omega minus should exist  because all the other members of the decuplet were experimentally seen already.

It was found, and it established the quark model, and nobel prizes were awarded to the researchers at the frontier of this work.

All quarks are assigned a baryon number of 1⁄3. Up, charm and top quarks have an electric charge of +2⁄3, while the down, strange, and bottom quarks have an electric charge of −1⁄3. Antiquarks have the opposite quantum numbers. Quarks are also spin-1⁄2 particles, meaning they are fermions.
Mesons are made of a valence quark−antiquark pair (thus have a baryon number of 0), while baryons are made of three quarks (thus have a baryon number of 1). This article discusses the quark model for the up, down, and strange flavours of quark (which form an approximate SU(3) symmetry). There are generalizations to larger number of flavours.

Now if you look at the plot of the ground state baryon octet, you can see that it does not matter what we call them, it is the symmetry that establishes their quark content. Thus the proton, which is the name of the particle  with the lowers mass in this representation, can be nothing but the building block of the solid matter we exist on. We call it a proton, its constituents quarks with given names. If one changed the quark content, one would have a different name, it would be a different particle and not the lowest energy state in the stable representation.
The proton has to add up to baryon charge of 1, your speculative quantum numbers cannot do that by the definition of quarks and their position in the SU(3) representations. Anti quarks add up to anti baryons, and the proton is not an antibaryon by definition.
The representations are restrictive by experimental measurements, and one cannot throw quarks like dice, because the whole quark representation is constrained , and the names of the particles are convenient, but are really a one to one correspondence with the quark content by construction of the mathematics that nature is using.
A: Basically, it's because the proton has spin $\frac{1}{2}$, which means the quarks' spins need to be split: two in one direction (let's say up) and one in the other direction (down). But having two quarks with opposite spins and different colors but the same flavor (antidown) violates the Pauli exclusion principle. So there can't be a spin-$\frac{1}{2}$ baryon made of three of the same flavor quark.
A: It is because the Baryon number would otherwise not be conserved!
The baryon number of a proton is 1, therefore using three antidown quarks would mean the proton has a Baryon number of -1 (-1/3+-1/3+-1/3), which is incorrect!
The same with an antiproton which has a Baryon number of -1. If you were to use three down quarks, the baryon number would equal +1 (1/3+1/3+1/3) which would be incorrect!
