This is potentially a very stupid question but I'm going to ask it anyway. With all these huge buildings such as the Abu Dhabi Mosque, where an unbelievable amount of materials such as marble was moved from one side of the earth to the other, is it possible that if we 'shift enough stuff' that we could change the earths centre of gravity, and potentially alter its orbit?
No, these building are still tiny compared to earth's crust mass distribution. One would need to build whole mountain ranges to detect changes in earth gravity field with high precision instruments. And even those wouldn't changed earth orbit measurably because even a mountain range is tiny compared to the mass of the whole earth.
However mountain ranges (and their rock density) do have a measurable influence on earth's gravity and this measured by satellite missions like grace.
PS: And no, it's not a stupid question. But you could submit it to xkcd's "what if?" section ;) .