What is the speed of the fastest moving body in our solar system? On Wikipedia I saw that the average orbital speed of planet Earth around the Sun is a whopping $29 783\text{ m/s}$, and it made me wonder are there bodies (planets, meteorites, asteroids) that move faster? 
My question is not about small photons or other (small-ish) particles and their speed (speed of light), or even about solar winds ($750\text{ km/s}$) but about meteorites, planets or other materials and their speed around the sun or other fixed point. 
 A: A comet doesn't need to impact the sun in order to come very close to solar escape velocity at perihelion. There is a class of comets known as sungrazers that pass very close to the sun. Although small ones evaporate on their first pass near the sun, larger ones can survive several orbits, and be considered periodic comets.
There is a class of sungrazing comets called the Kreutz family that has a very low perihelion and a reasonably high aphelion (150-200 AU) making them the best candidates that I know of for "fastest object in the solar system" when they pass near the sun. The comet Lovejoy (C/2011 W3) has an aphelion around 157 AU and a perihelion of 0.00555 AU (within the solar corona, note that the sun itself has a photosphere radius of 0.00465 AU!). As such it passed by the sun in December 2011 at a speed of 536 km/s, within a couple percent of the escape velocity at that height, which is 565 km/s. The Great Comet of 1843, another Kreutz-family comet, reportedly passed even lower without disintegrating, 0.00546 AU, giving it a speed of 570 km/s.
Pulsar did a fine job of working out the math, so I won't duplicate it here, except to emphasize the point that once your aphelion is tens of thousands of times higher than your perihelion, aphelion stops making much of a difference. If you're 100km above the surface of the sun and travelling at hundreds of km/s, the difference between the speed you need to go to get 100 AU out, and the speed you need to go to get 1000 AU out, is miniscule, and both are very close to escape velocity.
A: The asteroid "1566 Icarus" has a perihelion distance of 0.187 au and a semi-major axis of $a=1.078$ au, an orbital period of 1.119 years and eccentricity $e=0.827$.
Using
$$v_{\rm peri} = \sqrt{\frac{GM}{a}\frac{(1+e)}{(1-e)}},$$
where $M$ is a solar mass, then its fastest speed is 93.5 km/s.
So this does not come close to Comet Lovejoy (mentioned in other comments), but beats Mercury, and is perhaps the fastest object we can continue to study on a regular basis, since Comet Lovejoy disintegrated. Doubtless there will be other small chunks of rock that might beat this.
A: The maximum speed of an object that orbits the Sun at a certain distance $r$ is known as the escape velocity:
$$
v_\text{esc} = \sqrt{\frac{2GM_\odot}{r}},
$$
where $M_\odot$ is the mass of the Sun. If the object would have a greater speed, it would eventually leave the solar system. So I'd say that the absolute maximum possible speed of any object in the solar system would be the escape velocity at the radius of the Sun $R_\odot$:
$$
v_\max = \sqrt{\frac{2GM_\odot}{R_\odot}},
$$
which, as you can find in the wiki article, is $617.5\;\text{km/s}$. A comet that slams into the Sun, which occasionally happens, would have a speed close to this maximum. Alas, it's also the last speed it'll have before it meets its doom :-)

Update
If you want to know the fastest object in the solar system that didn't crash into the Sun, then the best candidates are sungrazing comets, i.e. comets with very eccentric orbits that pass very close to the Sun. One particular group are the Kreutz Sungrazers. The comet C/2011 W3 (Lovejoy) mentioned by hobbs in the comments belongs to this group, but there was another of these comets that passed the Sun even closer: the Great Comet of 1843. 
This comet has a perihelion of only 0.005460 AU (where 1 Astronomical Unit is 149 597 871 km). This means it came to within less than 121 000 km of the surface of the Sun, and amazingly it survived (most comets break up when they come this close). So what is its velocity at perihelion?
The general formula is (see this link)
$$
v_p = \sqrt{\frac{\mu}{a}\frac{1+e}{1-e}},
$$
with
$$
a = \frac{r_p + r_a}{2}
$$
the semi-major axis, $r_p$ and $r_a$ the peri- and aphelion, 
$$
e = \frac{r_a-r_p}{r_a+r_p}
$$
the eccentricity, and $\mu = GM_\odot$ the standard gravitational parameter of the Sun. So we can rewrite this as
$$
v_p = \sqrt{\frac{2GM_\odot}{r_p}\left(\frac{r_a}{r_a+r_p}\right)}.
$$
As you can see, this reduces indeed to the formula for the escape velocity if $r_a$ goes to infinity. For our comet, $r_p = 0.005460$ AU and $r_a = 156$ AU, and we find
$$
v_p = 570\;\text{km/s}.
$$
A: Kepler's Three Laws of Planetary Motion are particularly helpful when addressing this question. They state that (in informal language)


*

*The shape of a planet's orbit in an ellipse, with the Sun at one focus of the ellipse.

*As planets move around their elliptical orbits, the imaginary line drawn from the planet to the Sun sweeps out equal regions of equal area in equal amounts of time.

*The square of the period of a planets orbit, $T^2$ is equal to the cube of planet's orbit's semi-major axis (a^3)


Although not immediately obvious, Laws 2 and 3 combined both imply that as a satellite (planet, asteroid, comet, or otherwise) approaches closer to the sun, it can be expected to have a faster velocity.
Specifically, if we look just at the eight planets, and Law 3, $$T^2\propto a^3$$ which, when solved for period states that $$T\propto \sqrt{a^3}$$   So using the equation above, let's say planet $A$ travels in some orbit around the sun, and the semi-major axis has a length of $a$. If planet $B$ travels in and orbit, with a semi-major axis of $4a$ then the period has now increased by a factor of 8, even though the semimajor axis (and approximately the circumference, if the orbit has an eccentricity close to 0) only increased by a factor of 4. So as you move away from the sun, your period increases more than your distance, which means your orbital velocity is decreasing.  Just look at this graph below, taken from enchantedlearning.com.

You can see a clear relationship between velocity, and distance away from the sun.
Now let's look at interlopers to our solar system, like comets.  Compared to planets, most comets tend to have eccentricities very close to 1 (which means their orbits are very elliptical).  Some comets even have eccentricities greater than one, which means they're on one-time hyperbolic orbits around the sun.  As these comets approach perihelion (the close approach to the Sun) Kepler's Second Law tells us that thevelocity of the satellite increases.  The most extreme examples are sun-grazing comets, which have very close approaches to the Sun.  In fact, comet ISON was moving so quickly last November when it approached perihelion that had a) you been able to see the comet in daylight and b) Coment ISON not met an untimely demise you would have actually seen it change position in the sky (relative to background starts) by the hour.
A: When there aren't comets falling into the sun, Mercury is hard to beat.  This NASA fact sheet lists Mercury's orbital velocity around the sun as varying from $38.86$ to $58.98$ km/sec, not so much greater than Earth (less than a factor $2$, even at maximum).
A: The fastest moving object that does not get destroyed by crashing into the sun would be the apollo asteroids that get very close to the sun.  For example Icarus gets going pretty fast at perihelion, (0.18665203 AU from the sun) at just under 100 km/sec.
A: This question has received some excellent responses. As the person asking seems keen to get a larger variety of responses, I am going to give this question another twist by enquiring about the maximum speed relative to Earth:

Earth is a planet, which means it cleans its orbit around Sun 
  from material objects. What is the maximum speed at which such 
  object can hit earth's atmosphere?

Earth's orbit around Sun is very close to circular. Equating the centripetal force required to keep Earth in this orbit to the gravitational force exerted by Sun, it follows that Earth orbits Sun with a kinetic energy equal to half the energy needed to escape Sun. 
An object that orbits Sun along an extremely elongated elliptical path and reaches closest approach to Sun somewhere along Earth's path, has at that point (perihelion) a kinetic energy equal to the energy needed to escape from Sun. 
As kinetic energy scales quadratically with speed, it follows that Earth's speed along it's orbit around Sun equals $\frac12\sqrt2$ times the local escape velocity. This escape velocity, the  velocity required to escape from a location along Earth's orbit around Sun, equates to a marathon (a wee bit more than 42 km) per second. It follows that Earth orbits Sun at a speed of 29.8 km/s. 
If at closest approach the object moves in opposite direction to Earth, collision will be head-on and one has to add both speeds to get the total speed. This total speed equals 71.9 km/s.
This, however, does not equate to the speed at impact, as the gravitational attraction to Earth accelerates the object towards impact. So, to arrive at the speed at impact we have to add Earth's escape velocity (11.2 km/s) to the above derived velocity. 
The resulting maximum velocity at impact is 83.1 km/s. Solar system objects can not hit us at larger speed.
A: Depending on what you are looking for, here are some possible candidates for the fastest bodies in the solar system:


*

*Comets from outside the solar system that fall into the Sun, just
before impact

*Comets with a periodic elliptical orbit around the sun, at the
moment of their closest approach to the sun

*Mercury, with a mean orbital velocity of 47.9 km/sec

*Metis (the inner-most moon of Jupiter), with a mean orbital velocity
of 31.6 km/sec

*Since Metis orbits within Jupiter's main ring, one can assume some
of the particles of the ring that are closer to Jupiter have a
higher orbital velocity than Metis


If you want anything faster, you have to get into cosmic rays and such, which you said you weren't interested in.
A: As I understand the question, comets (or any thing coming from outside the solar system) can not be considered.  This leaves only the asteroids and other debris that still circle the sun at distance r.  If this mass starts "falling" towards the sun, it will attain a velocity given by Pulsar's equation, if it is corrected by replacing the term (1/Rsun) with (1/Rsun - 1/r)  
