infinite power from constant force? P = W/t = Fd/t = Fv
So, if we keep exerting a small constant force and the velocity keeps increasing to near infinite, we should be able to get near-infinite power from the small constant force. 
We are working within the limits of special relativity, ie not approaching  the speed of light, so I am not seeking a special relativity answer. 
It is possible to achieve this by using a small rocket motor burning a small amount of fuel for a long time. 
The thrust (F) produced is independent of the velocity of the rocket and is constant, and depending only on the velocity of the exhaust gas relative to the rocket, which will remain the same as the rocket accelerates. 
I am ignoring the fact that the mass of the rocket decreases as the fuel burns, and assuming that the fuel does not run out while there is a large increase in velocity.
So, basically the small thrust of the rocket motor (burning a small amount of fuel per unit time) will generate an ever-increasing power as the velocity of the rocket increases.
Where did I go wrong? How is it possible to get an increasing power from burning a small constant amount of fuel per unit time? Where does the increasing power come from?
 A: First of all, if you're not seeking a special-relativistic answer, then you're not allowed to consider what happens at higher and higher speeds. So, then, my initial answer to you is, "Infinite power doesn't happen when Newtonian mechanics is valid, because Newtonian mechanics is only valid at low speeds." Infinities in physics generally mean that your theory isn't valid in the region where you get the infinity.
But let's assume for a moment that we live in a completely Newtonian universe. Then your intuition is correct, in that a constant force exerts a greater and greater power with higher velocity. This can be intuitively explained by the fact that kinetic energy ($\frac{1}{2}mv^2$) grows faster than $v$ under constant acceleration (which you would get under a constant force and a constant mass). In order to keep the definitions of kinetic energy and velocity consistent with each other, the motor must increase the kinetic energy faster and faster as the object accelerates.
A: Well, the idea behind a rocket is that it expels some of its own mass to produce the force. So you are very limited by the initial mass and the more it accelerates, the lighter it gets. Also, there is such a thing as fiction which limits the gains in acceleration.
You don't seem to have an actual question, what are you trying to say? Of course if you accelerate something for infinity, it goes very fast. This reminds me of the Starshot project, which aims to propulse a space capsule by sending a beam on its solar sail. It's a bit of subject but I suggest you give it a look.
A: 
It is possible to achieve this by using a small rocket motor burning a small amount of fuel for a long time.

Unfortunately a "long time" is quite finite and it keeps your power from becoming infinite either.  A small amount of fuel must either provide a small amount of thrust or not burn for very long.  That's why probes take so long to reach even the outer solar system. 

I am ignoring the fact that the mass of the rocket decreases as the fuel burns, and assuming that the fuel does not run out while there is a large increase in velocity.

These are big things to ignore.  Yes, if we had a magic engine that didn't ever run out of fuel, we could get infinite power from it (in some frame of reference).  But such engines do not exist.  
probably_someone raised a point you may be wondering about: 

why does the power exerted by the rocket continually increase as the velocity increases, even though it seems like the engine doesn't do more work per unit time?

Because most of the time we look at a rocket and we think of the burning fuel and how that affects the rocket's speed.  But you also have the unburned fuel and the exhaust.  
If the rocket is on the pad, then at launch the energy is going into accelerating everything (the rocket and it's fuel forward, and the exhaust backward).  All the KE of the exhaust is basically thrown away and never collected.  It has to be done for the momentum exchange, but the energy is lost.  So at launch, the rocket is very inefficient.  
When the rocket is moving forward at high speed in our observation frame, then we now see the exhaust emerges more slowly.  That means the proportion of the energy of the combustion that is going into rocket KE and exhaust KE is changing.   All the energy is still being used, but the proportion that goes into the rocket (which we care about) and the proportion going into the exhaust (which we don't care about) is changing.  
EXAMPLE:
Assume your rocket has a mass of $1\text{kg}$. In an atomic fashion, it exhausts $1\text{g}$ of mass at a speed of $1000\text{m/s}$. Conservation of momentum means the rocket accelerates by $1.001\text{m/s}$.
Assuming it was at rest initially, we can see the total change in KE is:
$$KE_{\text{fuel}} = 0.5 (1\text{g}) (1000\text{m/s})^2 = 500\text{J}$$
$$KE_{\text{rocket}} = 0.5 (999\text{g}) (1.001\text{m/s})^2 = 0.5005\text{J}$$
So the energy of combustion is giving us almost $501\text{J}$ of energy, but most of it is going into the exhaust.  Less than one part in a thousand is going into increasing the rocket's KE.
Now we imagine the rocket is already moving at $10\text{km/s}$ (much faster than the exhaust speed) and it has the same thrust. What's the change in KE of the rocket and the system?
$$\Delta KE_\text{rocket} = KE_{\text{10001m/s}} - KE_{\text{10000m/s}}$$
$$\Delta KE_\text{rocket} = 49,960,001\text{J} - 49,950,000\text{J} = 10001\text{J}$$
$$\Delta KE_\text{fuel} = KE_\text{9000m/s} - KE_\text{10000m/s}$$
$$\Delta KE_\text{fuel} = 40,500\text{J} - 50,000\text{J} = -9500\text{J}$$
$$\Delta KE_\text{total} = 10001\text{J} - 9500\text{J} = 501\text{J}$$
In this frame, the rocket gets a huge increase in KE from the same burn (more than the total energy available from combustion), but only with a corresponding decrease in KE of the exhaust mass. The total change in KE of the system is constant.
A: The source of the misconception is not noticing that distance per unit time is increasing. This means work per unit time, or power consumed per unit time, must also increase. A rocket is a bad example because mass is also being accelerated in the opposite direction.
A better example to analyze is bicycling in a no friction environment. Say, on the Moon with a frictionless bicycle. Can you achieve orbital velocity by pedaling? How long will it take for a good cyclist? Can they put out enough power to keep accelerating at the same rate?
