How does the piston inside the cylinder exactly behave during an isothermal expansion? Let us say we have a perfect gas in a cylinder with a piston, with section $A$. We place the cylinder over a thermal reservoir at a certain temperature. Gradually, assuming an isothermal process, small amounts of heat $\delta Q$ will flow from the thermal reservoir into the cylinder, and the piston, being acted upon by a force $f=pA$, will get pushed forward a distance $dx$.
The initial work, $fdx=\delta Q$, at the very beginning will give the piston momentum, after initially being at rest. Thus, no further heat should be required to expand the gas after an initial infinitesimal flow of heat unless there are friction forces that will bring the piston to rest.
So what does it mean when one says a cylinder+piston over a thermal reservoir takes in an amount $Q_{1}$ of heat and does work on the piston so that the gas gets expanded isothermally. Is $Q_{1}$ here an infinitesimal amount of heat (as $\delta Q$ above)? Do we assume there are friction forces between the piston and the cylinder? If there is no friction, doesn't that mean the piston gets accelerated during the expansion, and that the additional heat that flows afterward, $Q_{1}-\delta Q$,  only adds momentum to the piston?
 A: The piston which is referenced in basic thermodynamics is not a real, dynamical object with mass or inertia.  It's an idealization that serves only as a "handle" by which the volume of the system can be changed.
To understand the model, consider an ideal gas with a fixed $N$.  The state of the system is a point in $(P,V,T)$ space which lies on a surface
$$\frac{PV}{T} = const$$

A quasi-static thermodynamic process is a curve embedded in this surface.  In particular, an isothermal process is a curve restricted to constant $T$, shown here:

From a mathematical perspective, that's all there is to it - we fix $T$ to be constant and observe that $P$ and $V$ must evolve such that $PV=const$. However, to make the connection with a physical system (e.g. a gas), we talk about how we would manipulate the system into behaving this way.
When we restrict the system to a constant $T$, we say we place it in contact with a thermal reservoir.  When we restrict it to constant $V$, we say we lock the piston.  When we restrict it to constant $P$, we say that we expose the piston to a fixed external pressure.  However, when we do so we are not modeling the physical mechanisms by which these constraints are being imposed.  "Heat bath" and "piston" and "external pressure" are just the words that we drape over the idea that the temperature, volume, and pressure of our gas can be constrained or manipulated at will.
That's not to say that we can't model these things.  You can give the piston some real mass $m$ and introduce frictional forces on it.  However, if you do this then you're going to have to couple Newton's 2nd law (applied to the piston) to the laws of thermodynamics (applied to the gas).  You would also have to assume that any dynamics occur on a time scale sufficiently slow to make thermodynamics applicable in the first place.

The initial work, $f\ dx = \delta Q$, at the very beginning will give the piston momentum [...]

Unless you are genuinely modeling the piston as a real physical object, you should not go down this road.  In real life, the work done by the gas is done on the external environment as well as on the piston.  In idealized thermodynamics, we neglect the latter because the piston is not to be thought of as a real object.

In the spirit of demonstrating my point, consider a simple case of isobaric expansion of an ideal gas. We will consider the process to be a sequence of adiabatic steps, during which the pressure increases and then relaxes to the ambient pressure.
Let the external pressure on the system be $P_{ext}=P_0-mg/A$ (the gas must be at a slightly higher pressure than the environment to support the weight of the piston), and the initial pressure, temperature, and volume of the gas be $P_0,T_0,$ and $V_0$.  The system begins at equilibrium, so $T_0 = P_0 V_0/N$ (I choose units in which $k_B=1$ for convenience).
The piston has mass $m$, cross-sectional area $A$, and, when moving, experiences a frictional force $\vec F=-2m\Gamma \vec v$ due to its contact with the walls.  Its initial position is $y=y_0 = V_0/A$.
At the time $t=0$, a small amount of heat $\delta Q$ is delivered to the gas.  We assume that this energy is distributed instantly and uniformly throughout the system (at least, before the volume can change), thereby raising its temperature by an amount $\delta T = \frac{2\delta Q}{3N}$.  Since the volume does not change, this raises the pressure by an amount $\delta P = \frac{N}{V_0} \delta T = \frac{2\delta Q}{3 V_0}$.  No more heat is added to the system.
We now want to know the details of the systems "relaxation" (or equilibriation). The equations which govern the evolution of the system are Newton's 2nd law and the ideal gas law.  They are:
$$m \ddot y(t) = A(P-P_{ext}) - mg - m\Gamma \dot y(t) $$
$$PV = NT$$
with the latter constrained by the adiabatic condition
$$PV^\gamma = const$$
with $\gamma = 5/3$.
Newton's law can be rearranged to yield
$$\ddot y(t) + 2\Gamma \dot y(t) = \frac{A}{m}(P-P_0)$$
while the adiabatic condition yields
$$PA^\gamma y^\gamma = (P_0+\delta P)A^\gamma y_0 \implies P = (P_0+\delta P) \left(\frac{y_0}{y}\right)^\gamma$$
Therefore, we have
$$\ddot y(t) + 2\Gamma \dot y(t) - \frac{P_0 A}{m}\left(\left(1+\frac{\delta P}{P_0}\right)\left[\frac{y_0}{y}\right]^\gamma-1\right) = 0 $$
In general this is not an easy problem to solve.  However, recall that the amount of heat added (and therefore the change in volume) is very small.  It follows that we would expect $y=y_0 + z(t)$ where $z(t)\ll y_0$.  In that approximation,
$$\left(\frac{y_0}{y_0 + z}\right)^\gamma \approx \left(1-\frac{z}{y_0}\right)^\gamma \approx 1 - \gamma\frac{z}{y_0}$$
$$\left(1+\frac{\delta P}{P_0}\right)\left(1-\gamma \frac{z}{y_0}\right) \approx 1 - \gamma \frac{z}{y_0} + \frac{\delta P}{P_0}$$
and so our equation becomes
$$\ddot z(t) + 2\Gamma \dot z(t) + \omega^2 z = \frac{A\delta P}{m}$$
This is the equation of the damped harmonic oscillator under a constant external force, where we've defined $\omega^2 \equiv \frac{P_0 A \gamma}{m y_0}$$ for convenience.
Our initial condition corresponds to $z(0)=0$ and $\dot z(0) = 0$.  The homogeneous solution to the equation is
$$z_H(t) = e^{-\Gamma t}\left(c_1 e^{\alpha t} + c_2 e^{-\alpha t}\right)$$
where $\alpha \equiv \sqrt{\Gamma^2-\omega^2}$. The inhomogeneous solution is simply a constant $z_\infty$ - plugging it into the equation yields $\omega^2 z_\infty = \frac{\delta P}{P_0} \implies z_\infty = \frac{A\delta P}{\omega^2 m}$.
Imposing the initial conditions $z(0) = \dot z(0) = 0$, we find
$$z(t) = z_\infty \left( 1 - e^{-\Gamma t}\left[\frac{\Gamma}{\alpha}\sinh(\alpha t) + \cosh(\alpha t)\right]\right)$$
where $z_\infty \equiv \frac{A\delta P}{\omega^2 m} = \frac{2 \delta Q}{3\gamma P_0 A}$.

Now let's examine this result.  The change in volume is $\delta V = z_\infty A = \frac{2\delta Q}{3 \gamma P_0}$. The final pressure is simply $P_0$, and the final temperature is $T = T_0 + \delta T = T_0 + \frac{2 \delta Q}{3 \gamma N}$.  Assuming that the damping is sufficient to prevent oscillatory behavior (i.e. $\Gamma^2 \gg \omega^2$), the time scale for this expansion is
$$\tau^{-1} \approx \Gamma - \sqrt{\Gamma^2-\omega^2} \approx \frac{\omega^2}{2\Gamma}$$
$$\implies \tau = \frac{2\Gamma}{\omega^2} = \frac{2m \Gamma y_0}{P_0 A \gamma}$$
If we take $M$ such steps, then we would find $M$ times the total change in temperature and volume (i.e. $\Delta V = M \cdot \delta V$, $\Delta T = M \cdot \delta T$, $\Delta Q = M \cdot \delta Q$).

Compare this with the vastly simpler thermodynamics problem.  If we supply a total quantity of heat $\Delta Q$ to an ideal gas at constant pressure, then its temperature will increase according to $\Delta Q = c_P \Delta T = \frac{5}{2}N\Delta T \rightarrow \Delta T = \frac{2\Delta Q}{5N}$.  Since $\Delta U = \frac{3}{2}N\Delta T$ and $\Delta U = \Delta Q - P \Delta V$, it follows that
$$\Delta V = \frac{1}{P}(\Delta Q - \Delta U) = \frac{1}{P}\left(\Delta Q - \frac{3}{2}N \frac{2}{5N} \Delta Q\right) = \frac{2 \Delta Q}{5P}$$
and
$$\Delta T = \frac{P}{N} \Delta V = \frac{2 \Delta Q}{5N}$$
A: 
The initial work, =, at the very beginning will give the
  piston momentum, after initially being at rest. Thus, no further heat
  should be required to expand the gas after an initial infinitesimal
  flow of heat, unless there are friction forces that will bring the
  piston to rest.

What you are describing would not be considered a reversible isothermal process. For a reversible isothermal process the temperature and pressure of the perfect gas is always in thermal and mechanical equilibrium with the surroundings. In other words, the difference in pressure and temperature between the gas and the surroundings is continuously infinitely small and the expansion takes place infinitely slowly.

So what does it mean when one says a cylinder+piston over a thermal
  reservoir takes in an amount 1 of heat, and does work on the piston
  so that the gas gets expanded isothermally.

For it to be a reversible expansion, it means the temperature and pressure difference between the system and surroundings is infinitely small, i.e., zero in the limit.

Is 1 here an infinitesimal amount of heat (as  above)? Do we
  assume there are friction forces between the piston and the cylinder?

For the process to be considered reversible, there can be no friction forces involved.

If there is no friction, doesn't that mean the piston gets accelerated
  during the expansion, and that the additional heat that flows
  afterward, 1−, only adds momentum to the piston?

No, it doesn't mean the piston accelerates. There is no change in momentum. Acceleration and change in momentum requires pressure differentials between the system and surroundings. By definition, there is no pressure differential in a reversible process. Pressure and temperature differentials are due to disequilibrium. Disequilibrium is a cause of irreversibility. 
Keep in mind that a reversible process is an idealization in which we imagine temperature and pressure differences shrinking to zero. This permits us to establish the maximum theoretical efficiency of a cycle. All real heat transfer processes involve finite temperature differences and all real work processes require net forces. Thus all real processes are irreversible.

"there is no pressure differential in a reversible process" If there
  is no pressure differentials, then how can the gas expand, i.e., the
  piston move, after initially being at rest.

Now you are beginning to understand the difference between the idealization of a reversible process and the fact that any real process has to be irreversible. For the reversible process the pressure differential is considered to be so small that, in the limit, it is considered to be zero so there is no disequilibrium. As I said, this is an idealization in order to place an upper limit on efficiency. 
But we know that all real processes require disequilibrium. Heat transfer will not occur without a temperature difference. Work cannot occur without a pressure difference. 
Hope this helps.
A: If the gas pressure is not already accelerating the piston even before it heats, there must be some force already equalizing the gas pressure on the piston. As the gas heats and pressure increases, then it can overcome the opposing force by moving the piston until forces are equal again.
