Remembering that entanglement is a property of the global quantum state, the statement "one of the entangled objects is at some finite temperature" makes no sense without some additional information. Either the local (marginal) states of $A_1$ and $A_2$ are thermal, or the global state is thermal, but not both.
If the two objects are called $A$ and $B$, their Hamiltonian can be written
$$H = H_A + H_B + H_{AB},$$
where $H_{AB}$ denotes their interaction, and $H_{A}$ ($H_B$) only affects $A$ ($B$). If the entire system is at thermal equilibrium, the quantum state is the Gibbs ensemble
$$ \tau_{AB} = \frac{\mathrm{e}^{-\beta (H_A + H_B + H_{AB})}}{\mathcal{Z}_{AB}},$$
where $\beta$ is the inverse temperature and $\mathcal{Z}_{AB} = \mathrm{Tr}[\mathrm{e}^{-\beta (H_A + H_B + H_{AB})}]$ is the partition function. However, if the subsystems are locally at equilibrium, the state of (say) $A$ is
$$ \tau_A = \frac{\mathrm{e}^{-\beta H_A}}{\mathcal{Z}_{A}}.$$
It is easy to see that, in general, $\tau_A \neq \mathrm{Tr}_B [\tau_{AB}]$, unless $H_{AB} = 0$. However, in this case $\tau_{AB}$ is not entangled (it is a product state).
If system is globally in an entangled thermal state $\tau_{AB}$, then the local states of $A$ and $B$ will not be at thermal equilibrium at inverse temperature $\beta$. Conversely, if the subsystems are at equilibrium, the system as a whole will not be. Note that this argument makes no reference to entanglement specifically, but rather just correlation in general. The same is true for composite classical systems.
On the other hand, one could consider entangled global states which lead to local equilibrium states, but where the subsystems are at different apparent temperatures. In this case one can see really weird effects, such as heat flowing from the "colder" subsystem to the "hot" one, apparently violating the thermodynamic arrow of time. See Partovi, Phys. Rev. E 77, 021110 (2008) (arXiv version here).
