We can derive this exponential behavior from the laws of thermodynamics as follows: on the one hand, heat is a quantity that is subject to a conservation law (first law of thermodynamics): locally, the amount of heat $H$ stored in a body can only change due to a flow of heat $J$ out of it. On the other hand, the second law of thermodynamics tells us that the flow of heat between two bodies is driven by a difference in temperature $T_2-T_1$. Let's say you have a reservoir at some constant temperature $T_0$ (temperature of the room your device is in), then the flow of heat out of your device is to first approximation given by $J\propto (T_{\textrm{device}} - T_0)$. Temperature (in Kelvin) of the device is proportional to the amount of heat stored in it, $T\propto H$, and thus the change of temperature of your device after some short time $\delta t$ is
$$\delta T_{\textrm{device}} \propto \delta H = -c J \delta t = -c' (T_{\textrm{device}}-T_0) \delta t,$$
where $c$ and $c'$ are some proportionality constants which will involve the specifics of your device and your room.
Divide both sides by $\delta t$ and take the limit $\delta t \to 0$, and you obtain the standard differential equation for an exponential, $$\frac{dT_\textrm{device}}{dt} \propto -(T_\textrm{device} - T_0)$$.
What happens if you turn on the power? Lets for simplicity assume that your device is fairly useless and does no work besides heating up. In keeping with the first law of thermodynamics, its power consumption will be an additional flow of heat which we will have to subtract from $J$ (we defined it as outward flow). We will have to be a bit more careful than before because temperature is what is called an intrinsic quantity (not dependent on the size, mass, volume of the object) whereas the total heat stored in the device and its power consumption are extrinsic (they increase as the object becomes bigger). Instead of thinking about how the various proportionality constants are affected, lets consider average power consumption per volume of your device $w=W/V$ and let's think of the heat flow $J$ as the heat flow per unit volume. Then we will have $J = k(T_\textrm{device}-T_0) - w$, where $k$ is a device-dependent constant that encodes how much work it takes to heat up a unit volume of "device" by one degree. We see that if we define $T_\textrm{equilibrium} \equiv T_0 + w/k$, we recover the same differential equation, but with $T_0$ replaced by $T_\textrm{equilibrium}$,
$$\frac{dT_\textrm{device}}{dt} \propto -(T_\textrm{device} - T_\textrm{equilibrium}),$$
and the system exponentially approaches this temperature: if you turn on power, your system will exponentially approach a temperature above room temperature, if you turn it off again, it will exponentially return to room temperature.
Note that you need nothing besides the laws of thermodynamics to reach these conclusions. The approximations that we make are that the temperature change is small enough to neither affect the way heat is transferred (no change in convection, e.g.) nor to affect the heat capacity of the device, i.e. $k$ is constant.