Yes, it's possible, but as others have mentioned, such "hot cubes" would be rather dangerous. Perhaps a self-warming coffee cup is possible, but it's still not a great idea.
To be specific, let's assume we're using plutonium-238, which has a half-life of ~87.7 years.
Plutonium-238 oxide pellet glowing from its decay heat
It primarily decays by alpha emission, but there's a probability of ~$1.9×10^{-7}$% that a decay instead results in spontaneous fission, releasing neutrons and hazardous daughter isotopes. Pu-238 also emits some X-rays under 20 keV, and some gamma rays at 0.0436, 0.0996, and 0.152 MeV.
The density of plutonium-238 at room temperature is about 19.8 g/cm³. It generates about 0.57 watts per gram, or 11.3 watts per cm³, so you need a few cubic centimetres to keep your coffee hot, and that's not counting the shielding.
It's easy enough to shield against the alpha particles. A sheet of paper stops alphas. A smoke particle stops alphas - that's how smoke detectors work (although they use americium-241 as the alpha source).
The X-rays and gammas are a different story. You can shield against the energies mentioned using metal, but you need a couple of centimetres, and some gammas will always make it through the shielding material. The shielding attenuates the radiation exponentially: it works like half-life. So if a certain thickness of shielding allows half the gammas through, doubling the thickness allows 1/4 of the gammas through, and triple the thickness allows 1/8 of the gammas through.
Another problem is that a RTG pellet cannot be totally sealed, it needs to "breathe". Alpha particles are just high speed helium nuclei, and that helium needs to escape. So the shielding needs to be a little porous. Fortunately, helium atoms are very good at leaking through stuff, and helium can leak through pores that are too small for water or air molecules to pass through. Still, over its half-life, 238 g of Pu-238 emits 2 g of helium, which is ~11 litres at STP.
Finally, Pu-238 isn't exactly cheap. :) I suppose the price could drop if production increases, though. It's made from neptunium extracted from reactor waste, and that waste contains some really nasty fission daughter isotopes, which can only be handled remotely. And waste can also contain some fissile material which emits neutrons, which tends to make other stuff radioactive.
The world reserves of Pu-238 are currently rather low. Both NASA and the ESA would like more to be produced for use on deep space missions, and a few years ago it was announced that Pu-238 was to be ramped up, but we're still waiting. There's some discussion (with relevant linked articles) on this topic on the Space Exploration stack.
Despite all of these issues, Pu-238 was used in heart pacemakers half a century ago. From The Oak Ridge Associated Universities (ORAU) Museum of Radiation and Radioactivity:
Plutonium Powered Pacemaker (1974)
Pacemakers are used to stimulate a regular heartbeat when the body's natural electrical pacing system is irregular or not transmitting properly
At present (2003), there are between 50 and 100 people in the U.S. who have nuclear powered pacemakers [...] containing 2 to 4 curies of plutonium-238.
The unit's electronics are embedded in epoxy. The hard titanium case is designed to withstand any credible event including gunshots and cremation.
Dose rates at the surface of the pacemaker are approximately 5 to 15 mrem per hour from the emitted gamma rays and neutrons. The whole-body exposure is estimated to be approximately 0.1 rem per year to the patient and approximately 7.5 mrem per year to the patient's spouse.
Size: ca. 2.75" in diameter.
