Energy Conservation in Holding Weights This is a two part question, and it might have to do with biology a bit.
Part 1: I grab a heavy dumbbell and curl it. Halfway up I stop the curl and just hold it there with my elbow at 90 degrees. There is no net acceleration, so there is no net force on the object, so no work is being done. But clearly I am using energy to do this, as my muscles are tensed. Since my chemical energy is not going into any sort of mechanical energy, is it simply going into heat?
Part 2: If we assume that all of the chemical energy of my muscles turns to heat, what if I now lift the weight? If I accelerate this weight up, now I'm converting some of this chemical energy into mechanical energy, so less can go to heat. Doesn't that mean your arms get hotter if you don’t do any work onto the object than if you do? This just seems counterintuitive to me.
 A: 
Part 1: I grab a heavy dumbbell and curl it. Halfway up I stop the curl and just hold it there with my elbow at 90 degrees. There is no net acceleration, so there is no net force on the object, so no work is being done. But clearly I am using energy to do this, as my muscles are tensed. Since my chemical energy is not going into any sort of mechanical energy, is it simply going into heat?

If you hold the dumbbell in place, your physical effort does not produce physics work. The energy expended is internal. Richard Feynman in his physics lectures explains it this way:
The fact that we have to generate effort to hold up a weight is simply due to to the design of striated muscle. What happens is when a nerve impulse reaches a muscle fiber, the fiber gives a little twitch and then relaxes, so that when we hold something up , enormous volleys of nerve impulses are coming in to the muscle, large numbers of twitches are maintaining the weight, while other fibers relax. When we hold a heavy weight we get tired, begin to shake, ...because the muscle is tired and not reacting fast enough.

Part 2: If we assume that all of the chemical energy of my muscles
  turns to heat, what if I now lift the weight? If I accelerate this
  weight up, now I'm converting some of this chemical energy into
  mechanical energy, so less can go to heat. Doesn't that mean your arms
  get hotter if you don’t do any work onto the object than if you do?
  This just seems counterintuitive to me.

When you are simply holding the weight, you do no physics work. But you expend chemical potential energy of your body when your muscle fibers alternatively "twitch and relax" as described by Feynman. Think of it as internal physiological work as opposed to external physics work. If you lift the weight (even at constant velocity) you are doing positive work separating the dumbbell from the earth resulting in the Earth-dumbbell system gaining gravitational potential energy. Some of your chemical potential energy will still be converted to thermal energy simply because your body is not 100% efficient performing the work. 
Conversion of chemical potential energy to thermal energy does not necessarily mean you arms will get "hotter" (i.e., increase in temperature), since the blood that continually circulates between your arms and the rest of your body will quickly distribute the thermal energy throughout your body.
Hope this helps. 
