In your problem, "Earth" is not an isolated system. The combined "Sun-Earth" system, however, is, so we can know that the angular momentum of the Sun-Earth system is conserved. As the earth's mass is accelerating the sun, you have to take its angular momentum into account as well.
While the mass and size of the sun mean that we can ignore its motion with respect to the rest of the solar system, you can't do that for your calculation.
Alternatively, you can consider the sun's gravitational force on the earth to be a torque in your case because the force does not go through your reference point.