When we lift an object from ground we say we did work on object . We also say that gravity has done negative work because we did work to overcome gravity . When we did work our chemical energy was converted to potential energy of object. Does any change in energy comes from gravity which did negative work ?
Work is a measure of how much energy is transferred from one source to another. You can either say that a positive amount of energy was transferred from the chemical potential energy in your body to the gravitational potential energy of the object, or you can say that a negative amount of energy was transferred from the gravitational potential energy of the block to the the chemical potential energy in your body. These are two equivalent ways of saying the same thing. What you cannot say is both at the same time, because that would be double-counting the transfer of energy.
In general, the sign of the work done is entirely a choice of convention, which is set when you choose a direction of energy transfer to be defined as positive. You have defined a transfer of energy from your body to the object as positive work being done, but you could have just as easily defined a transfer of energy from the object to your body as positive work being done. The physics doesn't change either way.