So I am a little confused on how to deal with the Kilowatt hours unit of power
Kilowatt-hours is not a unit of power, it is a unit of energy.
I think the easiest way to clarify this may be to think in terms of energy and power and try to avoid terms like kilowatt-hours per hour.
The average power consumption of a US household is 8,900 kW-hr per year and 25 kW-hr per day and approximate 1 kW-hr per hour. Does this mean that the power output of my fuel cell is 1 kW and if I wanted to use it for the entire day would it have to be designed to be 25 kW?
A fuel-cell with a power output of 1042 Watt operated for a day will deliver the 25 kWh of energy you mention. (Assuming you have a hydrogen-cylinder delivery man who visits regularly with fresh supplies)
It will not allow you to use a kettle.
It may not allow you to turn all the lights on and watch TV
It may help if you translate everything into SI units
1 kW is 1,000 Watts of power
1 kWh is 1,000 Watts for 3,600 seconds = 3,600,000 Joules of energy
So your target house uses 90,000,000 Joules in a day.
If the rate of energy usage is constant and never varies you can supply that amount of energy at a constant rate of 1,042 Joules per second (otherwise known as 1,042 Watts or just over 1 kW)
A typical kettle for boiling drinking water needs over 2 kW of power.
I eat 2,500 kCalories in 24 hours. (10,460,000 Joules)
That is an average of 121 Joules every second.
However I don't eat a tiny speck of food every second 24 hours a day. I spend some of that time sleeping and on activities where I do not necessarily want to concurrently eat blueberry pie at that rate.