A major part of the reason for this is due to the temperature of the ground. While the length of days in the Summer are effectively a mirror of those in Spring, you must take into consideration more than that.
When Spring commences in temperate climates, it is (usually) immediately preceded by winter. Due to the Winter, the ground and/or surrounding bodies of water are very cold. This has the effect of cooling the air for the first part of Spring while the ground/water begins to thaw/warm up. Furthermore, it takes much longer to warm or cool a body of water than a mass of air; even longer to warm or cool the ground and water. Therefore, as Spring progresses and the days become longer (also meaning the Sun is higher above the horizon, thus providing more heating power), the sunlight must first overcome the cooling effects of the ground and water bodies. Near the end of Spring - when the days are sufficiently long and the Sun is much higher above the horizon - you should notice the weather becoming hotter. This is because the ground and water has had time to warm up, which means it is not constantly cooling the air and making it feel colder.
When you then transition to Summer, the ground is already sufficiently warm but the days are still long and the Sun is still high in the sky. This means the Sun can heat the ground, water, and air even more and without any cooling effects. This allows the Summer temperature to be easily higher than that of the Spring temperatures. If Summer were immediately preceded by winter, you might notice the weather getting warmer much more quickly, but the average temperature would be very close to that of the Spring.
It is for much the same reason that Winter is colder than Autumn, even though they have the same amount of daylight hours.