Is it possible to construct simple circuits, that given a time-varying input, produce an output that represents the derivative or integral of the input with respect to time?
Apparently what you are looking for is called an "Integrator" and a "Differentiator". (I'm not too good with circuits, so I can't judge if these are simple enough.)
As David pointed out in his answer, the circuits are called the differentiator and the integrator respectively. They use operational amplifiers (op amps) to do this.
An op-amp isn't necessarily a "simple circuit" to build, since it consists of several transistors and hundreds of resistors and capacitors, but since it's extremely easily available as an IC, it's plenty easy to build an integrator or a differentiator circuit.