Can a Dyson Sphere around a Black Hole be built so that it would not radiate significant IR? This is related to this question If the sphere surrounded a BH and used it as a heat dump (as well as extracting energy from it by dropping in mass) could its exterior be engineered to match the cosmic background temperature?
 A: In principle, no, you cannot make a Dyson sphere which is indistinguishable from the CMB. The reason is fairly simple. Let's start with a blackbody DS which encloses nothing at all, and is so far from any nearby stars that no noticeable radiation reaches it. Since it is surrounded by CMB with an effective temperature of 2.75 K, it will reach an equilibrium temperature of 2.75 K. That's what thermal equilibrium means for a black body. Now if a star is inserted into the sphere, since the external radiation remains unchanged the surface of the DS must increase its temperature in order to radiate away the internally-generated heat.
Of course, you did specify "significant IR" so there is some wiggle room. Per the Stefan-Boltzmann Law, emitted power P is related to temperature T by $$P = \sigma T^4$$ where the Stefan-Boltzmann constant is $$\sigma = 5.67×10^{−8} W m^{−2} K^{−4}$$ 
From this you can calculate that, at 2.75 K, the emitted power of a black body is about 3.2 $\mu$W /$m^2$.
For a star the size of the sun, with an power output of about 3.8 x $10^{26}$ watts, this would require a DS area of about $$A = \frac{3.8 \times10^{26}}{3.2\times 10^{-6}} = 1.2\times 10^{32} m^2 $$
Solving for the radius of the sphere R gives $$R = 3\times 10^{15} meters$$
which is a just about $10^7$light-seconds, or 0.3 light-years. Since this is about 400,000 times the radius of the solar system (using the aphelion of Pluto as the marker - forget the Oort cloud for now), this suggests a pretty heroic structure, even by Dyson Sphere standards, and causes one to wonder exactly where the materials will come from. 
The power density emitted by the surface of the sphere will be twice that of the CBE, which will produce an apparent temperature of 3.27 K, or about 0.5 K above CBE. If that is close enough to "insignificant", there is no need to go further. You can, of course, decrease the difference by making the DS larger. Knock yourself out.
EDIT It's true that this has so far ignored the black hole, and there is a very good reason - a black hole is very small, particularly compared to the DS. But let's run the numbers anyways.
A solar-sized black hole (BH) has a diameter of about 6000 m. It's apparent area is then $$A' = 4\pi r^2 = 4 \pi ({3000}^2) = 1.1 \times 10^8 m^2$$
Since a blackbody radiates uniformly in all directions, the fraction of the energy emitted by any portion of the DS intercepted by the BH will simply be the BH area divided by half the area of DS. That is, if power is radiated uniformly in all directions, the power level produced by any point on the surface at the center will be the same as if the surface point were at the center radiating outwards to the surface. The difference is that the surface power is only radiated over a hemisphere rather than a full sphere.
So the efficiency $\eta$ of the BH as an energy absorber is $$\eta = \frac{A'}{\frac{A}{2}} = \frac{1.1 \times 10^8}{.6\times 10^{32}} = 2\times 10^{-24}$$
In the case of a civilization using power at solar system levels, the total power lost to the BH is $$P' = \eta \times P = 2\times 10^{-24} \times 3.8 \times 10^{26} = 7.6 \times 10^2$$ or 760 watts.
I hope it is clear that the BH heat sinking has little effect on the energy balance of the DS. In the case of a solid body, the swallowing of mass has a cumulative effect. Not so (in any meaningful way) for the energy which is constantly being generated by the civilization which operates the DS.
A: If you built the sphere, at the optimum  radial distance, then insulated the exterior as much as possible, would the gravitational redshift provided by the black hole not act to sort out your problem for you, if you want to dissipate it, as far as co-ordinate observers at any reasonable distance were concerned?
Would accretion discs and the massive gravity well pulling in everything around you not be a worry, in the future? I know it's a thought experiment now, but these are practical problems that must face any possible society capable of building these structures.
A: I don't think so Dirk. GRBs don't happen for nothing. See this 2001 paper by Friedwardt Winterberg. I think it's essentially correct. Why? Take a look at Einstein saying light curves because the speed of light is spatially variable. Also see Shapiro, and this Baez article:

Einstein talked about the speed of light changing in his new theory. In the English translation of his 1920 book "Relativity: the special and general theory" he wrote: "according to the general theory of relativity, the law of the constancy of the velocity [Einstein clearly means speed here, since velocity (a vector) is not in keeping with the rest of his sentence] of light in vacuo, which constitutes one of the two fundamental assumptions in the special theory of relativity [...] cannot claim any unlimited validity. A curvature of rays of light can only take place when the velocity [speed] of propagation of light varies with position". This difference in speeds is precisely that referred to above by ceiling and floor observers.

Stand on a chair and drop a pencil. Note that it hits the floor at say 4 m/s. That's  because the "coordinate" speed of light at the ceiling is circa 4 m/s greater then the coordinate speed of light at the floor. Also note that gravity is not a force in the Newtonian sense. As the pencil falls, potential energy is being converted into kinetic energy. When the pencil hits the floor this kinetic energy is dissipated, and you're left with a mass deficit. That's because the  potential energy is mass-energy, in the pencil. Then when you drop it into a black hole, all this is taken to extremis. Falling bodies do not slow down. Your pencil falls faster and faster and faster, and is allegedly going close to the speed of light at the event horizon, which would represent a huge mass deficit. Only at the event horizon, the coordinate speed of light is zero, and the pencil is supposedly falling 299,792,458 m/s faster than that. Because of the difference in the coordinate speed of light up here as opposed to down there. 
Something's wrong. And what Winterberg is saying is that something's got to give, like the wave nature of matter. Waves break. Electrons etc cannot maintain their identity. They suffer a form of annihilation, and the result is a gamma ray burst. I expect there would be neutrinos too, but it's still a pretty dramatic conversion of matter into energy. It would be tricky to insulate against that. As for how your managed BH Dyson sphere might fare, have you ever seen a Chinese lantern catch fire? Poof, gone. 
