The magnetic field is different, and is not the same result. The result using concentric circles actually makes use of the rotational symmetry of the system, but the same symmetry does not hold for the rectangular system. Don't expect a nice formula - these things can be difficult to calculate explicitly! Ampere's law and the Biot-Savart law still hold, but they may not be easy to apply. However, at large $r$ (imagine zooming out so that the whole rectangle looks like a point), your formula will be a very good approximation. You might be able to easily get further exact terms in your approximation by using the multipole expansion method.
You might be tempted to relate this to a charge carrying conductor, where the electric field always is exactly normal to the surface and is zero inside the surface. Obviously this will cause a problem at the corner of the conductor. You'll have this weird singularity/discontinuity. But you can draw the approximate fringe fields pretty well: Just connect the surface normals with lines. So the fringe field of a charge carrying rectangular cross-section conductor can be visualized easily, if not calculated.
If your wire carries all its current at the surface -- maybe it's just a shell over an insulator or something -- then this boundary condition tells you that the magnetic field, just outside the rectangular surface of the shell, will be of constant intensity and always parallel to the surface (clockwise or counterclockwise depending on which way your current is flowing, and always perpendicular to the current. Just use the righthand rule).
If your wire is an actual thick wire with a volume current, then it's harder to say. In the case of a rectangular cross section with constant charge density, you can draw the potential as a squarish lump and visualize the electric field that way. You can do the same thing with a $B$-field, but now you need a vector potential. From certain equations for the vector potential and a certain curl identity you can see that a prescription for finding the fringe field is this: Draw (in 3D, with two axes being the plane of the wire cross section and another being your potential) the squarish lump that is the potential for the corresponding constant charge density problem, slice off loops of constant potential (which will be approximately squarish), and then traverse them one way or the other (according to the right-hand rule). This will be the direction of the magnetic field.