This was also a bit confusing for me at the beginning - and what I think the real problem here is is how that integrals are notated and explained and, in particular, the stereotypical "calculus textbook" idea doesn't really make these things work very well. The problem is on the maths side, not the physics.
The integral expression is correct because you are adding up each tiny little bit of mass. Think about grinding down the object into a bunch of tiny grains, and then you add up the contributions from each little grain. That's why it is $dm$.
However, the trouble arises when this "strange" integral runs headlong into a dogma that is taught in beginning calculus which is "the 'd' thing just tells you what variable you're integrating with respect to". Hence, when you run into
$$I = \int_\text{obj} r^2\ dm$$
you're left with an odd puzzle: there is no $m$ in $r^2$, so $r$ is just a constant! Hence, why is this not equal to
$$I = r^2 \left(\int_\text{obj}\ dm\right)$$
? Or, if $r$ is to be taken as a function of $m$, that clearly becomes very bad because many different points may have the same mass $m$ and hence this function would be extremely multivalued, very ill-defined and so it would be likewise very hard to make sense of the integral.
But the real trick is this: calculus books, and the integration notation itself, lie to you. You see, in fact the $'d'$ bit in an integral is better understood - however you want to actually formalize this, as being an infinitesimal increment, as you might suggest. $d(\text{variable})$ means the infinitesimal increment when the named variable changes. It does not mean that that is directly the changing variable used when integrating. Instead, to make the distinction clear, one would do better to write integrals much more like a summation because, in effect, they are: from the infinitesimal perspective and integral is a summation of uncountably infinitely many contributions, while a summation is only countably many - with, of course, attendant increased sophistication in how to make that rigorous. Hence, this:
$$\int_{a}^{b}\ f(x)\ dx$$
should better be understood as this:
$$\int_{x=a}^{b}\ f(x)\ dx$$
where the variable being integrated over is now named explicitly, and there is no reason that $dx$ need have anything at all to do with this. Instead, this is to be understood by direct analogy to the sum expression
$$\sum_{n=a}^{b} a_n$$
where $a_n$ is a complete expression. The point then is that the analogy of $a_n$ for the integral is not $f(x)$, but rather all of $f(x)\ dx$ - an expression in its own right which involves this "infinitesimal value" (make that a Robinsonian nonstandard number, make it a differential form, make it the naive "limited sum" approach in elementary calc, or whatever other formalism you like best) $dx$ in it. The $dx$ there is not really to "tell us what variable to integrate with respect to", but is to instead, in an informal sense, keep the uncountable sum from blowing up. To drive that further, note that in this framework, it makes absolute sense to just write
$$\int_{x=a}^{b}\ f(x)$$
and then, unless $f$ is zero at all but a countable number of points, this integral will be infinite or divergent. Hence you need to include a compensating factor, and it's got to be very strong: infinitely strong, in fact, so it should be infinitesimal. And the most common infinitesimal to use simply happens to be the direct change in the variable $x$ as it smoothly is moved from $a$ to $b$ just as the index variable $n$ in a discrete summation
$$\sum_{n=a}^{b} a_n$$
is ticked discretely from $a$ to $b$ in steps of 1. And this infinitesimal is $dx$.
So what is happening in your integral, then? Well, $dm$ is an infinitesimal, $m$ is a variable, but such $m$ is NOT directly the variable of integration, i.e. the variable being incremented. Instead, the incrementation variable is a position, or point within the object, and so we should perhaps better write the integral as
$$I = \int_{P\ \in\ \text{obj}} r(P)^2\ dm$$
where now $r(P)$ is the radial coordinate of $P$ away from the axis, as in cylindrical coordinates, and we are then sweeping that point to and fro hither and thither all thence and around the interior of the object, at each point getting contributions $[r(P)]^2\ dm$ to the moment of inertia. The $dm$ is now just an infinitesimal chunk of mass centered at $P$. $P$ is changing, and as a result of that, so are both $r$ and $dm$, together.
Finally, to make it more explicit, we have
$$dm = \rho(P)\ dV$$
and the volume element $dV$ depends on the coordinate system we are using. If we are using the most natural choice, cylindrical coordinates, then $P = (r, \theta, z)$, $dV = r\ dr\ d\theta\ dz$, and "sweeping $P$ all over the inside of the object" now corresponds to the more rigorous "scanning" of a multilayered integral:
$$I = \int_{P\ \in\ \text{obj}} r^2\ dm = \int_{z=-\infty}^{\infty} \int_{\theta=0}^{2\pi} \int_{r=0}^{\infty} r^2\ [\rho(r, \theta, z)\ r\ dr\ d\theta\ dz]$$
where again, note the notation!
Conclusion: It is $dm$ because we are breaking up into little chunks of mass, at small points in space. The confusion results from bad maths teaching and historical problems with notation that don't match actual practice best.