I don't have an answer to the question "why would one want to consider such crazy stuff in physics?" since I don't know much physics, but as a mathematics student I do have an answer to the question "why would one want to consider such crazy stuff in mathematics?"
What physicists call Grassmann numbers are what mathematicians call elements of the exterior algebra $\Lambda(V)$ over a vector space $V$. The exterior algebra naturally arises as the solution to the following geometric problem. Say that $V$ has dimension $n$ and let $v_1, ... v_n$ be a basis of it. We would like a nice natural definition of the $n$-dimensional volume of the paralleletope defined by the vectors $\epsilon_1 v_1 + ... + \epsilon_n v_n, e_i \in \{ 0, 1 \}$. When $n = 2$ this is the standard parallelogram defined by two linearly independent vectors, and when $n = 3$ this is the standard paralellepiped defined by three linearly independent vectors.
The thing about the naive definition of volume is that it is very close to having really nice mathematical properties: it is almost multilinear. That is, if we denote the volume we're looking at by $\text{Vol}(v_1, ... v_n)$, then it is almost true that $\text{Vol}(v_1, ... v_i + cw, ... v_n) = \text{Vol}(v_1, ... v_n) + c \text{Vol}(v_1, ... v_{i-1}, w, v_{i+1}, ... v_n)$. You can draw nice diagrams to see this readily. However, it isn't actually completely multilinear: depending on how you vary $w$ you will find that sometimes the volume shrinks to zero and then goes back up in a non-smooth way when really it ought to keep getting more negative. (You can see this even in two dimensions, by varying one of the vectors until it goes past the other.)
To fix that, we need to look instead at oriented volume, which can be negative, but which has the enormous advantage of being completely multilinear and smooth. The other major property it satisfies is that if any of the two vectors $v_i$ agree (that is, the vectors are linearly dependent) then the oriented volume is zero, which makes sense. It turns out (and this is a nice exercise) that this is equivalent to oriented volume coming from a "product" operation, the exterior product, which is anticommutative. Formally, these two conditions define an element of the top exterior power $\Lambda^n(V)$ defined by the exterior product $v_1 \wedge v_2 ... \wedge v_n$, and choosing an element of this top exterior power (a volume form) allows us to associate an actual number to an $n$-tuple of vectors which we can call its oriented volume in the more naive sense. If $V$ is equipped with an inner product, then there are two distinguished elements of $\Lambda^n(V)$ given by a wedge product of an orthonormal basis in some order, and it's natural to pick one of these as a volume form.
Alright, so what about the rest of the exterior powers $\Lambda^p(V)$ that make up the exterior algebra? The point of these is that if $v_1, ... v_p, p < n$ is a tuple of vectors in $V$, we can consider the subspace they span and talk about the $p$-dimensional oriented volume of the paralleletope given by the $v_i$ in this subspace. But the result of this computation shouldn't just be a number: we need a way to do this that keeps track of what subspace we're in. It turns out that mathematically the most natural way to do this is to keep in mind the requirements we really want out of this computation (multilinearity and the fact that if the $v_i$ are not linearly independent then the answer should be zero), and then just define the result of the computation to be the universal thing that we get by imposing these requirements and nothing else, and this is nothing more than the exterior power $\Lambda^p(V)$.
This discussion hopefully motivated for you why the exterior algebra is a natural object from the perspective of geometry. Since Einstein, physicists have been aware that geometry has a lot to say about physics, so hopefully the concept makes a little more sense now.
Let me also say something about how modern mathematicians think about "space" in the abstract sense. The inspiration for the modern point of view actually derives at least partially from physics: the only thing you can really know about a space are observables defined on it. In classical physics, observables form a commutative ring, so one might say roughly speaking that the study of commutative rings is the study of "classical spaces." In mathematics this study, in the abstract, is called algebraic geometry. It is a very sophisticated theory that encompasses classical algebraic geometry, arithmetic geometry, and much more, and it is in large part because of the success of this theory and related commutative ring approaches to geometry (topological spaces, manifolds, measure spaces) that mathematicians have gotten used to the slogan that "commutative rings are rings of observables on some space."
Of course, quantum mechanics tells us that the actual universe around us doesn't work this way. The observables we care about don't commute, and this is a big issue. So mathematically what is needed is a way to think about noncommutative rings as "quantum spaces" in some sense. This subject is very broad, but roughly it goes by the name of noncommutative geometry. The idea is simple: if we want to take quantum mechanics completely seriously, our spaces shouldn't have "points" at all because points are classical phenomena that implicitly require a commutative ring of observables, which we know is not what we actually have. So our spaces should be more complicated things coming from noncommutative rings in some way.
Grassmann numbers satisfy one of the most tractable forms of noncommutativity (actually they are commutative if one alters the definition of "commutative" very slightly, but never mind that...), and even better it is a form of noncommutativity that is clearly related to something physicists care about (the properties of fermions), so anticommuting observables are a natural step up from commuting observables in order to get our mathematics to align more closely with reality while still being able to think in an approximately classical way.
