One possibility is that an electron and an anti-down quark simply scatter off each other. Another possibility is that the electron turns into an electron neutrino and the anti-down quark turns into an anti-up quark. With lower probability, other processes are possible in which more than two particles emerge.
They don’t annihilate because they aren’t antiparticles of each other. Annihilation into photons would not conserve electric charge, among other things.
I spent a few minutes looking for experiments studying collisions of electrons and antiprotons (which contain anti-down quarks) but didn’t find any.
Scattering of electrons from up and down quarks in protons and neutrons is well-studied, initially at SLAC in the late 1960’s.
As @probably_someone mentioned in a comment, there are virtual anti-down quarks in a proton, so some electrons colliding with a proton can be thought of as colliding with such an anti-down quark. This means that $ep$ scattering experiments presumably do test what you are asking about.
The antiparticle version of your question is well-studied with real (non-virtual) particles. For example, this paper discusses positron-proton collisions. Positrons are the antiparticles of electrons, and protons have down quarks inside.
[I’m] trying to get a better understanding of how certain particles interact with each other.
If you are interested in the possible interactions allowed by the Standard Model, this diagram from the Wikipedia article will be of interest. Unfortunately, it doesn’t include interactions involving Higgs bosons.