It seems like the previous answers were based on the abstract or third-party articles about the abstract, but I gather from this Wired article that the full paper provides more details (behind a paywall, so I can only speak for what the article says).
The described tests were on the Cannae drive, the inventor of which believed required slots in the drive to function. The "null test" previously described was a Cannae drive without slots, which actually did produce thrust, showing that the slots weren't necessary (much like the previously-tested EmDrive). The experiment's actual control, an inert load, didn't produce any thrust. Tellingly, when they reversed the drive, the thrust reversed as well.
They were testing in a vacuum, seemingly as insulated from the outside world as possible.
The previous Chinese tests used much more power - kilowatts to NASA's watts. If this is a true effect then it's expected NASA's experiment would produce less thrust.
I wouldn't bet money on this, but I also wouldn't discount it yet.