My title question relies on the answer to this question: Under the standard model and assuming baryon symmetry, is it possible (however unlikely) that some small local pocket of the universe might have had a little more surviving matter than antimatter?
If that's possible, than by the weak anthropic principle, it seems to me that we can't take a local imbalance as evidence of a fundamental asymmetry. Such an unlikely event is bound to happen countless times in a sufficiently large universe - and we have no reason to believe that we don't live in a sufficiently large universe.
Because we could only have come to exist in one of those rare, unlikely pockets with a local imbalance, we can't use that local imbalance as evidence of anything global. It may just as well be seen instead as evidence that the universe is large enough to have such pockets.
The observable universe might be profoundly unusual by the standards of the universe as a whole... but in the parts of the universe that had a more equal distribution of matter and antimatter, there are no planets or stars, and certainly no intelligent life present to wonder about symmetry!