No, that is not enough to say that $B=0$. You must also consider that $$\nabla\times E=-\frac{\partial B}{\partial t}$$ which means that for a magnetic field that is constant spatially but not in time, your conditions would be true but your $B$ field would not be $0$