This is a hypothetical question, since electrons are elementary particles and protons are composite.
The solutions of the potential problem would give stable orbitals with smaller average radii. Here is a Bohr model solution for the muonic hydrogen, where the muon is 200 times heavier than the elecron. The energies become KeV instead of eV. To go to the mass of the proton another factor of 10 enters.
One could imagine a chemistry coming out of such atoms, except then one should imagine also a new electroweak sector, to reduce the probability of instabilities.
a) The stability of atoms comes because electron capture is very rare, due to the small probabilities of overlap with the nucleus and consequent creation of a neutron and a neutrino. The proximity and the large energies involved would increase these probabilities. Have a look at exotic atoms.
b)excited states of the "hydrogen" atom might have enough energy and overlap with proton to turn into a neutron + neutrino and the cosmological models would have to be redone.
c) as @CuriousOne has pointed out in a comment, muon catalyzed fusion has shown another instability with respect to normal solid state matter:
If a muon replaces one of the electrons in a hydrogen molecule, the nuclei are consequently drawn 207 times closer together than in a normal molecule. When the nuclei are this close together, the probability of nuclear fusion is greatly increased, to the point where a significant number of fusion events can happen at room temperature.
Once the compositeness of protons ( three quarks) enters the program a completely new standard model would be needed as a number of reactions would not go by energy balances alone , example :
The Feynman diagram for beta decay of a neutron into a proton, electron, and electron antineutrino via an intermediate heavy W boson
All in all my opinion is that no stable solid state would result from such a hypothesis.