It is not possible to make a gigantic neutron ball because the neutrons will beta decay to protons, and emit electrons (and neutrinos) and the protons will sit on the surface, and blow the nugget apart through their electrostatic repulsion. The only way to resist this fragmentation is to have a star-massed neutron cluster, so that the cluster is held together in the compact configuration by gravity. There is no other attractive force which can counteract the electrostatic repulsion.
There is a large gap in size between the largest stable nucleus and the smallest stable neutron star essnetially caused by the weakness of gravity. If you made a neutron ball, it wouldn't even be stable for 8 minutes, since the only reason the neutron is long-lived is that the proton and neutron mass are so close, but in nuclear matter empty of protons, a proton's binding energy to the surrounding neutrons reduce the lifetime to nothing. Heavy neutron rich nuclei tend to beta-decay much more quickly than a free neutron.