You might be working in $1$D, but that doesn't mean that the right-hand side in Poisson's equation has magically changed: Poisson's equation still reads
$$
\nabla^2 V = \frac{1}{\epsilon_0}\rho,
$$
where $\rho$ is the volumetric charge density, with dimensionality of $\rm charge/length^3$, and the only thing that changes is that you're working in a stack of layers with translational symmetry in the $y$ and $z$ dimensions, so that
$$
\frac{\partial}{\partial y} = \frac{\partial}{\partial z} = 0,
$$
and the PDE becomes a one-dimensional ODE. The right-hand side remains a volumetric charge density, though.
EDIT: For clarity, the one-dimensional Poisson equation you write in the question,
$$
\frac{\mathrm d^2V}{\mathrm dx^2} = \frac{\rho(x)}{\epsilon_0}
$$
is only valid for situations of the following form,
if you'll forgive the hastily-drawn diagram, i.e. three-dimensional situations where there is complete translational equivalence on the $y$ and $z$ dimensions, and the charge density only depends on the $x$ coordinate. It should be clear that in this situation the source is a volumetric charge distribution, with a volumetric charge density $\rho(x)$, which eliminates the dimensionality problem in your question.
(This requires an infinite domain, which is obviously not physical, so you just consider domains which are large enough that edge effects are negligible, and you impose on the solutions of the Poisson equation the additional condition that they share the 2D translational invariance of the source. Infinite-span bulk charge densities can indeed get you into trouble, but asking for translational invariance, when done correctly, tends to work fine.)
On the other hand, this might not be what you want to describe - in some situations, what you really want to describe is an infinitely-thin wire of charge with longitudinal charge density $\lambda(x)$, surrounded by vacuum. In these situations, though, the solution is no longer independent of the $y$ and $z$ coordinates, and the $1$D Poisson equation is no longer appropriate.
(To see why, consider e.g. the longitudinal charge density $\lambda(x)=\lambda_0 \sin^2(\pi x/L)$, which oscillates at length-scale $L$. When seen from $|y|,|z|\ll L$, the potential $V(x,y,z) \sim \lambda(x) \ln(\sqrt{y^2+z^2})$ will track the local charge density, but if you see it from a far-away vantage point with $\sqrt{y^2+z^2}\gg L$, the local oscillations will wash out, and you'll be left with only the $x$-independent average $V(x,y,z) \sim \frac12 \lambda_0 \ln(\sqrt{y^2+z^2})$. This is not independent of $y$ or $z$, which means that you cannot drop those partial derivatives.)
If you do want to describe that situation using Poisson's equation, then eranreches and AccidentalFourierTransform's approach is the correct one: the charge density has the form
$$
\rho(\mathbf r) = \lambda(x) \delta(y)\delta(z),
$$
and (because the Dirac delta's dimensionality, $[\delta(a)]=1/[a]$, is fixed by its integration property $\int \delta(a)\mathrm da=1$) this still has the dimensionality of $\rm charge/length^3$ of a volumetric charge density.
