The comparison is viable, here's why:
Let's choose the positive $x$-direction to point upward, perpendicular to the water's surface. By Archimedes' principle, the magnitude of the buoyant force on an object of volume $V$ equals the weight of the displaced water;
$F_B = \rho_w V g$
where $\rho_w$ here denotes the density of water. The buoyant force points in the positive $x$-direction.
In addition to the buoyant force, an object submerged in water will experience a gravitational force
$F_g = \rho V g$
pointing in the negative $x$-direction, where $\rho$ is the density of the object. It follows that the net vertical force on the water is
$F_x = F_B - F_g = (\rho_w-\rho)V g$
Let's assume that the object is rigid, so that its volume changes negligibly with depth, and let's make the approximation that the density of water varies negligibly as a function of depth as well, then the work done in moving such an object to a depth $d$ below the water's surface along the $x$-axis will simply be
$W = F\Delta x = (\rho_w-\rho)V g(0-d) = \boxed{(\rho-\rho_w)V g d}$
Notice how similar this looks to $mgd$, the change in potential energy of moving an object upwards under the influence of gravity by an amount $d$. In fact, we could think of the water as effectively decreasing the mass of the object so that it has an effective mass $m_\mathrm{eff} = (\rho- \rho_w)V$, and then the analogy becomes clear
An important caveat to all of this is that water is viscous, so they will be an additional drag force you have to contend with that will change the answer in the event that you submerge the object quickly. However, by moving the object sufficiently slowly, you can make this contribution as small as you'd like. You can also relax the constant water density and constant object volume assumptions if you know how these things change with depth. If you'd like more details on this, then let me know!
Cheers!