Ultimately, the sea would rise on the other side of Earth, but maybe not where you stand, assuming your finger is big enough.
The first approximation to answering this question is that, as long as you are not floating on the
ocean, directly or in a boat, the effect is identical to increasing
the volume of water by an amount V equal to the immerged volume of
finger. But it will take some time for the effect to be felt around
the world, as it must propagate.
However, is there any effect worth considering ?
How much of a rise for a normal finger ?
Is there a significant effect. Let's do the back of the envelope
calculation. We will assume that the volume of the finger is about
10cc (cubic centimeters), i.e. $10^{-5}\ m^2$. The see surface of
Earth is 361 millions square kilometers, i.e., $3.61\ 10^{14}\
m^2$. Assuming you have an ideal incompressible liquid with no
granularity (which is true as a first approximation), we want to know
how much the level rises. We only have to divide the added volume by
the total surface, i.e.: $\frac{10^{-5}}{3.61\ 10^{14}}\ =\ 1/3.61\
10^{-19}\ =\ 2.8\ 10^{-20}\ m$. That is not very much, but what is it
to a water molecule ?
We know that one mole of water contains $6\ 10^{23}$ molecules
(Avogadro's number) and weights approximately 18g (16 for oxygen and
twice 1 for hydrogen), i.e. has a volume of 18cc. We only have to
divide this number by Avogadro's to get the average volume taken by
one molecule: $18\ 10^{-6}\ /\ 6\ 10^{23}\ =\ 3\ 10^{-29}\ =\ 30\
10^{-30}\ m^3$. Assuming the volume taken by each molecule is a small
cube, to simplify computation (we look only for orders of magnitude),
we simply take the cubic root, which give a height of $3.1\ 10^{-10}\
m$.
So the rise to be expected is at best about one tenth of a billionth
of the size of a water molecule. Is that measurable ? I am not sure we
can measure anything that small. The only devices I can think of are interferometers. But
I doubt. It would probably require frequencies far too high and
energetic (this is really beyond my competence). Anyway it would not
make sense for measuring the rise of water level as the frontier
between water and air cannot be defined with that much precision, even
when water is absolutely still.
Now, considering real liquids and other real phenomena is a waste of
time given all the approximation, and the infinitesimal character of
what might happen, even compared to brownian motion of molecules.
If you want anything significant, you need a much bigger finger.
Let's try to do that.
The big finger case
There is more to this problem. Since you take this as an abstract problem, one may assume that an
abstract finger can have any mass and volume (nail included).
One may also assume that since you listed out parameters that should be
ignored, all others are fair game. I will come back to the standard size finger in the end.
Suppose you have a huge finger, with a mass in trillions of tons or
more (that is when, in reality, the point I am making may becomes weakly significant, I think), or if you
prefer a volume in thousands of cubic kilometers (just a cube 10km on
each side). An even bigger finger (25 millions km$^3$) is
currently sitting on the south pole, and is likely to be dipped in the
ocean this century (see link below).
Then other things happen that may be measurable. The
continents are themselves floating on the Earth mantle, which has a mass
density around D=4 kg/dm$^3$.
Dipping your huge finger (volume V) in the ocean will ligthen the
continental plate you are sitting on by a weight equivalent to the
volume V of water (of your finger). Hence the plate will rise (just
give it time, plated things are slow, as we learned from
turtles). However, since the mantle has a greater density D than water
(D is actually also the density ratio), it will rise only by a volume V/D, which is
about V/4. But one datum is missing: what is the surface of the plate
your are sitting on.
All I know is that plates are generally smaller than
oceanic surface, smaller than 1/4 of that surface: there are 8 major
plates, and the ocean are 72% of the planet surface. So I would think
that, while people on the other side of the planet will experience a
rise in water level. People on your tectonic plate will experience, in
due time, a fall in the water level, which is really due to the fact that your plate - the ground you stand on - is rising.
The reasoning above uses very larges actual figures, taken from the real Earth, to give some
reality to the analysis. As I said, it may well happen sooner than we
should wish, with these figures, as discussed in an answer to another question.
To conclude
But if we come back to the original abstract small finger in an
abstract smooth world obeying without fuss the laws of fluid statics,
the same reasoning can apply. And since the continental plate rise is
in the same order of magnitude as the see level rise, there is no
reason not to take it into account. Hence the above conclusion
is also true in the case of the small finger, absurdly non significant as it may be.