Do we require full Lagrangian to be gauge invariant or only equations of motion?
The probable expectation of a newbie about the meaning of the statement "$X$ is gauge-invariant" is that numerical value of the physical concept $X$, all else being equal, is the same in both gauges. In this sense, the above Lagrangian is not gauge-invariant.
The Lagrangian density function is gauge-invariant in a different sense though. The Lagrangian density function for EM field is generally assumed the same in all gauges, that is, the form, the mathematical function of its arguments $A^\mu, j^\mu$ is assumed to be the same. This assumption says that in the first gauge, the Lagrangian density function to be used in the action principle is
$$
\mathscr L = \mathcal l(A^\mu,j^\mu) = -~\frac{1}{4}(\partial_\mu A_\nu - \partial_\nu A_\mu )(\partial^\mu\!A^\nu - \partial^\nu\!A^\mu) ~-~ j^\mu\!A_\mu
$$
and in the second gauge, the Lagrangian density function to be used is
$$
\mathscr L ' = \mathcal l(A'^\mu,j^\mu) = -~\frac{1}{4}(\partial_\mu A'_\nu - \partial_\nu A'_\mu )(\partial^\mu\! A'^\nu - \partial^\nu\!A'^\mu) ~-~ j^\mu\! A'_\mu.
$$
Thus, as shown, the function used is actually the same in both gauges - it's the function $l$ - it just uses different value of the argument, $A'$ instead of $A$. The relation defining values $A'$ is $A'^\mu = A^\mu + \partial^\mu\! N$, where $N$ is an arbitrary function of time and spatial coordinates.
Value of these two functions at the same spacetime point need not be the same, because value of the argument $A$ and its derivatives may have changed due to the gauge transformation. So gauge-invariance of the function does not imply gauge-invariance of the function value. Often the latter is not present.
Form of the equations of motion is gauge-invariant too, this is implied already by the fact the Euler-Lagrange equations in terms of $t,A,L$ are gauge-invariant, and the Lagrangian density function is gauge-invariant.
This topic is often confused by the ideas that the value of the Lagrangian, or terms in the equation of motion, or even the action itself, should be gauge-independent. These are not needed to formulate and apply the principle of stationary action and to get the equations of motion. They are additional requirements that may be imposed, but often they are not satisfied.
For example, above, values of both Lagrangian densities at the same space time point differ when at that point, the expression $j^\mu\, \partial_\mu N $ is non-zero.
The $FF$ terms have the same value since the $FF$ term is a function of gauge-invariant fields $\mathbf E,\mathbf B$, but the jA terms have in general different values, because $A$ has changed. The new Lagrangian density value differs from the value of the original one by the value of the term
$$
j^\mu\,\partial_\mu N.
$$
Similarly for the actions
$$
S = \int_T \int_V ~\mathscr{L}~ d^3\mathbf x\, dt,
$$
$$
S' = \int_T \int_V ~\mathscr{L}'~ d^3\mathbf x\, dt.
$$
They have the same form, but their values in different gauges, in general, differ.
Note. In the old version of this answer, I wrote "difference in value among the two Lagrangian densities does not lead to any difference in values of actions". This common belief is in general incorrect, because even assuming local conservation of charge, there will be boundary terms in the primed action, which a well-chosen $N$ will make sure they make a non-zero contribution to action.
This is because the difference of two actions due to change to $A'^\mu = A^\mu + \partial^\mu\!N$ is
$$
\int_\Omega j^\mu\,\partial_\mu N ~d^3\mathbf x\, dt.
$$
And value of this is obviously gauge-dependent.
It can be expressed as sum of two integrals:
$$
\int_\Omega \partial_\mu(j^\mu N) - \partial_\mu j^\mu\, N ~d^3\mathbf x\,dt = I_1 + I_2.
$$
The second integral $I_2$ is zero because of local conservation of charge, which is a constraint on $j$ we assume independently of the Lagrangian:
$$
\partial_\mu j^\mu = 0.
$$
The first integral, despite common belief to the contrary, is, in general, not ignorable.
It can be expressed as surface integral over the spacetime region $\Omega$ (via the 4D variant of the Gauss theorem); in the simple case of a box in space and interval $t_1..t_2$ in time, we can express it as:
$$
I_1 = \int_V N(t_2)\,c\rho(t_2) - N(t_1)\,c\rho(t_1) \, d^3\mathbf x + \int_{t_1}^{t_2} \oint_{\delta V} N\,\mathbf j \cdot d\boldsymbol \Sigma\, dt.
$$
If there is non-zero charge density at the boundary times, or current density on the spatial boundary, this integral can be easily made non-zero by an appropriate choice of the function $N$ (for example, $N$ that does not have a constant value throughout the spacetime region). $N$ can be any function we choose, thus the action value is not, in general, gauge-invariant.
Probably there will be some push-back on this statement from smart people, saying that action is gauge-invariant, because that is how it is taught, pointing out that the boundary term is immaterial, as it does not influence equations of motion, so the meaning of "action is gauge-invariant" is all about the equations of motion. Yes indeed, the boundary terms do not influence equations of motion, because those are gauge-invariant. Nevertheless, my point here is that value of the action is not gauge-invariant, only the fruit of the action and the action principle is - the equations of motion. Describing this fact by statements like "action is gauge-invariant" is pretty misleading.
The action value will be gauge-invariant when adopting this additional restriction on the spacetime region considered: that current density $j^\mu$ vanishes on the boundary of the spacetime region.
In our simple case, this implies 1) current cannot pass through the spatial boundary of the region at any time in between $t_1,t_2$; 2) charge density is zero everywhere in the region at the initial time $t_1$ and at the final time $t_2$. Thus action being gauge-invariant is a very special condition, not a general property. It is, for example, obeyed in empty spacetime with no charges. Then the action is just integral of the $FF$ term, which has gauge-invariant value.
Summary: both the Lagrangian density function $\mathscr{L}$ and the action functional of $\mathscr{L}$ are gauge-independent functions of their arguments - potentials, their spacetime derivatives, and possibly other (matter) variables. But Lagrangian density value at some spacetime point is not gauge-independent, because one argument in the function $l$ has different value - $A'^\mu$ instead of $A^\mu$. Action value is not, in general, gauge-independent either, as difference in value of the Lagrangian density integrates to a non-zero contribution on the region boundary. Only in special cases, such as a region of space in which charge density vanishes at the beginning and at the end of the time interval, and no current flows through the spatial boundary at any time in between, is the value of action guaranteed to be gauge-invariant.
