Your question addresses the direction of gravitational force on the pendulum, but there is a deeper idea that defines the direction of gravity. The concept of the gravitational force that mass 1 exerts on mass 2 is inherently the concept of a force vector, made explicit by a mathematical formulation that references a unit vector pointing from mass 2 to mass 1:
$\vec{F_{1,2}}=\frac{Gm_{1}m_{2}}{r^{2}}\hat{r}_{2,1}\:\:\:\:$(1)
This formula is a recipe for drawing the force vector that mass 1 exerts on mass 2. The unit vector $\hat{r}_{2,1}$ always points in the direction from mass 2 to mass 1. You should confirm for yourself that with this definition of the unit vector, we can switch the labels of mass 1 and mass 2 to find the force on the other mass, and that the force on the other mass will also be attractive.
Notice that we could also write the formula as
$\vec{F_{1,2}}=-\frac{Gm_{1}m_{2}}{r^{2}}\hat{r}_{1,2}\:\:\:\:$(2)
where the unit vector points from mass 1 to mass 2. Notice the trade-off in this definition: it preserves a typographical convention of typesetting each mass 1 reference before the counterpart mass 2 reference, but it now requires you to mentally flip the sign of vector using the minus sign. (If we didn't flip the sign, the force would be repulsive, which is the general structure for force interactions between like electrical charges).
The two vectors produced by both equations are identical, because in space they have the same length and point in the same direction.
Near the earth's surface where we assume constant gravitational acceleration, we can write the equation for downward force on the pendulum a few different ways, but they all yield the same vector:
$\vec{F_{g}}=\:\:\:mg\hat{y}\:\:\: $where$ \:\:\:(g = -9.8\: m/s^{2})\:\:\:\:$ (3)
$\vec{F_{g}}=-mg\hat{y}\:\:\: $where$ \:\:\:(g = 9.8\: m/s^{2})\:\:\:\:\:\:\:\:$ (4)
$\vec{F_{g}}=\:\:\:m\vec{g}\:\:\: $where$ \:\:\:(\vec{g} = -9.8\: m/s^{2}\:\:\hat{y})\:\:\:$ (5)
$\vec{F_{g}}=-m\vec{g}\:\:\: $where$ \:\:\:(\vec{g} = 9.8\: m/s^{2}\:\:\hat{y})\:\:\:\:\:\:$ (6)
Note that equation 6, while generating the correct force vector, can be misleading as vector $\vec{g}$ is positive. Physically this equation is still self-consistent, as from Newton's 2nd Law, $\vec{a}$ = $\vec{F} / m $. In equation 6, the vector $\vec{g}$ is NOT the acceleration, but rather the negative of the acceleration:
$\vec{a}$ = $\vec{F_{g}} / m $
$\vec{a}$ = $-m\vec{g} / m $
$\vec{a}$ = $-\vec{g} $
I point this out because it is not uncommon to see teachers use equation 6. My preference is to avoid using equation 6 due to the possibility of confusion about the direction of acceleration.