I just can picture it in my mind or on paper. Can someone explain it with examples?
This is the key idea behind the uniform circular motion: if the force has a component in direction of the object's motion, the magnitude. If the force is perpendicular, only direction change. But I just don't get it. I mean as long as you can make a triangle with three vectors, any two can be called the components of the remaining one.
But I just don't get it. I mean as long as you can make a triangle with three vectors, any two can be called the components of the remaining one.
Actually, you do get it: what you have said is perfectly correct, as long as you exclude degenerate triangles (ones that collapse to a line, which happens when the sum of two side lengths equals the third sidelength). But when you use an orthogonal basis (a special case of a basis), the component of a vector $U$ in the direction of basis vector $\hat{X}$, which $U$ is orthogonal, to vanishes. But this is only in the special case of orthogonal bases. In practice we use orthogonal bases all the time because calculations are messier if we don't, but orthogonal is actually a separate notion from the bare "vector" idea.
What you are groping for here is the mathematical notion of vector space or linear space. You're simply having trouble with the fact that notion of perpendicular is actually another concept beyond vectors. Have you learnt these in your maths classes yet? Anyhow, when you do, the whole idea of a linear space is very like an abstract version of your statement above: take a set of things ("edges of a triangle"), and then the vector space is the set of all the stuff that results from scaling and adding those things (if necessary, addition is simply an abstract group notion here). A vector space has a notion of "dimension", insofar that there is a unique maximal number of those "basis" things that has the following properties: (1) everything in the vector space can be gotten from linear operations on the basis but (2) none of the basis things can be gotten from linear operations on the other $N-1$ basis things.
Other than the number $N$ - the dimension of the vector space - there is almost infinite freedom in which $N$ vectors you use to be your basis. Any $N$ randomly chosen vectors will almost surely (in a measure theoretic sense) form a basis.
The physics enters your problem with the notion of orthogonality. The notion of orthogonality is further structure added to a vector space to make the notion of "angle" between vectors meaningful. And in Newtonian mechanics "orthogonal" has the physical meaning that if $X$ is the displacement vector begotten by force $F$, then the inner (dot) product $F\cdot X$ is the work done by the force. A force orthogonal to the displacement does no work, which is why the speed doesn't change in uniform circular motion, where the force is always orthogonal to the instantaneous velocity.
The vectors ${\bf F}$ and $\bf v$ cannot be added. That is to say: you can't draw a geometric line connecting the two because they cannot be added. They have different units and represent very different things. You actually have to use Newton's laws: ${\bf F}=m \dot{\bf v}$ to figure out how $\bf v$ changes.
There seem to be two different questions here: (1) How are the components of a vector actually defined, and (2) Why does the component of the force perpendicular to the velocity lead only to changes in the direction of motion and not the speed.
The first question has essentially been answered in one of the other answers, but let me make a few comments before moving on to question (2). Mathematics is only relevant insofar as it's useful for the physics, and here, it turns out that defining the components of a vector to be perpendicular to each other is a useful thing to do. Ignoring for the moment that $\vec{v}$ cannot be added to $\vec{F}$ (because they have different units) and assuming that the vectors that you drew can be added, we don't usually say that $\vec{v}$ and $\vec{h}$ (the hypotenuse) are the components of $\vec{F}$ because this isn't a useful thing to do.
Hence, the statement in one of the other answers that "A force orthogonal to the displacement does no work, which is why the speed doesn't change in uniform circular motion, where the force is always orthogonal to the instantaneous velocity." It is because of statements like this that we choose only to think of the components of a vector to be pointed along orthogonal (i.e. perpendicular) directions, even though (again ignoring the problem with units for the purpose of answering this part of the question) $\vec{v}+\vec{h} = \vec{F}$.
So I think you're thinking about things backwards. It is true that forces perpendicular to the velocity lead only to changes in direction, not speed, and because of this, we choose to define the components of the force as being perpendicular to each other, one ($\vec{F}_{\parallel}$) the component parallel to $\vec{v}$ and one ($\vec{F}_{\perp}$) perpendicular to $\vec{v}$.
$\vec{F}_{\parallel}$ leads to changes in speed and not direction. I think this is clear: If I push someone in the direction that they're already moving, I clearly can't change their direction, and therefore I can only act to change their speed.
$\vec{F}_{\perp}$ leads to changes in direction and not speed. This one is tricky to think about, and to get a true handle on it, it's necessary to understand the math (which I will include below for completeness). However, in order to get a feel for why this is, I always imagine someone walking slowly by me, and I push on their shoulder. If I just give them a quick, gentle nudge, they're going to change their direction of motion slightly, but not really their speed.
As far as the math goes, here's the most direct way I know of to see that $\vec{F}_{\perp}$ changes only the direction. It's not exactly straight-forward, depending on the level at which you're studying, but let's do it anyway. For simplicity, let's suppose that this is the only force acting on the object in question, in which case
$$ \frac{d\vec{v}}{dt} = \vec{a} = \frac{\vec{F}}{m} $$
so that we can work with the acceleration $\vec{a}$ instead of $\vec{F}$.
First, note that the rate at which the speed $v$ (not $\vec{v}$!) is changing is given by
$$ \frac{dv}{dt} = \frac{d}{dt}\sqrt{\vec{v}\cdot\vec{v}} $$
By the chain and product rules of differentiation this becomes
$$ \frac{dv}{dt} = \frac{1}{2\sqrt{\vec{v}\cdot\vec{v}}}\frac{d}{dt}\vec{v}\cdot\vec{v} = \frac{1}{2v}\left(\frac{d\vec{v}}{dt}\cdot\vec{v}+\vec{v}\cdot\frac{d\vec{v}}{dt}\right) = \frac{1}{2v}\left(\vec{a}\cdot\vec{v}+\vec{v}\cdot\vec{a}\right) = \frac{\vec{a}\cdot\vec{v}}{v} = \vec{a}\cdot\hat{v} $$
where $\hat{v}$ is the unit vector in the direction of $\vec{v}$; essentially, it is the direction of $\vec{v}$. From this, we can see that since we are dotting the acceleration into the velocity, we get the component of $\vec{a}$ along $\hat{v}$ (i.e. the parallel component) that leads to changes in speed. Once we write $\vec{F} = m\vec{a}$, this is exactly the idea that forces parallel to velocity lead to changes in speed only.
Next, we look at how the direction of $\vec{v}$ is changing, so we consider
$$ \frac{d\hat{v}}{dt} = \frac{d}{dt}\frac{\vec{v}}{v} = \frac{1}{v}\frac{d\vec{v}}{dt} - \vec{v}\frac{1}{v^2}\frac{dv}{dt} $$
where we again used the product rule (first) and then the chain rule. Now, we rearrange this equation carefully and substitute in for $dv/dt$ from our previous calculation:
$$ \frac{d\hat{v}}{dt} = \frac{1}{v}\left(\vec{a} - (\vec{a}\cdot\hat{v})\hat{v}\right) $$
After a little bit of thought, we can see that the quantity in parentheses is exactly the component of $\vec{a}$ perpendicular to the velocity, and hence the change in direction $d\hat{v}/dt$ depends only on this perpendicular component.
Just going to guess what you are getting at, sorry if I misunderstand you. Perpendicular vectors are orthogonal, you cannot express one in terms of the other and you can extend this idea into 3,4 or more dimensions.
They are called basis vectors. If you Google basis vectors that might help.
As an example, I can walk in the left or right directions in a room, but I cannot express this as an up or downwards vector.
Another name for it is linear independence, again googling this term will give you a range of suitable explanations, depending on your level of study so far,
