Is resistance or voltage more effective in reduction of power loss? It is commonly stated that increasing voltage of the source reduces power loss. If voltage of source is increased current through the circuit increases, resulting in more power loss (than with less voltage) across the component and also through the wire.
$P=I^2R$
To reduce power loss across wire, we need less current through it. If we increase the resistance of component can't we effectively reduce power loss across wire ? (As drop in power loss across component is low than that across wire)
 A: 
If we increase the resistance of component can't we effectively reduce power loss across wire

This is the same thing as saying design the component to run at high voltage low current which is exactly what we do in high voltage power lines.
However you seem muddled throughout your question. You start confusing power delivered to the load as power wasted in the load.
You also talk about increasing component resistance after increasing voltage to decrease power loss in the wire due to increased current from the higher voltage. This thinking about it the wrong way. If I have a 100W load that runs at 50V@2A, and I want to reduce losses in the wire, I redesign the load to have a higher resistance so that it can produce the same power at lower current. That means voltage must be higher. I increased resistance because that is required to keep the load power the same running off a higher voltage source.
A: Taking the resistance of the wire as $r$ and the external resistance used in the circuit $R$ , then the total resistance of the circuit is $R+r$ .
Now if voltage supplied is $V$ then current in the wire is
$I = \frac {V}{R+r}$ .
Now total power loss in the circuit is
$ I ^2 R + I^2r = \frac{V^2}{(R+r)^2} (R + r)  = \frac{V^2}{R+r}$ .
So decreasing the potential difference by half the power loss will be one fourth . And increasing the external resistance will lead to decrease in current in the circuit  but the drop in power loss will be lesser than the earlier case.
A: It's not clear exactly what you are asking. But if you are asking if the voltage source or resistance has a greater effect on power dissipation, then the form of the equation for power that you should be looking at is
$$P=\frac{V^2}{R}$$
Let's assume the voltage source $V$ is a battery connected across the resistor $R$. Let's further assume the internal resistance of the battery is negligibly small compared to any value of $R$ we wish to consider.
From the equation we see that power dissipation varies inversely with the resistance across the battery while it varies as the square of the battery terminal voltage $V$. Therefore a decrease in voltage results in a greater reduction in power dissipation than an increase in resistance.
If this is not what you wanted to know, please let me know and I'll either delete my answer or revise it.
Hope it helps.
A: 
If voltage of source is increased, [then] current through the circuit increases...

Actually you've got that exactly backward.
The problem is, you misunderstand the context in which they "increase the voltage." They aren't talking about turning up the voltage in some existing circuit that supplies, for example, some existing light bulb. They are talking about tweaking the design of a power distribution system.
Suppose you want to deliver 100W to a light bulb. If you design a system in which light bulbs expect a 10V power supply, then you will have to supply 10A to light the bulb. But if you design the system to operate at 100V, then you will only have to supply 1A. "Turning up" the design voltage gives you a system in which less current will be needed.

It is commonly stated that increasing voltage of the source reduces power loss.

There, they are specifically talking about power lost in the distribution network. The wires that distribute the power have resistance. It's small, but it's enough to matter. Suppose the wire that connects your light bulb to the generating station has 1/10th of an Ohm of resistance. If the system supplies 1A of current to your light bulb, then 1/10th of a Watt ($1^2\times{}0.1$) will be "lost" in the wire. But if your system needs to supply 10A to the light bulb, then the loss will be $10^2\times{}0.1$ or 10W.
The distribution network that uses $10\times{}$ the voltage has only 1/100th of the $I^2R$ losses.
A: Increasing voltage of source does not decrease power loss.
I think you are talking about transmission of electrical power over high voltages. But that case is different from the usual (resistor only) case. There you can't use the V = I*R relation because transforming low voltage to high voltage is different from increasing the source voltage. The increase in voltage level in the step up transformer comes at the expense of decrease in current. You should use constant power relation that is P = I1*V1 = I2*V2 = constant to calculate the current with stepped up voltage.
Coming to second question, how will you increase the resistance of the component. If you add an external resistor with the component in series, it will definitely reduce the current through the circuit but then there will be additional power loss across the added resistor. Also the voltage drop across the component will reduce. But electrical components require certain level of voltage to operate.
So we can't reduce power loss this way.
