Small vessels generally lean into a turn, whereas big vessels lean out.
Why do ships lean to the outside, but boats lean to the inside of a turn?
For example, a boat leaning into a turn:
And a ship leaning out:
Small vessels generally lean into a turn, whereas big vessels lean out.
Why do ships lean to the outside, but boats lean to the inside of a turn?
For example, a boat leaning into a turn:
And a ship leaning out:
Your question became clear after you posted the images.
This corresponds to the difference in a turn as accomplished by a bike/motorcycle and a car/bus/truck. So let's study this case first.
During the turn within the accelerated reference frame there's an imaginary "centrifugal" force, which is directed outward the turn of course. OTOH the force applied by the ground has a component toward the turn (due to friction to prevent/reduce sliding). This creates a momentum that tends to lean the object outward the turn.
This is indeed what happens with most 4-wheel vehicles. They lean outward the turn, this transfers the pressure from the inner wheels to the outers, which causes appropriate change in the normal force applied to the wheels. This in turn creates a momentum which tends to lean the object toward the turn. By such the vehicle is leaned to some angle, after which equilibrium is achieved.
Now let's see what happens with 2-wheel vehicles (like bike). Since the normal force is applied in just 2 points, leaning outward the turn does not transfer the pressure, there's no change in the normal force, hence no momentum toward the turn is created. Moreover, leaning outward the turn displaces the mass from the 2-wheel axis, hence the gravitational force creates even more momentum outward the turn. The bike would just fall.
To accomplish the turn however, the biker leans the bike toward the turn deliberately. They do this by counter-steering, which involves microscopic inputs on the steering pushing the handlebars in the opposite direction to which they want to turn, thus leaning the bike into the turn. Displacing the mass causes gravitational force to create a momentum toward the turn. Which is in equilibrium with the momentum of the "centrifugal" force.
Now let's see what happens with vessels.
As with vehicles, the "centrifugal" force is applied outward the turn, the force applied by the water has a component toward the turn (due to the viscosity). Hence the "centrifugal" force's momentum is outward the turn. The difference is that there's a considerable part of the vessel under the water. Moreover, the center of mass is not required to sit above the water level. Another difference is that there are no discrete contact points with the water, instead water pressure is applied on all the underwater part of the vessel.
When the vessel leans (to either side) its configuration changes: its center of mass is displaced, its underwater part is changed, the volume and shape of the water "pushed out" changes as well. If the center of mass of the vessel + "pushed out" water raises - there's a momentum that tends to return the vessel back to its original state, hence it's stable.
Theoretically all the vessels are stable when at rest (otherwise they'd turn around). However during the motion some vessels are raised (like the small boat in your question) and become unstable. Such vessels definitely may not perform the turn unless deliberately leaned toward the turn. Simply because there's nothing to compensate for the "centrifugal" momentum. OTOH big vessels may remain stable even during the motion, with enough reserve to perform the turn as-is.
So, the factors to consider are:
Based on those one may see to which side the vessel leans during the turn.
There's however another interesting moment. If the vessel is unstable it should lean toward the turn. But how does this actually happen? Bicycle rider leans intentionally, otherwise he'd fall. He does it by counter-steering or displacing his own mass toward the turn, which is considerable WRT the mass of the bicycle.
But is this the case with the motor boats? I doubt if the mass of the rider is considerable WRT the mass of the boat. Plus, if this was the case, unskilled riders would turn around frequently, and I personally never saw this. There may be two explanations of this:
The above two answers both have flaws. valdo doesn't answer the key part of the original question, but asks why does the small boat lean in? The second response says that small boats turn in because the drive force is being applied at the rear of the vessel, which isn't quite relevant.
Turning is accomplished by applying a sideways force at the stern of the boat. Seen looking down from above, this is a torque, and attempts to rotate the boat about its COM. As noted in the other responses, the water must push against the side of the hull towards the inside of the turn to turn a moving boat in a circle. (If the boat was not moving, and it had, say, trolling motors at the bow and stern pointing in opposite directions, it could rotate in place, and the water wouldn't have to provide any force.)
We can see the rolling effect come into play if we look horizontally at the boat from behind. In a large ship, where the sideways force is provided by a rudder that extends pretty much from the water line to the keel, the center of the force is operating at a point below the water, about half way between the water line and the keel. The direction of this force is towards the outside of the turn (we are pushing the stern to the outside to turn the bow to the inside). If the center of mass is above this force, a roll-in torque is generated. However, the force provided by the water on the hull to turn the moving boat acts to the inside. This creates a roll-out torque (again, if below the COM). At speed, this is greater than the rudder torque and so the ship rolls to the outside.
In a small boat, the turn-rotation force is provided by changing the thrust direction, in the case of an outboard engine or water jet, or deflecting a low rudder in the prop stream below the keel in the case of an inboard engine. This turning force is very low, not centered between the keel and the waterline, but below the keel. This longer distance below the boat center of mass creates a high roll-in torque that is greater than the roll-out torque created by the water on the hull. This is even more true as the boat rolls, when the thrust vector of the engine actually has an upward component.
By the way, re stability, the center of buoyancy does not generally stay in a fixed location as a boat rolls. If the center of buoyancy is directly below the center of mass, then you would expect that when a boat rolls about its COM, this would position the COB toward high side of the boat. However, when a boat rolls, the shape of the hull that is under water changes, and the center of buoyancy often moves toward the lowered side. This provides a righting torque.
Nice question.
In both boats and ships, the center of mass of the vessel tends to be above the water line. This is a result of design; one wishes to have as little of the vessel as possible below the water line as it is the part of the vessel that is below the water line that causes the most friction in movement. At the same time, the humanly usable portions of a boat tend to have a density less than that of water. (An exception is a submarine; note that submarines tend to lean towards the inside and are subject to a problem called "snap roll" where they roll too far. For more, read this MIT thesis.)
To turn a vessel, the water must apply a force to the vessel. Since that force is applied by the water, it is typically applied below the center of mass. Thus one expects that a typical vessel will lean out during a turn. And in fact, this is what a large ship does.
A small boat, when turned without power, will do the same thing as a large ship. To counter the effect, one applies power and this makes a small boat lean in. This is due to the fact that the drive force is being applied at the rear of the vessel. This pushes the rear out on a wider turn and the small boat leans in. Try turning a small boat with no throttle.
Sail boats can either lean out or in depending on which way the wind is blowing. I.e. they lean out on the first part of a tack and lean in on the first part of a jibe, and then reverse. But the natural tendency is to lean out and so the force on a sail boat will be making it lean out as it goes through the wind during a tack. (I.e. at the point in the turn where it would be in irons if it stopped.) Since sail boats do not have propulsion at the rear of the boat (as in a small powered boat), these tendencies apply to both sail boats and sailing ships.
Newtonian physics and some fluid dynamics but the oversimplified punchline: boats that get up on a plane (planing boats, 'power boats', etc) "lean in". Displacement boats (ships, etc) "lean out". You can 'lean a planing boat out' by rapidly decelerating and turning hard. Once the boat starts displacing a volume of water equal to or greater than its weight, (instead of 'skipping' over the water) it will start leaning out.
*tl;dr ...reapplying power in said deceleration/turn of planing boat counteracts the role by countering the angular momentum since the force of propeller+angle of the transom body (essentially rudder angle) applies acceration or force opposite the direction of the roll ('lean'). There are additional contributing forces but these are the main ones.
Humorous examples: novice jetskiiers, rapid deceleration in turn caused by boat wakes = roll in opposite direction of yaw (turn) when watercraft rapidly stops planing.
I think the answer is pretty simple. First you have to understand the shape of a ship vs a boat. A boat, like a jet boat a boat with an outboard, has a much flatter bottom than a big ship and a small keel, the part that juts underwater, if any. A large ship has alot more area relative to its size under water that a small boat. Also, a boat turns by rotating its outboard or water jet and these jut out under the boat much farther relative to the rudder of a big ship.
To see how this creates the difference in leaning between a ship and a boat, imagine two different ships with these different characteristics: ship A and ship B. Ship A has its rudder very far below its keel. In addition its keel is small, the underside is nearly flat, and its center of mass is very near the water. Ship B on the other hand has a very long keel that sticks under very far and a rudder near the surface. Its center of mass farther above the water.
When a ship goes to turn, the water hitting the rutter causes the ship to try to rotate about its center of mass and lean in. While this is happening, the ship is moving through the water sideways because it cant turn perfectly. Its like a car with poor traction trying to turn, its slides along the ground while trying to turn. This sideways motion trys and rotates the ship the other way. In the case of ship A, there is very little force from the water on the bottom of the boat when sliding because it has no keel. Also the rudder is low in the water and works to rotate the ship to lean in. On the other hand, ship B has a large keel and experiences a large amount of force from the water as it slides. Its rudder is also near the surface and though it may apply the same force, it causes very little torque so the boat doesnt try as hard to rotate in. The effect is that it rotates out.
I think thats it. In summary, big ships turn because they have a keel and this causes them to lean out. Small boats turn by leaning in and facing their underside to the water. Google jetboats and you can see what i mean about them turning.
The answer to the question is; leaning out or leaning in is a result of the vessel's rudder's position relative to the keel. If the rudder is ABOVE the keel, the vessel will lean OUT; if the rudder is below the keel, the vessel will lean IN. ("Rudder"- the method or mechanism by which the flow of water under the vessel is diverted to cause a change direction of the vessel. This method or mechanism might be a movable plate, or a movable jet of water generated by a propellor.)
The answer is much simpler than explained above. Small boats turn with a change in the direction of the propeller. Large boats turn with a rudder. To turn left, the turn of the propeller applies a force in the back of the boat that makes it lean to the left. This does not happens with a rudder.
TYPICAL boats have outboard motors. Look at the back end of the boat when it turns right. The outboard motors swing right, the propellers push water to the right, and the boat turns right. The propeller is under the boat pushing all of the propeller force towards the right. This tries to make the bottom of the boat move towards the left - so looking at the back of the boat this forces it to rotate clockwise - “into” the turn. TYPICAL ship propellers always push straight out the back. The rudders then push a portion - only a portion - of the propeller force to one side. This force does try to rotate the ship into the turn but it’s much less than full propeller force, and the centrifugal acceleration of the superstructure of the ship is larger and causes it to roll “out of” the turn.
If we ignore centrifugal force (red in the diagram), both ships and boats would tend to lean into the turn due to the turning thrust of the rudder or outboard motor below the water line, causing an anticlockwise torque when turning left and when looking from the rear. (green in the diagram). Centrifugal force is proportional to $m\omega^2 r$. Since the ship has a greater mass, it has a greater centrifugal force for the same turning angular velocity ($\omega$) and the same turning radius ($r$). Together with the longer lever arm due to the centre of mass of the ship being much higher up from the water line than the rudder is below the water line, the clockwise torque due to the centrifugal force is greater than the anticlockwise torque of the rotating rudder thrust, so the ship leans outward. Not shown in the diagrams is the compensating force due to the shape and buoyancy of the hulls that prevents the vessels from completely rotating and capsizing. An important concept here is that the various torques act about a pivot point that is approximately at the water line, and the rotation is not about the com as is normally the case in free space. The greater mass of the ship and greater height of the com of the ship above the water line both contribute to the outward, rotating torque of a ship being greater than the inward rotating torque of the rudder thrust.
Imagine a leisurely deep-sea fishing boat with a lookout platform above the main cabin. If there were enough passengers onboard and they all somehow managed to get on the top lookout platform at the same time, it's possible the fishing boat would lean outwards during a fast turn.
P.S. The semi-circular shape of the hull below the water line that I have drawn for simplicity is possibly the worst design for a hull cross-section from the point of view of stability and preventing a capsize.