Is there any way to predict the amount of mass a star will lose when turning into a black hole? I know that the star loses it's outer layers, but is there any way to predict it's final mass?
TL;DR: It's extremely difficult, and there are huge uncertainties, but there are models and numerical simulations which give relations between initial stellar masses and eventual remnant black-hole masses. For a range of models and masses, if stars are very low metallicity than as much as half of the stars initial mass may end up in a Black Hole. In high metallicity stars (like our sun), as little as 10% or so may survive in the remnant. Some (unknown) fraction of stars in certain mass ranges may not leave a black hole remnant at all.
In the simplest model of star formation, stars form, they evolve, and they leave some sort of remnant. A (white)dwarf for low mass stars (something like $M \lesssim 8 M_\odot$), a Neutron Star (NS) for intermediate masses ($8 M_\odot \lesssim M \lesssim 20 M_\odot$) and finally Black Holes for the most massive stars ($20 M_\odot \lesssim M$). The simplest way to predict the remnant mass is to consider the mass of the iron core which forms in the star soon before it explodes (see, e.g., Heger+2002). The connection between a star's initial mass (often called the "Zero-Age Main-Sequence [ZAMS] mass"), and the eventual mass of its iron core can be very complicated, and
Especially for massive stars, the processes connecting stellar evolution on the Main-Sequence to the eventual remnant can be complicated and exotic, involving stellar winds, explosive outbursts, mass-transfer to binaries, and finally an eventual explosion (supernova, gamma-ray burst, etc) which can significantly alter the end-state. The results seem to depend on poorly-understood physics concerning rapid rotation, strong magnetic fields, variations in metallicity, and the presence of close binary companions.
Before supernovae, massive stars undergo phases of high mass-loss through 'winds' (massive outflows from the stellar surfaces). These winds are especially strong in high metallicity stars formed in the recent, low redshift, universe - and are believed to be the primary method by which massive stars lose mass. Winds can be seen in 'Wolf-Rayet' Stars (Like WR 124 pictured above), which continuously blow out huge amounts of material into the surrounding environment.
Explosive/Eruptive Mass Loss
Stars called 'luminous blue variables' show puncuated, transient episodes of mass-loss which can eject the entire massive envelopes of stars. The best example is Eta Carinae, a star of initially 100--300 $M_\odot$ which (relatively) recently ejected 10s of solar masses of material in a rapid outburst.
Stellar Death and Explosions
Most stars end their lives in supernovae (why? see my related answer here: explosive outbursts powered by the bright, radioative decay of heavy elements produced in the stars core. In general, the outburst is driven by a shockwave produced from gravitational collapse. Is material outside of the core is effectively ejected, then the core will likely settle into a neutron star. In very massive stars, however, much of the outer material is too tightly bound to the core and will 'fall-back', triggering accretion onto the proto-neutron-star/core --- leading to its collapse into a black hole. How much material is blown out, and how much falls back, depends sensitively on the hydrodynamics of the explosion and even more subtle details like how rapidly its spinning.
Some explosions, however, may not leave a remnant at all. In a 'pair-instability supernova', it is believed that no remnant is left at all. These explosions may occur for low metallicity stars with ZAMS masses above $\approx 150 M_\odot$.
The only way to take all of these factors (and many more, e.g., binarity, spin, etc) is in detailed numerical simulations. Below is a good summary figure showing the relationship between black-hole remnant mass and initial mass from Belczynski+2010. Each panel shows variations in different parameters, focusing on different mass-loss rates and different metallicities. I think the bottom panel is representative of what most people in the field have in mind... but there are very few observational constraints --- although the recent LIGO detections show that stronger constraints could be coming in rapidly.
What these results show is that metallicity plays a huge effect. For low-metallicity stars (like those formed in the early universe), as much as half of a stars initial mass could end up in the remnant BH. For higher metallicity stars (like more massive versions of our sun), the higher rates of mass-loss from winds could mean than $\lesssim 10\%$ of the mass survives in the BH. These models do not include pair-instability supernovae, which could mean that the remnant mass above some threshold would suddenly turn to zero.