What determines whether colors you can't see are visible or not? So, when someone is red-green colorblind, the colors appear the same to them, like this:

Source: https://iristech.co/what-do-colorblind-people-see/
And if you're totally colorblind, then things presumably just appear like they would in a black-and-white movie.
However, this isn't how ultraviolet patterns seem to work. Compare how we see this flower to the version where ultraviolet is visible:

Source: Dr Klaus Schmitt
The UV pattern is completely invisible here. However, unlike with the red and green, this isn't because yellow and UV are colors that appear identical when you can't see UV. Look at these flowers:

Source: https://blog.zoo.org/2012/01/ultra-awesome-ultraviolet-eyesight-in.html
This time the flowers are purple, but the UV pattern is still invisible. Why is that? Shouldn't the UV pattern still be apparent on at least one of the flowers, just in a different color? And on some other flowers, the UV does appear as a different color. So:

*

*Why is the UV invisible only sometimes?

*Does it have to do with the flower using iridescent structures to produce color, instead of a pigment?

*Can this happen with red and green, as well?

Image sources:
https://iristech.co/what-do-colorblind-people-see/
 A: Color is a double valued variable.For physics there is a one to one correspondence between frequency of light and the color assigned to visible frequencies. As far as the spectrum of colors (rainbow) ultraviolet frequencies are invisible to our eye.
The eye is a biological entity, the retina of the eye has color receptors, and these receptors do see the spectrum . BUT there is also a color perception, that the same color can be accepted by the brain although it has many different frequencies.

Color blindness is  due to this biological mechanism being misaligned .

.    Why is the UV invisible only sometimes?
.    Does it have to do with the flower using iridescent structures to produce color, instead of a pigment?

Now ultraviolet frequency reflecting from materials as in the photos you show, may interact with them and give the perception of "seeing"  ultraviolet, and that will depend on the material, which explains the differences in seeing an ultraviolet effect or not in the visible.

Can this happen with red and green, as well?

It might, i.e. the frequency scattering off a material may be degraded in energy and change the frequency( color) a bit. One would have to shine a fixed frequency red or green to see if there is an effect on the particular material.
A: There are two different mechanisms at work here. It's not the case that humans are "ultraviolet colorblind" or something like that.
1) There is the spectrum that the flower petal reflects or absorbs. This spectrum is continuous and includes ultraviolet and everything at lower wavelengths, visible light, and infrared and everything at higher wavelengths.
2) There is how the different wavelengths within that spectrum are perceived by our eyes and brain. Here's how we perceive color: our eyes have cone-cell receptors that have peak sensitivity at three different wavelengths (roughly, red, green, and blue). We can't see light that our receptors are not sensitive to. In addition, the cornea and lens of our eyes block ultraviolet light so most of it never even reaches the receptors. This is why "visible" wavelengths are visible: we are physically unable to perceive wavelengths outside of that range.
In the example you gave of a flower with yellow petals, where the tips are bright in the ultraviolet, let's look at mechanism (1) first. The tips of the petals reflect yellow and ultraviolet. The center parts of the petals reflect only yellow. Then looking at mechanism (2), the two parts don't look any different to us, because we can't perceive the reflected ultraviolet that differs between them. Bees' eyes cover a different range of wavelengths, so the ultraviolet light reflected from the tips is outside the human-visible range, but within the bee-visible range.
In the example picture of beans and carrots demonstrating red-green colorblindness, looking at mechanism (1): the carrots reflect orange light and the beans reflect green light. Considering mechanism (2), we know that most people with typical vision can see the difference. In the case of red-green colorblindness as in the photo, the two different wavelengths of red and green light are perceived the same by the brain. (There are a number of causes of colorblindness, but usually it's some genetic mutation that causes some malfunction in the cone cells.) This is different than being unable to perceive ultraviolet.
A: An unaided and healthy (see below) human eye cannot see anything ultraviolet. That's why it is called ultra-violet - in the whole picture of the electromagnetic spectrum it is between the violet visible light and X-rays.
What we CAN see related to UV is the tails of mainly-UV spectral features (be they light or absorbtion). That's why we can see the "black light" lamps - they are mostly UV, but some of their light protrudes the visible area of the spectrum.
A great variety of pigments (both natural and artifical) we see as yellow or orange because some strong and wide UV absorbtion band absorbs also in the violet-blue end of the visible spectrum. Most yellow flowers are yellow exactly because of such spectral feature.
We are not color-blind, we are completely blind to the UV. We can use uv-sensitive sensors and cameras if we need to.
Then again, flowers developed their colors and patterns not for us, but in co-evolution with the insects that not only can see near-UV, but have their very own colors in the UV band. What we see is just the part of the picture, painted for the bees.
(The bees cannot see the red end of the human vision so we are not completely at loss.)

As for the "healthy" point - the violet/UV limit of the human vision is imposed by the eye lens. People using early generations artificial lenses can see way into the UV. No much of a colors there, though.
A: I think part of the problem is the way you're defining color.  As other answers mention, humans generally have three color receptors, which are sensitive to frequency ranges that we call red, green, and blue.  Color is what we percieve when those receptors are excited by light, and we percieve a range of intermediate colors when more than one type of receptor is excited.
When you consider ultraviolet (or infrared), it's not a color, both because (most) humans don't have receptors for it, and perhaps more importantly, because it's a range of frequencies that is actually  broader than visible light.  So a creature with receptors for ranges within that spectrum would percieve multiple colors of UV.
When we build detectors for UV - film cameras, CCDs, and whatnot - we do something similar.  The devices translate UV frequency ranges into colors that we can see.  So the "colors" shown in your images are basically artistic choices made by the photographer.  Whether the UV reflectance of the tips of those flowers is shown as yellow, bright green, or blue depends on the hardware used and choices made by the photographer.
This is all just a subset of false color imaging, as used in space probes, for instance: https://en.wikipedia.org/wiki/False_color
Another possible factor here is UV fluorescence. When illuminated by UV, some objects will emit visible light - the familiar "black light" effect.  I suspect that might be what's happening in the picture of the flower with the bright green tips.  See e.g https://adaptalux.com/fluorescent-flowers-ultraviolet-light/ for more examples of flowers and such fluorescing under UV.
A: • Why is the UV invisible only sometimes?
Ultraviolet Light is always invisible to the human eye, because it lies beyond our visible spectra range . Only UV detectors and specially designed cameras can " See" the UV light. 
• Does it have to do with the flower using iridescent structures to produce color, instead of a pigment?
That is very unlikely , pertaining to your question. 
• Can this happen with red and green, as well?
The fact that red and green and yellow flowers are totally black under UV, is because they absorb the complementary color (blue and shorter wavelength spectra including UV), hence they appear black. 
Whereas blue flowers, reflect the blue ( and shorter wavelength spectra including UV) in totally hence appear to be blue. 
In short, no that can't happen
A: Reception and perception
Our eyes have receptors (cones and rods), and these are naturally evolved for Sunlight, which is a combination of many wavelengths (containing non visible wavelengths too), and our receptors have evolved so that they are mainly sensitive for visible wavelengths, a tricolor system, red, green and blue wavelength light. Now the receptors are sensing these different wavelength photons and react on them by sending a combined signal to the brain. Our brain is where the perception happens, but the brain can only work with information it actually receives. If the receptors (some of them) in our eyes are not sensitive enough for certain wavelength photons, then you see what happens on the top pictures. It is very important to understand that the brain would still be able to perceive all the colors, but if it does not receive the information from the receptors in the eyes, the color vision will be different.
Our receptors have naturally adopted to Sunlight (which contains for example UV too), and our receptors have evolved to be sensitive for visible wavelengths (but they are not sensitive for non-visible, like UV). 
UV light
Now the pictures about the flowers show a different phenomenon. Certain materials, and these flowers, have a very special ability, they are able to absorb certain wavelength photons and re-emit different wavelength photons.
Now in your case the flower has evolved so, that the are on the tips are able to absorb UV photons, and re-emit visible wavelength photons. The reason we see it is not because we would see UV photons, we do not. Our receptors are only sensitive in the visible wavelength. The reason we see these areas, is because when UV photons are shone on them, they are able to absorb these UV photons, and re-emit visible wavelength photons.
There are many types of this phenomenon, fluorescence is only one of them. The difference between the absorbed and emitted photons can be energy (fluorescence), or temporal (meaning a delay between absorption and re-emission, like phosphorescence).
https://en.wikipedia.org/wiki/Photoluminescence
A very interesting question would be why these flowers have evolved this way, why, for what reason do they want certain areas on them to be able to absorb UV and re-emit visible wavelength.
A: There are two important properties that any waveform intrinsically has: amplitude and frequency. If you're looking at a waveform, the amplitude is how far from the mid line the height of the peak/low of the troughs are, and the frequency is how many peak-to-peak there are in a certain distance; frequency and wavelength are thus linked - the wavelength being the distance from one peak to the next peak, shorter wavelengths mean you can jam a higher number of them into the same distance. Shorter wavelength, higher frequency. I suppose you could also call amplitude waveheight, or wavewidth depending on the orientation of your head when you're looking at it
So there's a huge spectrum of electromagnetic radiation emanating from the sun and we see (with our eyes) just a tiny part of the spectrum. We see because our eyes have cells that can detect the amplitude and frequencies of some small sub set of the huge range of electromagnetic radiation that exists
In your eyes, you have some cells that are rod shaped, and some cells that are cone shaped, and hence colloquially get called "rods" and "cones". Rods don't need a lot of stimulation to register the presence of radiation so they do low amplitude stuff, but they don't discriminate frequency. Cones are responsible for detecting frequency but they need more of a kick to get them to respond. A single cone doesn't detect different frequencies; there are three different types of cone and each type is more sensitive to a range of frequencies than the other types. 
Strictly (but simplistically) speaking, one cone type does radiation in the yellow frequencies, one does green the green frequencies and one does blue frequencies. In turn, and our brain maps the relative amounts of stimulation that each of them detect, to intensity and color. 
If you look at an intense source of electromagnetic radiation in the blue part of the frequency spectrum (hereafter referred to as blue light), your blue cones are going "ooh, that's a lot of light" but the green and yellow-green cones don't have a lot to say at all - your brain turns that into "that's a blue light". 
I picked on blue because it's easiest to explain - blue rods, blue light, blue stimulation, blue perception. Now we kinda need to talk about light composition
About light composition: 
Red, Green and Blue are just terms of reference we've made up to help us describe and communicate with each other. You could hold up a ball that most people would say is red and teach a child that it's called green and they will really struggle to use traffic lights if anyone ever says "green means go". I mention this because what I see and what you see when we look at a red apple is quite possibly a very different thing, but if all we have ever known is "that's red" then we will both call it red even though what we are both seeing might be completely different
If all we have is red light, green light and blue light, we can make the others by mixing. Red and Green is Yellow, Green and Blue is Cyan, Blue and Red is Magenta. In reality there is a spectrum of these things, and how they are detected by the eye isn't so cut and dried either. 
If blue light had a frequency of 5, and green was 3, then cyan light would be 4. Let's say the green cones are mostsensitive at green 3, but they can detect 2 and 4 (a little bit either side). For blue cones that detect 5 really well, they can do a bit of 4 and a bit of 6 too.
If blue 3 and green 5 light were shone into our eye, then the blue cones and the green cones detect it. Because blue and green cones are activating, we see it as cyan. 
If cyan (4) light were shone in, blue and green cones would again activate because they have a range of frequencies they're sensitive to; the cyan light activates both of them in that overlap zone where they both weakly respond to cyan - we see it as cyan again even though the light entering is different to before. In practice there are many different combinations of different frequencies of beams of light that are detected in this varying map of intensity, and they get mapped to similar colors
I didn't talk about yellow cones much because they add a dimension of confusion. A yellow cone responds to what we might refer to as red, yellow or green light. It needs help from the other cones to determine what color it is seeing. If a yellow is activated, but green is not then the light source appears to be red. If yellow activates strongly and green activates weakly, it maps to yellow light, and if yellow activates weakly, green activates strongly, it maps as green
You talk about color blindness, and most commonly it refers to a deficiency of the green cone - if a person's green coneset is deficient they will have trouble telling red from green simply because the green cone set registers more closely to how the yellow cone set registers, and they have a reduced ability to detect greens as a result. Because detecting green light is vital for interpreting the information from the yellow cone set, but the deficient green cones behave more like yellow cones you end up with a situation of:


*

*"weak red light causes weak activation of yellow cone and incorrect activation of green when it shouldn't" and 

*"weak green light causes weak activation of yellow cone and deficiently low activation of green cone when it should be more" 


Bright reds and greens might not be as much of a problem; they wouldn't look very distinct but other cues might help a red-green color blind person discriminate. When intensity lowers, things become a problem because the amount of cone activation is so similar in different color situations. 
Your carrots picture isn't exactly accurate because it wouldn't look "that intensely green" - all the veg would look a more bland form of yellowy brown rather than being this green; it's the lack of help from the green components that rives everything toward being perceived as yellow/brown
People who are completely color blind are incredibly rare, and it's probably not quite like watching a black and white movie. Black and white movies are varying shades of white because white light makes all your cone sets activate. People who are monochromatic are more likely to see one color in varyiong intensity, so rather than being black white and grey, your movie would be more like a black and white tv with a faintly colored sheet of transparent plastic placed in front of it
In low light, the rods take over; they just register an intensity of light. At dusk and darker, everything starts to look the same because we can only see the presence or absence of light rather than its color

So that's "how we see". Next up to discuss is "how we see things"
We see because light enters our eye and tickles our rods and cones. We see things because visible light is coming off them and travelling into our eye, and tickling those rods and cones. Some things emit their own visible light; other things "emit" light because they're reflecting it from something else.
A vital consequence of the last sentence is that in order to reflect a frequency of light, the object has to receive it in the first place and the object has to be made of something that reflects it rather than absorbs it. 
In terms of reflection, absorption and transmission in the real world:
Your eye pupils look black because they do a good job of capturing most of the light that enters them. Your goth friend's skin looks white because it's reflecting a lot of the visible light falling on it. Your other friend from Papua New Guinea has one of the darkest known skin tones because his skin absorbs a large amount of the incident light. Both your friends can go and get an X-ray and it works out because the X-ray waves are transmitted by/pass through your skin but not your bones (absorbed). All 3 friends can get a sun burnt by the UV present in sunlight, but not if they're in the shade, unless someone puts up a mirror reflecting the UV onto them again. Hiding behind a mirror might not do much good if you're close to a source of gamma radiation
The whole world is potentially being illuminated by a huge range of frequencies of electromagnatic radiation. If your eyes were sensitive to ~2.4GHz you'd see wifi routers and microwave ovens flashing like crazy. If you could see 500 - 600 Mhz, TV satellites in the sky would be flickering away. If you could do 900 or 1800 MHz GSM cell towers might look like the white walls of your living room when the TV is on at 2am..
As it is, we see what we see, and we don't see things outside that range. If we use some device to shift the frequency into a form that we can detect (point your cellphone camera at an infra red remote control, use a Geiger counter to turn radiation into an audible "flickering" then we can "see" more of the world around us. Without those things, we rely on our senses and what they do or don't pick up. If yellow light falls on a blue object, we see it as black; the object only looks blue because it is capable of reflecting blue light and it absorbs all others. Yellow light doesn't have a blue component, only pure yellow or some mix of red and green, so the object appears black; it can only reflect blue, and none of the light falling on it is blue.
Inks in your inkjet printer are cyan, magenta and yellow, because the paper can't emit light on its own - it has to rely on reflection. A paper painted red can only reflect red light. A paper painted green can only reflect green light. If inkjet in was red, green and blue we couldn't print yellow, because to get yellow, we need to reflect red and green. Simultanoeusly painting the paper with red and green ink would mean the red ink absorbed all the green light, the green in absorbed all the red light, and our yellow (or red+green_ light doesn't work. The inkjet printer will spray the paper with yellow ink and magenta ink, the common reflective component of these two being red, if you want red. Green is a spraying of cyan+yellow, as both these colors (in a reflective sense) can reflect a green component, and each of them filters out one of the other components (yellow filters out blue, cyan filters out red) leaving only the green (fro red,green,blue)
We don't see UV light; if something is visible under UV light on CSI, it's because the UV light is causing the object to emit a frequency of light that we can see. We don't see infrared either, but we can perceive some infrared radiation as heat because it feels warm on our skin. We could probably detect microwave radiation too; it will vibrate the molecules of water in our skin just like it heats the food in the oven. Don't stick your hand in a microwave oven, but do appreciate that after microwave ovens were invented, it was mooted possible to replace conventional heating systems with microwave systems that gently heated the humans in the house, using microwave radiation

If you're still awake, hopefully you know now how we see, and how we see things - so your questions are simpler to answer:

What determines whether colors you can't see are visible or not?

Whether it is present; something has to be emitting it, something has to be transmitting or reflecting it - these two things alone determine its presence in your location - it has to be being generated there or arriving there. 
Whether you can detect it is another question. You can't detect the radiation from your cellphone/network so you have to rely on your signal bars to tell you whether you have service or not. If you're near a tower (emitter) and nothing is blocking it(the free air is transmitting it) then its present (it's shining on you until you walk into that concrete bunker...)

So, when someone is red-green colorblind, the colors appear the same to them, like this

Yeess.. More like they have a diminished ability to tell certain different colors apart based on the normal metric you might apply. There are many variations of color-blindness

And if you're totally colorblind, then things presumably just appear like they would in a black-and-white movie

It's more likely to be that everything appears in a varying intensity of shade of a color (other than white)

However, this isn't how ultraviolet patterns seem to work. Compare how we see this flower to the version where ultraviolet is visible

There's no magic with UV; it's simply light/electromagnetic waveform just like anything else. The tips of that flower reflect UV, the main body of the flower doesn't reflect UV. UV light might not be falling on the flower, or it might. Your eyes can't detect it either way; that image has been made using technology (camera lens) that can detect UV and it was used in some context where there was UV light falling on the flower, being reflected and detected by the camera. It's been re-represented as visible color so you can appreciate it. Just like a Geiger counter makes a horrific noise to help you appreciate how well irradiated you're being

This time the flowers are purple, but the UV pattern is still invisible

They're different flowers, that might or might not have some part of their surface that reflects UV into your detector.. but you don't seem to have a detector with you so you're right - it's invisible

Shouldn't the UV pattern still be apparent on at least one of the flowers, just in a different color?

No. Not in "a different color" anyway. Whatever color UV is, it's not a color we see so we won't have evolved to have a name for it. Other than perhaps "UV". We can't see it, so we need a detector that can, and it might say "this part of the flower is reflecting 100% of the incident UV light, that part is reflecting only 80%.." etc, so a computer could generate an image using visible colors to describe the intensity of UV reflection different parts of the flower... This is in exactly the same way that a standard camera lens samples the intensity (and frequency) of light apparently-emanating from everything it can "see"

And on some other flowers, the UV does appear as a different color

If you're seeing it, it's not UV. Maybe it's violet or blue or some other electromagnetic radiation from a part of the frequency spectrum that is close to UV's frequency.. Like maybe the UV torch in your hand is also chucking out some visible light too. A true 100% UV light would emit no visible light. Just like you can't see the infrared LED in your TV remote flashing away either

Why is the UV invisible only sometimes?

It's invisible to you or I all the time 

Does it have to do with the flower using iridescent structures to produce color, instead of a pigment?

It's perhaps not the right word; iridescence refers to a surface's ability to reflect incident light in such a way that it appears to have multi or varying colors depending on the angle. It will likely be like a prism and is causing a split or divergence of the different inbound light frequencies so that they appear at different angles and are no longer perceived as combined. A rainbow, prism or diamong ring might be similarly effective. 
Fluorescence might be the word you're looking for; a substance that receives a higher energy radiation like UV or X-ray and begins glowing with a lower energy radiation emission like visible blue light. Kinda like microwaving something until it's so incredibly hot it's emitting infrared

Can this happen with red and green, as well?

Can UV appear as red or green? No; by definition it can't appear ans anything other than UV. If you could run it through some downsampling/frequency changing device so it went in as UV and emerged as red, then it wouldn't be UV any more. Remember that these are all just different speeds of oscillation of the same elementary particle, so there isn't any magical difference between them
A: I assume that not being able to see specific colors means that your cones (see below) are insensitive (by malfunctioning) to photons that have wavelengths corresponding to these colors, while they are visible if your cones functioned properly. Your brain doesn't receive signals from the red and green cones, so you can't see a range of visible red colors and a range of visible green colors. Let's further assume that the visual cortex in your brain is functioning as it should and a normal functioning lens.
So let's go.
Without going into further detail (see this article for more information), the three different color cones on the surface of your retina, have a range of photon frequencies to which they respond (the image is taken from this article):

The S(mall)-, M(edium)-, and L(ong)-wavelength curve correspond to the sensitivity of the "blue", "green", and "red" cones as a function of the wavelength of the photons that fall on the cones. It's noticeable that the M and L curve overlap for the greatest part. The S curve overlaps with the other two, but less. You can see that if your red and green cones don't function, you can't see the normally visible photons with energies lying beneath these two curves
So if your green and red cones don't work at all, a range of pure red colors will be impossible for you to see, as well as the biggest part of the range of pure green colors, you can't see these colors which are normally visible. But because in the range of photon wavelengths beneath the S-curve (on the right side of it) a small range of long-wavelength blue photons and small-wavelength green is contained (where the S- and M-curve overlap these photons can stimulate your intact blue cone) your perceived image will show a faint blue-green hue (faint because the response of the blue rod is small for the photons considered).
The intensity of the perceived image is determined by the rods. If your cones don't work or are absent your perceived image is black and white, while the image is colored by the cones. So if your green and red rods don't function you see an image with the same intensity as when they do function properly which means you see an image with the same intensity in black and white but with a faint blue-green hue, as already said.  
Now suppose (in theory) that only the left side of the M-curve and the right side of the L-curve applies to the reaction of the green and red cones. I.e. your green cones react only to pure colors left to the top of the M-curve and your red cones react only to pure colors on the right of the L-curve. So a reaction of photons with a wavelength lying between the top of the M- and the top of the L-curve with the green and red rods won't occur, so, one would think, these pure colors are not visible.
A combination of pure colors one can see though (with different wavelengths as the pure photon that can't stimulate the cones) can give the same visible color as the color associated with the pure color.
It's like the color addition of different colors of light (in contrast to color subtraction in making new colors with paint). For example:

The combination of green and red gives yellow. Which happens in the eye and brain too, as you can see. So if the pure color yellow isn't visible for someone, yellow can be made visible for him or her as a combination of two different pure colors (assuming the cones are sensitive for the two colors).
One can consider many failures of the cones, and nevertheless the pure colors that are not visible can be made visible as a combination of pure colors. But your color vision is, of course, impaired because for example some pure colors in a painting are not visible.
Insofar the UV photon is concerned, different people have small differences in the S curve, and though the curve maybe extends to the UV-wavelengths, with a normal lens UV photons light will never be visible. Or very faint with a new plastic lens.
About the flowers. I guess the first photo in black and white is for you visible in black and white, the second is visible almost the same as it is for me (in what kind of light is this photo made?) because there is some faint yellow visible on the tips of the petals, namely in black and white (it stimulates your rods in the same way as mine, while a bit my green and red cones), and the third is for you visible in black and white (because green only stimulates your rods), while for me it's visible in green because the UV-light (I guess that this is the case when this photo was made) is reflected by the flower in green (other flowers reflect UV in different colors, and mostly the UV itself too, especially in blue flowers). So UV-light isn't visible directly, but only when this light is transformed into visible light.
The first photo of the two below you see the same as I do, the second obviously not. You see the photo is in many different kinds and intensities of blue (when your blue cone is stimulated) and black and white (the green, yellow, and orange), insofar your red and green cones can't be stimulated by colors coming from the picture. Your rods though can be stimulated by the green, yellow, and orange (which makes you see the black and white).  
I don't understand though what you mean when asking this question:

Shouldn't the UV pattern still be apparent on at least one of the flowers, just in a different color? 

Here is a nice optical illusion, in connection with color and black and white, which seems to change a black and white picture into a color picture (assuming your eyes work well, so I'm not sure how you experience this).
A: One color blind friend worked with his father, a TV tech, and knew how to adjust color tv so it looked right to normal (majority) humans.  Their color receptors center on different frequencies or partly or completely miss some ranges, so their perceived color is different!  https://en.wikipedia.org/wiki/Color_blindness
Color can be blended because the eye is such a crude instrument.  If we mixed sounds of different frequencies, we would not be fooled.  Maybe a DSP might if it assumed a single frequency doing a FFT?  Even "tone deaf" people in languages where meaning is encoded into intonation are not confused, as these use major frequency shifts!  https://en.wikipedia.org/wiki/Amusia
