Buy blander colors and this becomes a non-issue. 

WTF is this blue & green nonsense?  Removing garments prominently featuring both blue and green would cost me nine ties, four shirts, one belt and one pocket square.

Cultural convention of some sort, perhaps?

Just as a curious aside, from Michel Pastoureau's Black: The History of a Color:
"Consider a single example drawn from the spectrum. For us, following Newton’s experiments and the spectral classification of colors, it is indisputable that green is located somewhere between yellow and blue. Many social customs, scientific calculations, “natural” proofs (the rainbow, for example), and everyday practices of all kinds are constantly present to remind or convince us of this. Now, for men of antiquity or the Middle Ages, that idea hardly made sense. In no ancient or medieval color system is green located between yellow and blue. The latter two colors are not present in the same ranges or along the same axes; thus they cannot have an intermediary stage, a “middle” that would be green. Green maintains direct relations with blue but has no relationship with yellow. Moreover, with regard to painting and dyeing, no recipe before the fifteenth century taught us that it was necessary to mix yellow and blue to obtain green."

"Two juxtaposed colors that constitute a strong contrast for us could form a relatively weak contrast for them, and vice versa. Let us stay with the example of green. In the Middle Ages, to juxtapose red and green (the most common color combination for clothing between the time of Charlemagne and Louis IX) represented a weak contrast, almost a monochrome. Now for us it represents a violent contrast, opposing a primary color and its complementary color. Conversely, to juxtapose yellow and green, two neighboring colors in the spectrum, forms hardly a noticeable contrast for us. Yet in the Middle Ages it was the strongest contrast that could be created; lunatics were dressed in it and it served to indicate dangerous, transgressive, or diabolical behavior!"

The beige on beige, or any same colour looks awful. 

White on off white works, but only if you're in the tropics without air conditioning and are a rumbullion drunk captain on the local tramp steamer smuggling illegals, Cutty-Sark whisky and martini-henries through the islands.

so many checks and stripes have both...

heard the same in Germany though: "Gruen und blau schmückt die sau" = green and blue graces the sow