See above.
I am thinking about buying a second jumper (at the moment, I only own one red cashmere jumper from Turnbull & Asser). While a yellow or green one might lead to "look, there comes the guy with the yellow/green jumper", with a navy one that somehow doesnt apply.
Thoughts on navy as an almost invisible colour?
Depends if your standing infront of a navy blue wall.
I like navy. Its invisibility depends on the context, but I think it is a reliably understated color, and I tend to agree with you about the difference between navy and yellow when it comes to sweaters. Green, I think, is not the same as yellow in this way, but of course it depends on the shade.
It is so much more than black, a colour that I don't rate at all, except for funerals.
A navy blue crew neck has got to be the most versatile sweater out there, it goes with almost anything. My favourite is an old M&S lambswool one which, unfortunately has now got a hole in it because I have worn it so much.