Very interesting style! Trying to think what I'd wear this with... I feel like they'd clash with a shirt collar, maybe a t-shirt really would be best...
varsity sweaters with college letters and such predate Ivy by decades, same with cardigan sweaters with letters, though the latter kind usually featured the standard cardigan style, which lacks a collar per se; this varsity cardigan sweater, with its pronounced rounded, ribbed collar borrowed from the knit collar of the classic contrast-sleeved varsity jacket, may have existed but is different from the more typical shawl-collared varsity cardigan sweater ......
to recap, I've seen them without collars and with shawl collars but not with the collars usually found on varsity jackets, which does not however mean they never existed at all. It's hard to believe that that combination was never tried before .....
are they Ivy? historically, maybe not exactly, but they are certainly Ivyesque, and if they had been around with those details in the '50s and '60s they would have fit in with all the other variants of varsity gear, people would have worn them and liked them, there is nothing inherently non-Ivy about that style ....