I think the key is to keep things fairly slim, the tie, the trouser leg. A navy sack blazer would do me fine 9 days out of 10. My strong preference is for brass buttons, and I'm not a 'gold' person.
One look I did rely on a lot over the winter was white OCBD, navy Smedley crewneck, and Brooks navy blazer. Either with charcoal flannels or dark jeans.

Mmm...  Having bought this Dalkeith I'm doing a good deal of pondering, even though it probably won't be seen in public for another four months. 
Apparently I owned a Brooks blazer once upon a time.  I can only vaguely remember doing so and I probably didn't like it.
I would want, above all, to steer well clear of the 'associations' a navy blazer can have in England, so a 'Warhol approach' might be favourite, dressing it right down with a pale blue Makers, Levis and sneakers.

I no longer own a blazer and i can't imagine wearing one nowadays. The blazer was always part of school uniforms and regimentation in my mind. I did wear them with grey flannels and black sebagos, OCBD and stripped tie but that was a long time ago now . My blazers were always single breasted. Double breasted never, ever considered. Now where is my hoddie? I'm off out.

Whatever happens, I shall certainly play around with it. 
I'm sometimes amused (and flattered) when people tell me how smart I look.  Not really my intention.