Not a fan of Tattersall. It's seen a lot round these parts. Usually blue/red under a well worn fisherman's jumper with baggy cords and Cornish pasty shoes or green wellies.
I could see it working if you sharpened up everything else around it. Lose the country colours pallette. Navy trousers and knitted tie? Black loafers or longwings?
I think I'd end up changing the shirt just before I left the house though.
Boyer wears his with a decent tweed jacket and tie and solid-looking shoes (from what I can make out). But it'll have a different feel to it in Midtown or wherever he hangs out than in the chinless wonder/cow-dung smelling towns of Merrie England; will it not?
I disagree with losing all of the country colours. Vary them a little instead.