Your justification: "Gotta prove I'm the pec gawd even in the winter."
Why you're wrong: Figuring out what fits your body type best is something every dude has to go through. But once you've sampled different sizes from various brands, going with the tighter fit will result in a classic "what you think you look like" vs "what you actually look like" scenario IRL. No one wants to see your struggle vanity muscles/nipples through a knit, so if you want to avoid the L, don't opt for the M.
That smedium sweater you thought showed off your build.