I noticed this when I started to wear bras that actually fit - for a long while I was too broke to get new ones when the 38B and the 36C stopped fitting, so I had bizarre bulges. Bulges that made me think I was weird. Made me think maybe I had Things Gone Wrong inside, even.
(Never actually thought it was fat. I was lighter then, and more athletic before I embarked upon the Thesis From Hell.)
But it turns out that when I wear 38C my hairy armpits fail to bulge breastily. So! Properly fitting bras, the solution to many problems.