There certainly aren't many of us. According to Name Statistics, 0.004% of men in the US (about 4900) are named Asa. It's so uncommon that there are more women named "Richard" than there are men named Asa.