only way that can be an issue if you ask me, would be if you have very very low deer numbers
reason being if you have very few doe on your lands, and you kill one, that is one less to attract bucks...