If you can afford to.
To me, the thing to remember on this particular controversy is that the deer doesn't care why or how you killed it. An animal removed from the breeding population is an animal removed from the breeding population regardless of who removed it, how they removed it, or why they removed it. And fish and game laws are there to protect breeding populations.
That's the consideration that needs to be topmost in the CO's mind. Maybe numbers are up and the population can sustain the pressure; you might turn a blind eye. But if numbers are crashing ... you don't have the right to steal from the grocery store to feed your family, either.
The guy poaching to feed his family is less morally culpable, sure. But the population dynamics don't care.