From what you've told me the way you respond to him is blunt and rude. It's not the best route with your family in any circumstance. Be smart about your responses from here on out.