> You say you are a person, I say you are a banana. Who are you to dispute my facts?

I'm not quite sure about your intent, but I think this is a more effective phrasing:

"You say you are a person, but I say you are a banana. As a banana, who are you to dispute my facts?"

Or maybe "What are you to dispute my facts?"