Lumping totally dis-similar arguments together is just intellectually lazy, which is most of my problem with the use of words like "racist, sexist, etc." They have valid applications, but are often used to circumvent rational debate.
Most people are intellectually lazy. Better get used to it. If you want your arguments to be consistently distinguished from the racist ones you will have to put in more effort to make them seem more distinct.
