I've seen this argument happen before: conservatives (at least libertarian-oriented conservatives) point out that, while communists may be liberals... fascists also happen to be liberals. The stock example - and I've used it myself - is to simply point out that the Nazi party in Germany were explicitly socialists, right in their name.
Now, the resistance still happens - and while I think the counter-arguments are wrong, despite there being some interesting rationales in them, I think a fundamental problem is closer to something like this: the commonly acknowledged political sides need a kind of roided-up monster-version of itself. For the liberals, that side is the communists. The communists and the fascists hated each other, ergo it makes sense that the fascists were on the other end of the spectrum, and since the communists are 'extreme left', then the fascists must be 'extreme right'. Left and right, yin and yang - the opposing side, taken to an extreme, must be scary.
Otherwise... well, it's just not fair. If communists are liberals and FASCISTS are liberals, well shit. Liberals are kind of double-monsters in that case, aren't they? Where's the scary small-government, religious-rights conservative example in history? That there may not be one seems unfathomable.