Well, the two guys carrying assault rifles through downtown Portland "to educate people on the Second Amendment" did trigger a lot of people's reflexes for "scary nutjob", so there's clearly some sort of line based on how scary the gun looks.

Which, admittedly, is not the best of places to draw the line, though now that I think about it more, I think the amount of inconvenience it is to carry something is maybe a hint towards how likely you are to be planning to use it soon. I'm just as much of a nutjob (albeit a less dangerous one) if I'm carrying around one of those great big trash cans with me all the time, just in case I need to throw something away.