The first couple zones are still drastically overpopulated, so invasions are happening all the time. It slows down as you move into the higher level zones. In fact all the people who rushed to 50 have been complaining about the lack of invasions, because there apparently aren't enough level capped people yet to trigger them.
As a pure melee toon on my main soul, I feel your pain. Generally when I get in the huge groups I pull my zoom all the way out so I can see the target indicator better. I'm hoping they do some tweaking there or allow UI customization.
My understanding is that if all the players sat back and did absolutely nothing, an invasion would overrun the map, even the "safe" areas like Sanctum. So you wouldn't be able to quest, vendor items, etc, until it was restored by fighting the invasion back or waiting for it to reset (after a couple hours?).
AFAIK there are no real penalties for failing the invasion quests, other than losing out on rewards.