It wouldn't be a passive if the opponent has to use it, it's a passive because it's always active automatically, meaning at the very instant the match starts at zero time it's already affecting the opponent.
Of course there are some passives that don't have an immediate effect, maybe they poison the opponent over time or they slowly destroy them, those passives can be avoided even with less than infinite speed, but I'm not talking about those passives, I'm talking about the type of passive where you just instantly die if you're within range, or get instantly mind haxed if you're within range.