This isn't about the rareness, its about the timeframe. The varying tiers of characters doesn't matter and it doesn't matter if people have done it, the point is that the timeframe in which to do it is not readily accessible to high tiers like Gojo, the timeframe is what's required to activate BF and the concentration is what's needed to reach that. Given no one can stay at that level of concentration is what shows that the characters aren't at that level, they are getting lucky through concentrating. As we saw with Yuji he
tried hitting BF but since he was not concentrating to such a level that's needed so he couldn't do it.
Another way we can look at this is that we take someone playing a game where they have to be precise with their timing to hit a certain target at an exact moment, unfortunately for this person (sorcerers) they can never quite do this accurately nor can they do this with readability. Instead this person (sorcerers) require themselves to focus greatly to the point they
might just hit the target and even then given this concentration they still can not hit it at will.
If this doesn't explain it, I don't know what will.