First.
"Known as the fastest sorcerer"
Besides the fact that it was the narrator saying this,
the databook states this
Second, the calculation is done by Hakari. He counts. With the databook statement, Naobito is clearly over Hakari.
Third, I'm referencing the characters who were existing during and before that statement, so Hakari, Yuta, Yuki, etc. The special grades wouldn't be known as special grades if their capabilities weren't known.
Fourth, everyone keeps using "people don't know about Hakari's strength", but Yuta's statement of knowing about Hakari's strength and saying he's possibly stronger than him from back when he was a student means that others would know his capabilities, which means others would know how strong he is.
Fifth, a curse user is a sorcerer. Saying "he's a curse user" means nothing.
Now the Heian era characters, okay they can stand out. But they scale to the modern day sorcerers. And if the modern day sorcerers are slower than Naobito, then that's that