In case of need, there are ways of having immeasurable speed and not scaling to the characters; it depends on how the immeasurable speed is reached.
That is, if a character only reaches immeasurable speed in a specific state (Like by moving outside linear time), it's possible to separate its speed as "inside time" and "outside time". It's based on one of the many hypotheses of eternal nature, one in which it doesn't extend to linear time, but exists as its own thing, separate from it (Personally, I think the way Lygus was defeated as a clear exploration of that idea, it clearly exists outside of Amphoreus' cycles, but needs to descend to its level to interact with it outside of restarting time).