The cosmology is clarified in OV.
In UA, Paradox takes the gang to "the space beyond" "the universe." In this scene, they only show us two universes depicted as clusters in the black void within the overall timestream. You can consider this the "space between universes" in the present tense.
We are given more context to this "space beyond" universes in OV. After the CTB goes off, Paradox takes Ben to the same space beyond. Here we see numerous clusters. Now, we are given clarity on what each cluster represents. Each cluster represents a branch universe, and those clusters are destroyed, bringing us to the same black void within the overall timestream as before.
Each cluster represents a different universe timeline branch which, in turn, is a collection of space-time dimensions that make up that universe. We only see clusters, not lines, since they are depicted in the present tense.
This matches with how OV treats the term "the universe" collectively.