I was gonna give a longer reply but Zamasu gave me the opportunity to shorten my comment.
The reply to this would be "B-but the Daizenshuu said that there are 4 galaxies".
That is also not true. Sorry if this sounds repetitive but it's important.
The Daizenshuu 7 itself states that the "NWES Galaxy/Galaxies" are just denominations of areas and not literal galaxies. This is also present in the raws.
The four Kaiou who rule over the east, west, south, and north [sections] of the universe actually govern these galaxies.
Because the Kami in the Heavenly Realm and the Kaiou supervise the galaxies that exist infinitely in all the universe, the sections known as the East, West, South, and North Galaxies are denominations that came into use through their duty.
And about Herms :
Herms initially leaves space to the "4 galaxies" interpretation as in 4 galaxies in the literal sense. But he was already doubting this.
perhaps "Area" is the more accurate term that the Kaio use when talking among themselves (like with North and South Kaio), while "galaxy" is a more descriptive term they use when talking to the universe's inhabitants (like with North Kaio and the Great Elder)?
Herms here already brought up the possibility of the "NEWS Galaxies" being just denominations. Although he never confirmed anything. So this ain't evidence against the Daizenshuu. Rather it vaguely supports it.
But then, in his fan guide...
These 4 galaxies serve as the governing areas for the gods, and so each one is overseen by the corresponding Kaio and Kaioshin. In fact, Daizenshuu 7 says that the "galaxies" of DB are merely administrational units used by the gods, and so there's nothing stopping races from travelling from one galaxy to another, provided they have the technology to do so. So basically, these "galaxies" don't have much reality as physical divisions, and are sort of like the boarders between countries. It's not clear what significance the cardinal directions could have in space, but the map of the DB world shows what appear to be division lines between galaxies engraved into the living world's outer walls, so that might have something to do with it.
Herms brings up the Daizenshuu, and he himself admits that the "NEWS galaxies" are just denominations/units used by the gods to supervise the various areas of the universe, which is filled with actual galaxies.
Also I wanna bring up the fact that Herms mentions that Toriyama's map of the universe says that the universe is splitted along the 4 cardinal directions, and not in "4 galaxies".
So no, Herms never supported the "Daizenshuu said the universe is splitted into 4 literal galaxies". This was just the result of scans that have been taken out of context.