Broly constantly got stronger during that fight, Goku did well against him, but was eventually overwhelmed and left with no choice but to fuse. Broly was even advancing against Gogeta himself, which is probably the reason he went SSB in the first place. Not necessarily because he was losing in SSJ, but because Broly's Reactive Power Level is so insane that you need to overwhelm it quickly with a massive power gap.
My point is, Goku can probably downscale from a weaker version of Broly, but it doesn't seem reliable to me to scale him to Broly's peak, and certainly not to Gogeta.