This basically sums it up. Even Prime Rayleigh has shown nothing that implies that he could beat the Oden who faced Kaido, where Oden considered himself as the only one who could take Kaido down at that time.
1.Clashed with Primebeard who sensed Oden from far away and considered him as something incredible.
2.Got back up immediately from a haki-imbued, named attack from Roger, and was barely injured. (He was also ready to take on both Prime Rayleigh and Gaban who wanted to 2 vs 1 Oden).
^In both those feats, Oden was nowhere in his prime since it was said that he has gotten much stronger when he came back to Wano. This happend 5 years before he returned to Wano, look at how Luffy and co. developed in 2 years (or hell, look at how much Luffy developed in 2 weeks when he was in Kaido's prison, from getting one-shotted by him to matching him).
3.This.
.
4.Also this.
.
5.Kaido said that there will never another Samurai as powerful as Oden.
.
5.What Oden said in his letter to Toki (Where Rayleigh was in his prime).
6.Permanently scarred Kaido with his first attack, while injured, and was about to finish him off with his second attack. This is a Oden who naked danced on the streets every day for 5 years, so he probably wasn't even in his top form.
7.Oden is a stamina machine, he fought and beat an entire yakuza gang when he was 9, bunch of samurais and families when he was 15, an entire army of hundreds of bandits, including Ashura, in one night when he was 18, he spent 3 days and 3 nights being dragged by WB's ship while he had only one rope for support and even after being attacked by sea monsters and facing storms, he didn't release the rope at any time, he fought Kaido and his army in a 11 vs 1000 (Where Kaido needed 5 years to even have a chance and yet they would have lost if it wasn't for the granny with Mr.2's powers who distracted Oden as he was about to kill Kaido), and the legendary hour is still one of the most impressive stamina feats in the whole series. Oden's stamina is ridiculous while Rayleigh has shown nothing that implies that his stamina is anywhere near Oden's level, he started to pant and get tired after holding off Kizaru, who wasn't even completely serious.
Oden takes this against either Rayleigh, old or prime.