Go to 8 teams would be a good start. Should have been that way from the beginning.
Or even 6 and give 1 & 2 a first round bye for being ranked top. Assuming they really are the best, it would mean nobody would play anymore games than are already played in the current system. Now if 3-6 make it to the champ game, then yeah, there's an extra game played.