I've thought the same thing for a while now. Completely different cultures that don't seem to have any respect for the other. It would be pretty fun, I think.
It would create some very intriguing match-ups. Maybe even a rotating bowl game. One year, it could be played in Phoenix or LA, and the next year it could be played in Dallas or Atlanta. I think it would be awesome.