The ACC is just really top heavy and the fact that Miami has been mediocre to bad for the last 11 seasons isn't helping because they should be one of the top programs in the country but have refused to get rid of Al Golden for some reason.
So you've really just got FSU, Va Tech and Clemson as the football "powers" and then you have Ga Tech who nobody really respects and doesn't have a real football reputation and they're not helped out by the fact that they run a gimmicky, triple option offense. Which is effective and fun to watch but still I think makes people perceive them as a side-show and doesn't help them gain respect. Va Tech is that former great program that we've watched slowly die and one that needed to make a change at HC in recent years but refuses to do so.
Next you have Louisville. A relative newcomer to the college football "elite" or upper echelon. A historic basketball program thats labeled as a "basketball" school and a program that was stuck in CUSA, Big East, CAA limbo/hell for the last 10 years. Louisville has produced a number of draft picks and does have 2 BCS Bowls to their name but still doesn't carry the national "weight" or perception of a big time program.
The same thing I said about Louisville can be said about Duke only on a smaller scale. They haven't had the success or the draft picks that UL had and they weren't stuck in crappy conference purgatory like Louisville but everything else applies.
We're also not helped out by the fact that the ACC has a lot of either bad or just mediocre programs. Syracuse, Virginia, Pittsburgh, UNC, Boston College, Wake Forest...are all bad or just consistently mediocre.
So the ACC is really only FSU and Clemson and to be honest Clemson isn't really that big of a power or elite program either. So its understandable that the ACC would have a bit of an uncertain identity considering the programs that sit atop its conference.