Your theory assumes that we would immediately fall back in line if we went back to the ACC. That's a big assumption.
It's not a theory nor an assumption that SC is a mediocre football program, they have been in any era. I'm also not assuming they would return to the ACC. Given their history with the conference both as a member and nonmember it was logical to consider how they have done against them. When comparing them to the ACC since their departure until now, as far as records go, it's simply nothing special. If you remove Clemson from the equation as that's the only ACC team they've always played since leaving, I think that does come out to a record a little over .500.
But something I do theorize on is this: I tend to think SC made certain investments and improvements in the football program because of joining the SEC which they otherwise would not have. So that elevated the program from where it was, but the competition on field increased as well. So if all things were the same for SC since 1992 (same players, same coaches, same facility upgrades, etc...) and they rejoined the ACC instead of joining the SEC then the on field results could look much better. If you narrow the record of SC vs ACC opponents since 1992, not named Clemson, I think the record is 10-6. Not dominate, but clearly good (but is admittedly stretched out over 24 years). And they had that run of being better than Clemson when Clemson began climbing the mountain and was the class of the ACC and not yet a natty contender. So there's indicators they could have done much better in this imaginary world, on the field.
So there's reason to assume they would not plummet, and there is reason to believe they still would not be much better off outside the SEC.