I'd wait until about midway through conference play. Then you'll know how bad a loss hurts you.
The one thing you have going for you is that you'll be playing with two very important players at the end of the year, and typically the committee takes injuries and suspensions into account when looking at certain bad losses. They tend to give you a slight bump based on the team you currently have on the floor at the time of selection.
I don't envy you though. You really don't know what's going to happen with your basketball season. You could sneak in the tourney. You could end up being great once you get everyone back, and you could challenge for the SEC. You could be mediocre and miss the NCAA tourney.
That's a lot of stress. I'd much rather be in our position where I know we're going to the NIT at best. If we somehow fail miserably and miss the NIT, it's just the NIT. If we do play up to our level and make the NIT, it's just the NIT. See, no stress, no worries, no reason to get up or down. It's so much easier this way than having to stress about silly things like NCAA tournament selection.