Fan negativity might hurt us more than the team losing