Being impaired from anything is an issue. But I'd rather be driving with a stoned driver behind me than one on the cell phone talking and texting.
What about a stoned person texting?
Distraction is distraction and a stoned driver could be just as distractible as a sober person. For example, the stoned driver could suddenly become distracted by the sunlight refracting so awesomely when reflecting off a store window, and forget to watch ahead. Back when I'd get high, if I were really stoned, I could get so distracted by a bass line or a drum fill in a song that I could easily fail to notice a car braking ahead of me.
I would vastly prefer whomever is behind me to be 100% sober, highly alert and focused forward. Anything less than that is bad and the degree of badness is kind of irrelevant if they rear-end you. I've avoided being rear ended several times in the past couple years alone just by watching for non-alert drivers when braking and taking avoiding action.
But to your point, when I see someone who appears to be texting behind me, I pull over, soon as it's safe to do so, and let them go past. Texting and driving is incredibly moronic.