The nonstop commentary pushed him off social media, out of public spaces, and is now impacting his work.