Have ridden the train in every borough at every time of day.
Have run into more troubled people than could be counted. As others have described, if you avoid eye contact and keep your distance you're generally fine. It doesn't hurt to be a large man with a poker face.
I was fortunate to never see a situation rise to the level that it did for Neely and Penny. But importantly, I never heard anyone explicitly threaten to kill or harm.
People rely on explicit threats being taken seriously to ensure their safety. This includes women who need protection from bad men in their life, public figures who are credibly threatened by the public, and individuals who are explicitly threatened by others. Our laws and courts have correctly realized that threats warrant policing, and are not even protected speech under the first amendment.
Penny and the riders on the train were directly threatened with bodily harm and death. Neely can't speak for what his intentions were in making those threats, but there's an inherent risk in threatening to kill people, and one of those people on that day decided the threat was credible.
You say you don't want the average citizen making threat assessments until things get physical, but that's a luxury only larger men can afford. If shit goes south we can run, divert, or fight. If we get assaulted we're more likely to recover physically.
Where does that leave weaker people? Are they supposed to play sacrificial sheep while every Tom, Dick, and Harry yells threats at them? Having to give up their cash or hope they can handle the first hit and that someone strong enough to stop their assailant will react? The generalizations and absolutes you describe cut both ways. The resolution process our legal system gives us is the jury trial.
And with respect to the danger to civil life: most believe that has already happened and that it originates with the permissiveness you exhort we broaden. You get more of what you encourage, and right now we're encouraging harassment, assault, drug use, and theft in our public spaces.
IMO it is a delusion that crime tolerance is beneficial to the economically disadvantaged. Much as a teacher who passes students who cannot read creates illiterate adults, a society that forgives criminality creates criminals.