Out of curiosity, what approximate (for the sake of anonymity) field did you teach?
My opinion of various introductory "weed-out" courses is:
Introductory chemistry requires a lot of memorization, but not a lot of mathematics.
Introductory physics requires a lot of mathematics, a lot of calculation, a lot of approximation and a lot of practice.
Introductory mathematics requires a lot of proof and a lot of philosophy for the proofs to make sense - otherwise, one is stuck with relying on "mathematical maturity."
Based on my experience and being a TA*, I don't believe that most high schools adequately prepare students for pursuing chemistry, physics* or mathematics. (Chemistry, though, would be the easiest of the three.)
I am not sure that many of the students who are applying themselves but not succeeding are at fault for their failure or for their poor performance. The gulf between the regular "football" high school and college is significant.
Your thoughts would be interesting. (I don't totally disagree with your opening sentiment.)