Calculus was pretty brutal...
It took me a few weeks to "get it" in Calc 1, then it all made sense.
Calc 2 I thought was pretty brutal because you had to apply so many trig identities to simplify the problem down to something solvable.
Calc 3 was somewhat simple after that.
Diff Eq was really cool because you take complex equations, transform them, then solve via algebra and transform them back.
I took vector calculus (grad level course) my senior year also, which was pretty brutal. I think that was mostly because I was spending so much time on my senior design project though and didn't put enough time into vector calculus