The biggest downside to the campus size, in my experience, is that we have fewer courses each semester than bigger schools. You won't necessarily be able to find that narrow speciality course on the one decade of Albanian history that really fascinates you or whatnot. For its size, though, the course offerings are pretty good, and the Rochester Curriculum gives you tons of flexibility.
Speaking of the Rochester Curriculum... It's awesome! General ed requirements sound like a horrible idea, and they don't exist at Rochester. You have to take either a major, minor or cluster (3 related courses) in each of 3 broad thematic areas. Plus, you have to meet your major/minor requirements and take one freshman writing class. Other than that, it's up to you! With 32 classes in a normal full-time schedule over 4 years, and many majors taking less than half that many to complete (Poli Sci takes 12. History takes 10. Math/sciences usually take a lot more), there's usually tons of room (for Humanities/Social Sciences majors, anyway) to take random classes just for fun.
There's a wide difficulty range between different classes. If you plan your schedule entirely around easy grades, you could probably get a 4.0 without too much difficulty with a moderate amount of work. In some classes, you'll be buried to the neck in work just to pull a B. It pays to ask around, or to use various online resources to pick profs and classes.
As with the classes, students' grade consciousness varies a lot, too. There are a fair number of GPA competitive students trying to go to law school, grad school, etc. There are also a fair number of students who could care less so long as they don't get kicked out or lose their scholarships/grants. As I said, there's a wide range in class difficulty, so if you want a challenge it's there, but if you want to coast through you probably can do that as well.
As with difficulty, classes range a lot in terms of quality of discussion. I've taken some classes with discussion that was, in my opinion, easily grad school level. I've taken some classes where, every time certain students opened their mouths, I began to fantasise about jabbing my pen deep into my eye socket to poke a hole in my brain and end the misery. Smaller, higher level classes are obviously more conducive to good discussion. There are a variety of classes in the Comp Lit department that are usually under 20 students with over half grad students. These classes are awesome if you like good discussions and knowledge for its own sake. They will also never help you get a job outside teaching ;)
The Quest courses offered to incoming freshman are great. I took the one on Nonviolence, and it's one of the best classes I've ever taken. I've heard mostly good things about the other ones, too. If you find one that interests you, I highly suggest signing up for one.