Unfortunately, yes. Some people through the years have done pretty much that. It's not looked upon too kindly by the modding community in general though, for a number of reasons.
edit: Though, on a more pleasant note, you can certainly use staggering amounts of varying types of mods together without so much as a crash or stutter. I should know.