I'm about to begin work on a mod, very similar to Falskaar, in the sense of adding a significant portion of quests, new lands, and dialogue. Now I'm wondering will SKSE be worth it, like will I run into problems with out it? I would like to do some of different things with audio like implementing them into different parts of the game, so I fear i might need it. But I don't want to have people to unnecessarily download it.

Help is appreciated.

If you run into problems like vanilla Script Object limitations, and you need alternatives(if possible) to get around them, that's where you will need SKSE. And that is one of the key reasons it's worth it. Granted, there are some things not even SKSE can get around.

With Falskaar, every time SKSE would have been helpful it would have been for a SUPER minor little tweak that would've been 'neat'. As such, I went entirely without so that people that happened to not have it wouldn't need it. However, if you're working on your mod and you find that SKSE would enable you to make massive feature additions and make your mod what you want it to be, then definitely go for it. Don't let "not wanting people to have to get it" get in your way. If the benefit would be super minor and hardly noticeable, maybe no without. But if the benefits would be plentiful and really change your mod, by all means dive right in.

Oh, and the same applies to the use/requirement of DLC.