1 - turn off the HUD, 100% transparent. (no compass, quest markers, or POI's)
I used to disable my compass. But I decided that is more unrealistic than having a compass. With all the Dwemer tech lying around and magical devices we encounter I have to assume there is some form of compass or some way of locating directions in Tamriel.
I use a mod that allows me to change to a slimmer, less eye-catching compass and I also use it to turn markers off. So I now play with a small compass that displays only east, west, north and south. I find this to be a nice compromise.
2 - turn music off (no combat warnings, hear the sounds of nature)
I use a mod that eliminates combat music only, allowing the exploring music to play uninterrupted. Like you, I hate being psychically alerted whenever an enemy spots my character. But I also love Skyrim's music, so this is a great compromise for me.
3 - only carry a realistic carry amount (no having 16 swords in your invent!)
I mod my characters to have beginning carry weights similar to the past games (in the 150 range). I rarely put points in Stamina on any of my characters, so they tend to stay with this carry weight for most of their games. In addition, my characters carry a Woodcutter's Axe (needed for making camp fires for the Frostfall mod) and a Miner's Pickaxe, which further reduces the amount they can carry. I also rarely take perks that reduce armor encumbrance.
4 - no fast travel unless necessary
I do not fast travel at all. Ever.
I don't know if this qualifies as "immersion" but one thing I have begun doing over the past year is avoid looking at my map. It's amazing how gigantic the game world feels when I don't see a miniature version of it every two minutes and when I need to navigate by studying the landmarks around me.