Well They probably have been working on it the entire 5 years, but in the same way they worked on Fallout 3. Some development was being done on it, but it wasn't their primary concern until after they released all the Oblivion stuff they were gonna do.
they did not just ameadiatly started working on TES V the day OB was released. no, they probably waited a while before they even started brain storming what the game was going to even be. todd howard certainly had enough time for conan the barbarian to inspire him and I know for a fact that at the very least takes a couple of months.
besides, they were working on other games remember, FO3. even if they had a couple of guys working on concept art and other
planning jobs for SK that isn't the point of this topic, its about delaying the release if there are major bugs, bugs will pop up as long as they add new things into the game, it doesn't matter how long they work on the game. they can spend 10 years and as long as they added quests and content into the game a few months prior to the release it is still going to have bugs that need to be ironed out.