It may depend on your PC system, stuff like how much RAM you have could effect how well is switches between apps. Load the web browser you wanna use and then load Oblivion. While in Oblivion windowed mode, just press ESC to pause and then try ALT+TAB to the browser and ALT+TAB back to Oblivion when you want to play again.
I play Oblivion in a window to give the FPS a little boost and I do this all the time. I keep notepad open at the same time as Oblivion and if I need to make a quick note or if there's something I want to look up, I can just ALT+TAB to switch between windows, easy as pie. If I don't have any other windows open while I'm playing though I can't use ALT+TAB so what I'll do if I need to do something outside of the game is CTRL+ALT+DELETE to open the task manager which is a window in and of itself. From there I can open any program or file that I want while Oblivion is still running and waiting for me to return to it. Just remember to close the task manager once you've got whatever other program you want open. Pushing CTRL+ALT+DELETE again while it's open can cause your PC to restart (may depend on the computer or computer's settings.)