Is there a single mind behind most of the stories within the elder scrolls series, or is each game written by a committee of quest designers and such? Who came up with the mythology? The in-game books? I'm curious!

I think many members of the dev team have written books. The writing was done by the dev team, mostly by the writers of course but I'm sure the designers pitched in ideas and helped out.

http://www.imperial-library.info/

At the Imperial Library go to the left of the screen at the Library Navigation and click Developers. The Obscure texts, so I've learned from Lady N recently, are articles posted by members of the dev team that expand and explain the lore between game releases. There is also an interview section as well.

So the lore is pretty much an ever ongoing project by the Bethesda writers. I don't know how many writers there are exactly but as you can see at least from the obscure texts that there are a good handful, and those are just the one posting those.