Well the majority of the map is PVE with only the one province being PVP, and from what I understand there will even be PVE content there as well, just more dangerous as you have to watch out for other players. So yeah I'm not worried about there being lack of PVE content.
There will be a more of PvP at start because they have prepared few PvE updates just post launch. Generic rules for PvP will not change for lone time, but PvE content might be added all the time.
They just haven't showed off much PvE stuff yet, but since we have 9 provinces of PvE and one of PvP, presumably there should be plenty of stuff to do outside of Cyrodiil
As others have pointed out, the alliance territories are exclusively PvE, and there is quite a lot of content there. As for your specific concerns about dungeons, there are quite a number you'll be able to explore, and we even featured a portion of one during our https://www.youtube.com/watch?v=_oWuw-mo_MQ. You'll be able to run into quite a variety of sites and creatures, hostile and not, while traipsing across the land.
dont let the map size lead you to believe anything, this game could be more pvp than pve even with a much smaller pvp zone. being said i feel they will have plenty of solid pve in story for us. the thing is, pve content takes more work. pvp is much easier to design for imo as balance is the main thing. if the players want more pve then they will develop it even more so after launch. of course if people want pvp they will go more that way. either way right now i feel there is a good balance and pve will be focused on in their upcoming updates