The main reason for this is namely that it can be a bit difficult to swap outfits. You drop into town and put your business suit/fedora on to do business but back in the wasteland you need to requip you outfit and half a dozen pieces of armor - you could also accidentally get mixed up if you are hailing a lot of different pieces.
Would be cool if you could craft together all the pieces of, for example, leather armor(if all pieces are the same you get an added bonus to that armor type) to say a vault suit - in your pip boy inventory that would all be one item now.
The disadvantage is that you wouldn't be able to swap individually piece about without any workbench, but of course you still have the option to equip it as you do currently. Win win