Best way is to find the entry for the sword that originally uses the NIF file you want to use, edit that entry and give it a new EditorID. When you go to save, it will ask if you want to create a new entry. Say yes and you'll have a duplicate of the original weapon with your new changes.
Meshes pretty much need to be in exactly the same place as they were in the BSA, unless you plan to alter the mesh itself.
Easiest way to do this is to always extract more than one Mesh from the BSA. If you extract a single mesh, it will often just give you the mesh itself. If you extract more than one, it will usually create all the appropriate directories for you from the "meshes" directory on inward. Saves you from having to create those directories yourself.