It's strange how this isn't in the bugs list in the uesp.

I can't progress in the quest The Elven Maiden because when I try to plant the bust in the cupboard, I get the message "You cannot remove quest items from your inventory".

I reload the save file and try again, same happens.

Are you trying to place it in the correct cupboard.