The fact that you have only done the Temple quests is the problem.
You have to get Helseth to set-up the duel (which is its own seperate quest that he initiates) and after that quest is finished (by talking to Helseth after the duel and getting a reward) Karrod will then speak to you about the sword.
Otherwise, the piece of broken sword is on Karrod's body, so you'll have to kill him.