Then explain how only Dragonborn folks can wear it, please.
"only Dragonborn folks can wear the Amulet of Kings" is simply a theory that doesn't hold up to scrutiny. Dragonborn is not even a well defined concept. It's just a title Talos claimed and that the Septims tried to keep to legitimate themselves. But the Septims coming after Pelagius I were not even direct heirs of Talos.
This theory also makes de facto Mankar a dragonborn (uh ?)
The ritual of the Amulet of Kings does exist since at least Reman, so probably an Emperor cannot rule without wearing the Amulet of Kings (he/she is binding his/her soul to the gem anyway) and lighting the dragonfires, that's part of coronation.
Now explain to me how Katariah (a dunmer) and Uriel IV could rule Cyrodiil for one hundred years without even being able to wear the Amulet. Because she had exactly zero Septim blood in her veins, and Uriel IV might technically be a Septim, but it's not a great lineage anyway. If he could wear the Amulet all fine and the Amulet was so important, then why did the Elder Council not see him as legitimate and then proceeded to prevent his son Andorak from becoming Emperor ?
Events like Potema speaking at the Elder Council to prevent the soon-to-be Kintyra II from becoming Empress would not even make sense in this context : either Kintyra were the legit daughter of Antiochus and would be able to wear the Amulet, or she couldn't. But in practice, it was always the Elder Council that decided who would become the ruler of Cyrodiil. No case of the Amulet rejecting someone as Emperor/Empress is known.
To sum it up, the choice is between discarding Baurus' claims and liturgy books like Trials of St Alessia and Amulet of Kings, or retconning entirely 500 years of Septim history. It's a no brainer really.
Well strictly speaking, there has been only one person who could not wear it.
And that person became a shezzarine later on by mantling Pelinal, go figure.