I'd agree with that, but after Daggerfall, elves were given a more modest lifespan, anywhere from 120 to 200 years, excluding the use of magic.
However, in Daggerfall, the Bretons are implied to age just as an other human would.
If we use Daggerfall as evidence, and at the very least we can say that Daggerfall is the undisputed source on Breton culture and politics (at the least) and these aspects of Daggerfall have remained unchanged after the numerous post-Daggerfall retconns, then we can assume that Bretons age in the same way the other human races do.
In regards to your quote Albidies, let us remember that Cassynder was half Breton and half Dunmer. Thus, the phrase "aged like a Breton" simply implied that he did not inherit his mother's long life and does not really imply that Bretons live longer then most men, but rather it actually implies they live the same span of time.