You're preaching to the choir here. Bethesda is done with Oblivion, period.
As for your problem, it is in fact a very well known bug. Here's the description on UESP:
It is possible for one of the Knights of the Nine to be stuck permanently following your character if you ask them to follow you at the wrong point in time. Since the knights subject to this bug will also be essential when the bug is triggered, it is impossible to ever get rid of them. The bug occurs if you do the following:
1. Complete the entire quest line (up to the point where Umaril is killed) without recruiting all nine knights.
2. During Sir Thedret's speech at the end of the Umaril the Unfeathered quest, you approach an unrecruited knight, and at that time simultaneously recruit the knight and ask him to follow you.
To avoid this problem, be sure to recruit the knights either before or after Thedret's speech, and do not ask a knight to follow you immediately after recruiting them.
Once the bug has been triggered, it is not normally possible (other than using the Console) to make the NPC stop following you.
On Xbox 360, one option is to disable the entire Knights of the Nine plug-in, load your saved game and resave it, then re-enable the plug-in. Before doing this, make sure to remove any items stored in the Priory or other locations added by KotN. You will lose all items added by KotN, and you will need to redo the KotN questline from the start.
If you have not done the Daedra Quest Molag Bal then you can simply complete that quest while he is following you and he will disappear.