Sounds like a bug. The Staff should cause the plates of the Eye to drift together until it's a solid ball. (Then switch to your best bow with best arrows and shoot Ancano until the arrows longer longer damage him, then switch back to the Staff and zap the Eye again. Repeat until Ancano is dead.
There is the possibility that while the graphics are failing to show it, the program considers that the Eye is, in fact, closed. In which case, keep cycling Staff and weapon until Ancano is dead. If this isn't the case, go back to an earlier Save -- there should be a fairly recent Autosave from when you entered the building prior to the climactic battle -- and see if the same thing happens. If it does, you may be hopelessly bugged.