I really can't think of any way. You could force bringing up the rest menu via ShowRestMenu, and then detect if the player sleeps, but I don't see how you could use this to detect whether resting is blocked in the cell (it will probably work even in non-resting cells?), or differentiate from the PC using normal beds found in the cell.
Is tightly controlling the PC's access routes into the area and then just keeping tabs on whether or not they've entered / exited the area an option?
Edit: alternatively, maybe having the area not flagged as 'no sleeping' and then lacing it with activators with scripts to detect if PC sleeps and then instantly wakes them with a message like "sleeping is not allowed here" or something? Downside is that the PC can't wait either...
Sorry, not very helpful!