I always thought this was amusing. It almost always came up as some sort of rule that students said but was never, in any school I attended, an actual rule. If anyone actually left, regardless of how late the teacher was, they would be marked as not attending the class. The only time you were excused is when someone from the administration came by and let everyone go.
Funny that this seems to be an international thing.