I couldn't get off that elevator fast enough

I'd just laugh and push the down button, if he said it in a creepy manner (though it doesn't seem as he did) then I'd just get out of the elevator

It would depend on how it was said. In a funny or awkward way, it would work. Intense with leering eyes, i'd get out and take the stairs.

Admit it, it would depend the most on what the guy looked like.
If he was a creeper, you'd have the cops called and waiting before the elevator even got to the next floor.

True. If he was covered in blood or dressed as a clown, no amount of adorable awkwardness is going to help.
(Edited because autocorrect is trying to ruin my life.)