You don't have to be horny to consent to sex - I think, as long as you say "yes, this is ok" - and technically, you sort of consent to something by doing it to yourself. Like, you can't really do something without your brain giving it the ok first? Does that make sense?
Being aroused by not horny sucks. I still prefer it to being horny but not aroused tho.