I do this all the time

That could make the person seem like you're intentionally ignoring them

But you would be intentionally ignoring them and why should that be a problem?

Because if you let the phone call go the entire time it'll seem like you're not able to get to your phone, I suppose