It's a pretty safe rule. Another saying I've heard is that if two people get in a knife fight, two people die. One dies on the pavement, the other dies in the ambulance.
In close range the knife wins. Actually within a 3 step radius the knife has the advantage. Beyond that the gun is superior. The saying you heard stands is correct, however there have been many attacks and fights involving knives wherein the original victim escaped unscathed either due to circumstances or skill of some sort (perhaps martial arts)
Depends on if it's a simple holster or a if it strap and whether or not the safety is on. Also if the knife is drawn. In a sort of cowboy standoff with no safety and both weapons holstered at 7 steps the knife will lose
So fucking sexy