If you have an iPhone, there are a few ways to take off a contact. The easiest way is to use the three-finger swipe gesture. To do this, you will need to hold your finger over the contact and then swipe your finger up and down.