RIght now, the primary flag on contact details can not be removed if it's the last record on file and you wish to make it inactive or DNC. It's defaulting to at least one record should be primary.

Going to benefit users as they update contact details rather than counting on Admin tasks to clean this up.