Duplicate Detection Process - Online View
1/ Duplicate Detection within Online View is becoming more sophisticated, including Soundex checks and a check on Gender.
However users who are hoping to use the report in order to Merge Duplicates, will only want to see those pairs for whom there is sufficient evidence to be sure that the pair of records are for the same individual.
2/ In UK, we have various identity numbers (Passport Number, Driving Licence Number, National Insurance Number, National Health Service Number (NHS), Tax Number, Student Number), however not everybody has one of these, and if a charity were to ask for these details they would usually not be supplied.
The NHS used to use Full Name, Date of Birth, and Gender. However there were sufficient individuals sharing the same identification details with another individual that the NHS Number System was devised, and charities are usually not given the DOB anyway.
It leaves us trying to identify individuals by their name, and one or more contact details (Postal Address, Email, Phone Number).
It is a long way off perfect, since there can be two "John Smith"s at the same Postcode (part of a Street or a Nursing Home, etc.), and couples and sometimes friends share a postal address, an email address or a landline.
Also whenever an individual changes one of their contact details, then unless they inform the charity of their previous contact detail (usually do not) then they will be given a new record in RE, which ruins any history of involvement analyses and actions based upon them.
GDPR (EU Data Protection Regulations) offers no ideas on this subject.
Not for Profits who operate a Membership scheme are a little better off since for members they can use the Membership Number (e.g. Alumni).
3/ Duplicate Detection in Online View offers no filtering. We cannot even Detect Potential Duplicate Individuals, then later on Detect Potential Duplicate Organisations, or filter on those with a postcode.
The only sort sequence useful in this regard is Match likelihood (High to Low), but that produces swathes of pairs who only match on name (or Soundex of name).
We cannot export the list of pairs to Excel to sort/filter it there.
Would it not be more helpful to save us wading through so many pairs to provide an option to filter the list of pairs to only show those pairs for whom we have enough information to merge them?
The information that we need to see when Merging Duplicates is different from the information that we need to see when adding a new constituent.
4/ I have already checked with BB Helpdesk, that there is no point in marking these potential dupes as “not a dupe”.
If we have two records with same name, then they both might acquire more contact details in the future.
But if we flag them now as “not a dupe”, then RE will never reassess them again, and also we cannot unflag them.
So therefore we cannot flag these potential dupes as “not a dupe”, and must wade through the same list of “might be a dupe but we do not have enough information to merge them” every time we look at a Duplicate Report.
5/ This is a temporary problem - but currently Online View will not merge Tribute information, and Database View will not merge Attachments added in Online View. Hence what we would like to do at the moment is use the improved Duplicate Report from Online View and then run the Merge Duplicates in Database View. However we can only do this by printing the Duplicate Report from Online View, which is five pairs at a time and full of pairs that only match on name (or Soundex of name).
Would it be possible to export the Duplicate Report into a list in Excel?
6/ Would you be able to include a list of common firstname abbreviations? Then you would be able to consider Susan as same as Sue, Nicky as Nicholas, Bob as Robert, Bill as William, etc.. Non English names may also have some common abbreviations.
7/ Would it be better if the check on Gender was - if both of the pair have it specified and they are not equal, then the pair is not a possible dupe? Currently the process just seems to lower their confidence score.
Would it be possible to include a check on Date of Birth - if both of the pair have it specified and they are not equal, then the pair is not a possible dupe?
8/ Many charities write the Cons Id onto any accompanying paperwork from the supporter.
Hence when merging records, they record the Merged From Id(s) into the Merged To Record, often in Alias or in an Attribute, in order to be able to still find these records.
Would the Merge Process be able either to do this for us, or after merging a pair make available to us the Merged From ID Number, and a link to the remaining Merged To Record?