im importing Master data via SSIS and Im trying to figure out the best place to do de-duplication of master data.
The data is a list of Applicants. The thing is though that aside from an ApplicantID from the source system (incrementing integer) two applicants can be identical same firstname, surname, even address potentially.
I was considering using the DQS Matching Policy to resolve this, but there is no real need for a DQS domain for ApplicantID, and this data definitely does not belong in DQS due to its nature. It does however belong in MDS.
Instead of using DQS matching policy, i was considering using a subscription view as a lookup and de-euplicating against the data in MDS.
Is this an inferior method to de-duplicating via DQS and if so why?