Hi
We had a consultant that implemented DQS and MDS for us, so I am very green when it comes to DQS and MDS. We are loading data from 4 divisions and creating a customer master across the 4 divisions. Now that the consultant has left, I have been tasked with running the load on a monthly basis. The load runs through an SSIS package. I am currently trying to run a load for a month for one division (approximately 90k records). Everything is smooth sailing till it reaches the stored proc to update the relationship. This step is running for 4 days (96 hours). The validation of the model is also running for 4 days.
I have run the stored proc through the estimated execution plan and implemented the recommended index where I could, but this has not made any difference. I have also added in steps into the Stored proc so that I could trace where the hold-up is and it is after the error checking but before the creation of the staging table. I have run SP_Who2 and there are a bunch of Inserts happening at this point with ridiculously high CPU times (hundreds of thousand milliseconds). Can anyone advise what these insert statements are doing and why they would take such an extremely long time?
Any other pointers on how we can improve the load would be much appreciated. The instance is running on a 8 core server with 20GB ram.
Thanx
Vic