Data migrations can be complex. Whether you are migrating from Salesforce.com to Microsoft Dynamics CRM or from another contact management solution, you will come across challenges with data.
This blog will highlight 11 Dynamics CRM data migration tips that migration package developers might find helpful.
Know the size of the database, and chose appropriate tool.
Scribe is best known for shorter development cycle as it provides the point and click features. SSIS development takes time, but SSIS packages perform well in terms of speed.
Know the age of your database.
In addition to the obvious question—do we need all this data?—consider the age of the database when looking at the format. Were data formats changed during past upgrades? Was the data stored differently in previous years? It’s particularly common to see changes in the way email is stored. When this is the case, your test run might look fine, but a full run reveals numerous problems.
Do some data analysis.
Run some aggregates on columns to know the type of data you have using queries or a tool like SQL analyzer. For example:
- How many distinct values do you have? You expect 2, but find 6.
- How many nulls are there? You expect zero but find a bunch in records older than 2003.
- How many empty strings? You expect zero but might find a bunch.
- Max length expected is 255? You find some with 1080.
Use staging database/tables if applicable.
Staging tables become handy when the source data needs to be cleaned or transformed before moving it to CRM. Typical examples of data cleaning include:
- Parsing a comma separated values
- Removing special characters from the data
- Translating values to fit into a multi-select field
Resolve any lookups and references in the source query rather than in the package script.
SQL queries are faster than web-service calls. So if a lookup needs to be resolved, it is good to perform the lookup in the source query itself. This saves the time of a web service call.
Setup the migration packages with error handling.
Store the error rows in a table or implement a mechanism to flag the source records that are processed. The server running the migration can go down any time for maintenance. Coming to work on Monday morning and starting the whole migration from scratch is a disaster nobody wants to have.
Perform a trial run before actual migration.
Ideally perform a trial run with the entire source dataset. This gives an estimate of time it takes to complete the actual migration and also helps identify roadblocks if any.
Run the migration under a special migration user.
Since Microsoft Dynamics CRM has named users, consider running the migration as a service account or a domain user. This will ensure that no records are assigned to the developer once the migration is done.
Disable indexes, workflows, and plugins.
Before starting migration, disable database index, deactivate workflow processes in CRM, and unregister plugins that should not be running during migration. AND remember to turn them back on once done!
Involve the users who use the data in testing data.
In addition to you checking the data and matching counts of the source and target data, have users review and test the data. The users know the data. It’s important to have testing scripts identified for data testing rather than “eyeballing” the data and assuming it looks right.
Run the packages in parallel.
Divide the data into buckets. Run multiple instances of the packages in parallel. This speeds up performance drastically.
Lastly, and probably most importantly, if you need assistance, call PowerObjects! Experts at data migration from C2CRM, Salesforce.com, or whether you are just trying to migrate to Microsoft Dynamics CRM Online, let us help!