Looking for PowerObjects? Don’t worry, you’re in the right place! We’ve been part of HCL for several years, and we’ve now taken the final step in our acquisition journey: moving our website to the HCL domain. Nothing else is changing – we are still fanatically focused on Microsoft Business Applications!

PowerObjects Blog 

for Microsoft Business Applications


11 Dynamics CRM Data Migration Tips

Post Author: Joe D365 | May 8th, 2013

Data migrations can be complex. Whether you are migrating from Salesforce.com to Microsoft Dynamics CRM or from another contact management solution, you will come across challenges with data.

This blog will highlight 11 Dynamics CRM data migration tips that migration package developers might find helpful.

  1. Know the size of the database, and chose appropriate tool.

    Scribe is best known for shorter development cycle as it provides the point and click features. SSIS development takes time, but SSIS packages perform well in terms of speed.

  2. Know the age of your database.

    In addition to the obvious question—do we need all this data?—consider the age of the database when looking at the format. Were data formats changed during past upgrades? Was the data stored differently in previous years? It's particularly common to see changes in the way email is stored. When this is the case, your test run might look fine, but a full run reveals numerous problems.

  3. Do some data analysis.

    Run some aggregates on columns to know the type of data you have using queries or a tool like SQL analyzer. For example:

    1. How many distinct values do you have? You expect 2, but find 6.
    2. How many nulls are there? You expect zero but find a bunch in records older than 2003.
    3. How many empty strings? You expect zero but might find a bunch.
    4. Max length expected is 255? You find some with 1080.
  4. Dynamics CRM data migration tips
    Use staging database/tables if applicable.

    Staging tables become handy when the source data needs to be cleaned or transformed before moving it to CRM. Typical examples of data cleaning include:

    • Parsing a comma separated values
    • Removing special characters from the data
    • Translating values to fit into a multi-select field
  5. Resolve any lookups and references in the source query rather than in the package script.

    SQL queries are faster than web-service calls. So if a lookup needs to be resolved, it is good to perform the lookup in the source query itself. This saves the time of a web service call.

  6. Setup the migration packages with error handling.

    Store the error rows in a table or implement a mechanism to flag the source records that are processed. The server running the migration can go down any time for maintenance. Coming to work on Monday morning and starting the whole migration from scratch is a disaster nobody wants to have.

  7. Perform a trial run before actual migration.

    Ideally perform a trial run with the entire source dataset. This gives an estimate of time it takes to complete the actual migration and also helps identify roadblocks if any.

  8. Run the migration under a special migration user.

    Since Microsoft Dynamics CRM has named users, consider running the migration as a service account or a domain user. This will ensure that no records are assigned to the developer once the migration is done.

  9. Disable indexes, workflows, and plugins.

    Before starting migration, disable database index, deactivate workflow processes in CRM, and unregister plugins that should not be running during migration. AND remember to turn them back on once done!

  10. Involve the users who use the data in testing data.

    In addition to you checking the data and matching counts of the source and target data, have users review and test the data. The users know the data. It's important to have testing scripts identified for data testing rather than "eyeballing" the data and assuming it looks right.

  11. Run the packages in parallel.

    Divide the data into buckets. Run multiple instances of the packages in parallel. This speeds up performance drastically.

Lastly, and probably most importantly, if you need assistance, call PowerObjects! Experts at data migration from C2CRM, Salesforce.com, or whether you are just trying to migrate to Microsoft Dynamics CRM Online, let us help!

Happy CRM'ing!

Joe CRM
By Joe D365
Joe D365 is a Microsoft Dynamics 365 superhero who runs on pure Dynamics adrenaline. As the face of PowerObjects, Joe D365’s mission is to reveal innovative ways to use Dynamics 365 and bring the application to more businesses and organizations around the world.

5 comments on “11 Dynamics CRM Data Migration Tips”

  1. Nice Article indeed, I have 'I should have written that' knocking about my head.

    1. I would use DS3 from Simego is performance was a problem (I am biased).

    2.It is good to have some type of reconciliation or 'tick-back' process as some customers need some proof that all of their data has been moved into the new system or has a known location.

    3. Spot on.

    4. Staging tables/database are very valuable and can form a read-only copy for reporting. There is a lot of value in turning off the legacy system you are migrating from as they require licenses and maintenance to keep going.

    5. Great idea to resolve lookups or at least cache them, BUT do not be too tempted to create GUIDs yourself as Dynamics CRM will create sequential GUIDs. Doing it yourself can affect the performance of your new Dynamics CRM Online solution.

    6. For migrations I normally 'synchronize' from the staging tables allowing for easy restarts and no requirement to blank the target system every time. Just update with the new data/columns.

    7. If you can automate the migration, then do it every night. This allows the migration team to keep adding more migration artefacts every day and if you can use last nights backup then your users who know the data will be looking at yesterdays data (very powerful for them buying into the new system).

    8. Running under a special migration user is a great idea and allows users to see where the data came from in the future.

    9. Excellent tip! Turn off as much workflow as you can.

    10. Essential to work with the users as they will give feedback and will often resolve any issues about historic data entry practices like putting contact data into 'spare fields'. Document any decisions they make to help you later understand why you changed a process or transformed data in some way.

    11. Parallel packages for importing are a good idea but really only against each entity. Inserting different types of entity in parallel will cause reverential issues. This is a performance enhancement so it is best left until the end or it can confuse issues.

    12. Clean Data and 'Streamlined Processes' are very valuable and migrating data can teach you a lot. Be patient when cleaning data as sometimes this is more valuable than the shiny new system!

  2. Great list of Dynamics CRM data migration tips. This tips are very useful. Thanks for sharing.

  4. Please , I would like to know how to move dynamics CRM 2013 from one domain to another domain. Keep in mind CRM already has an admin account as well as some other users, custom scripts and custom entities in the deployment

  5. Hi Joe, How to migrate Microsoft Dynamics CRM 2016 to new cloud platform. AD/ADFS/CRM/SQL Environment.

    Thanks,
    Kingson

PowerObjects Recommends