Looking for PowerObjects? Don’t worry, you’re in the right place! We’ve been part of HCL for several years, and we’ve now taken the final step in our acquisition journey: moving our website to the HCL domain. Nothing else is changing – we are still fanatically focused on Microsoft Business Applications!

PowerObjects Blog 

for Microsoft Business Applications


CRM 4.0 - The Microsoft CRM Asynchronous Processing Service service terminated unexpectedly

Post Author: Joe D365 |

Updated April 16, 2008.

I'll be writing a series of articles regarding the various issues that we have encountered while installing CRM 4.0 and migration CRM 3.0 to 4.0.   We have found solutions and fixes for all of these.  Hopefuly these articles will save you some time or tickets with Microsoft.

Article one - the dreaded "The Microsoft CRM Asynchronous Processing Service service terminated unexpectedly"  service crashes.

After finaly installing CRM 4.0 and upgrading your organization, everything appears to be running OK.  However, a small crm service called MSCRMAsyncService keeps crashing.  The quick fix is to set the service to automatically restart after a crash and do this perpetually:

services.jpg

However, this is not a good fix as it will lead to a small memory lead that will eventually cause CRM to completely stop responding.  This will also make your system log not very usable as it will be filled with service crashes and restart:
systemlog1.jpg

So what is the fix?

Here are the 3 action items should fix this if the issue is related to workflows being 'stuck' or not running properly.

Action A.  Set the  MSCRMAsyncService to run as 'local system' instead of 'network service' account.
localsystem.jpg

Action B.  The MSCRMAsyncService is processing a workflow and for some reason it is crashing.  These workflows are probably old, stuck, and perhaps missing some type of required resource.  To identify these workflows, use the System Jobs in the Settings area.
systemjobs.jpg

Then created an advanced find view to search for 'in progress' or 'waiting for resource' workflow jobs.
advanced-find-inprogress.jpg

Examine the actual workflow to see why they are not running properly.  First,  fix or stop the actual workflow that is not running properly.  Then go back to this advanced find, and 'cancel' any in progress and 'waiting for resources' workflow that has been in this status for a while.
cancel.jpg

Depending on the number of running/stuck workflows that are being cancelled, this may take a while.

This alone may solve this issue.  If it does not, then proceed to plan B.

Action C.  So after having this blog post for a few weeks I received an email from Microsoft Support stating that Action C can cause major damage.  Action C used to suggest to a full sql backup on the config database and the organization(s) database then delete dbo.WorkflowWaitSubscriptionBase, dbo.WorkflowLogBase, dbo.DuplicateRecordBase, and dbo.AsyncOperationBase.  Do not do this as it could have side effects including stopping out of the box jobs such as duplicate detection jobs, contract state jobs, and full text catalog jobs.

Instead take a look at KB949844 and KB950680.  These are two hotfixes related to the CRM Asynchronous job.

We would also like to offer our services if you are sick and tired of dealing with this. We host Dynamics CRM / XRM applications for many other partners and companies. We have put together a series of packages that we have found fit the needs of most groups - you can see information on the program here PowerISV for Dynamics CRM

If this helps you, please let us know!!!

Joe CRM
By Joe D365
Joe D365 is a Microsoft Dynamics 365 superhero who runs on pure Dynamics adrenaline. As the face of PowerObjects, Joe D365’s mission is to reveal innovative ways to use Dynamics 365 and bring the application to more businesses and organizations around the world.

Leave a Reply

Your email address will not be published. Required fields are marked *

22 comments on “CRM 4.0 - The Microsoft CRM Asynchronous Processing Service service terminated unexpectedly”

  1. I have upgraded 3.0 to 4.0 and nothing is being delete from the DB. One example, incident.deletionstatecode is set to '2' for 3 days now and there are no eror in the event log. I have re-started the Crm Async. Process. Any ideas?

  2. Thanks for the idea but it made it worse. Work Flow stop working. After re-install 4.0 to reset to network service Work Flow still does not work. I will try to rebuild tonight.

  3. Check to see if any workflows are 'hung'. Is the workflow service keeps constantly crashing - clean out these tables:
    DELETE FROM dbo.WorkflowWaitSubscriptionBase
    DELETE FROM dbo.WorkflowLogBase
    DELETE FROM dbo.DuplicateRecordBase
    DELETE FROM dbo.AsyncOperationBase

    Alex

  4. Nice post. In CRM 3.0 can i use following script to delete old stuck/failed Workflow, because in Workflow monitor status shows as "Running"

    DELETE FROM dbo.WorkflowWaitSubscriptionBase
    DELETE FROM dbo.WorkflowLogBase
    DELETE FROM dbo.DuplicateRecordBase
    DELETE FROM dbo.AsyncOperationBase"

    -Manish

  5. Guys (Gals if we are out there)... Be careful deleting the contents of the four db's as I have been on phone with Microsoft for almost 9 hours after I did it. They suggested not doing it. I must give Kudo's to the MS team though (especially Tami). I will update this site if we figure out what happened. Thanks

  6. I am unable to delete from the AsyncOperationBase. When I try, I get the following:

    Msg 547, Level 16, State 0, Line 1
    The DELETE statement conflicted with the REFERENCE constraint "AsyncOperation_BulkDeleteOperation". The conflict occurred in database "AmericanProfitRecovery_MSCRM", table "dbo.BulkDeleteOperationBase", column 'AsyncOperationId'.
    The statement has been terminated.

    Any suggestions?

  7. Tim how did you resolve it?
    I am trying to delete out part of my asynoperationbase table and am getting the same error. My table is huge and takeing up all the disk space on the server. 45,332.695 MB and counting.

  8. Hi,

    I am a newbie to Dynamics - one last thing I am unable to resolve following an installation for a client.

    Environment:
    Small Business Server 2008 (Premium Edition). Therefore SQL 2008 as well.
    And yes, I know we should not run all the server roles on 1 box, but there are only 10 users, and it is a powerful server.

    We have a problem with import jobs that just have a status of "waiting". I have read and re-read the blogs and other posts, and so far none the wiser.

    I have tried the following:

    Installed Rollup 6 - no difference
    Cancelled all the jobs via Settings -> system jobs, and deleted them
    Re-created the small import job (5 records)

    Help greatly appreciated,

    Don

    1. Hi Don,

      First thing I'd try is to change the async service to run as 'local system'.

      Do other workflows run?
      Did you change any iis settings (such as ports) after installin gms crm?

      Alex

  9. Tony,

    One thing to check would be to make sure that the Asynchronous Process service is running as 'Local System' rather than 'Network Service.' I have seen the Async service stop when it is checked as 'Local System'

  10. Changed Async to run as Local System, rebooted the server, and boom, thousands of stacked up workflows completed successfully. Thanks a million for this post!

  11. ***Answer to Tim's response***

    It looks to me like the system is trying to remove a record from the AsyncOperationsBase table, however the record it is trying to delete is needed by a record in the BulkDeleteOperationBase table. The BDOB table has a column, AsyncOperationId which points back to AsyncOperationsBase table. So I would have to remove from BDOB first, then AOB table.

  12. Thanks for the tip (and actual name) for the Async Service. Found that mine had stopped unexpectedly. I will pursue the suggestions in this post.

    As for deleted records not disappearing...

    After our upgrade to CRM 4.0 we ran into an issue with records not being deleted from CRM for many days. While I was at Convergence I visited the help desk and inquired about the deletion service.

    I learned that the deletion service in CRM 4.0 is set, by default, to run once per day with the time defaulted to when CRM was first installed.

    They also told me about a utility that allows you to change the run time and frequency. These changes cannot be made without the use of this utility.

    After some testing in DEV we rolled this utility into Production. When we deployed to Production we discovered that the default settings were to run once per WEEK.

    Microsoft refers to this utility as the “Scale Group Job Editor”.

    You can download the tool from the following location:

    http://code.msdn.microsoft.com/ScaleGroupJobEditor/Release/ProjectReleases.aspx?ReleaseId=3251

    Good luck.

    Bob

    1. Thanks for sharing Bob.

      Yes - the Scale Group Job Editor is a useful utility. There are a couple items to keep in mind too. You can kickoff a 'hard delete' right away by schedule the 'next run time' for a date/time in the past. It will then run right away. Also, one advantage of only running the hard delete jobs weekly or monthly is that you then have an easy way to 'undelete' data without having to do a database restore.

  13. Hi Alexf,

    I was looking into ways to "undelete" a few accounts that were accidently removed. Is there a tool I can use to easily recover them?

    Thank you,
    Mike

    1. Hi Mike,

      I'm not aware of any tools with a GUI that do this. However, when items are deleted in crm a 'flag' is set behind the scenes. If the async hard delete job has not run yet, you can 'undelete' by setting the deletionstatus=2 (marked for deletion) back to deletionstatus=0.

PowerObjects Recommends