Asynchronous Recurring Batch Jobs solutions for Online Dynamics CRM

Time to time we always face a challenge where we want to process batch job records in online Dynamics CRM due to any business requirement. As we know Dynamics 365 CRM Online doesn’t have the flexibility to schedule reoccurring jobs in terms of weekly, monthly or daily basis. One of the simple solution for this requirement is to create custom Console App or Windows service (implement your business logic there) and schedule it through task scheduler. 

However, there is a catch in this approach that you need an extra Azure or a standalone machine or other solution like Azure function etc.  to handle batch job scheduler. This might not be a financially viable option as for this task you need to buy a server and it also required maintenance in future.

In this blog I will try to show you multiple ways through which you can schedule a batch job logic by using Dynamics CRM ecosystem only. Based on your requirement and suitability you can opt any one of the below approaches. Basically, the main purpose of this blog is to show you how can we schedule a recurrence event through which we can trigger our business logic on daily basis.

1. Recurrence through Microsoft Flow
2. Recurrence through Bulk Delete Dynamics CRM System Job
3. Recurrence through Workflow Wait – Timeout (Duration Condition)

Create an action: For all the above-mentioned approaches as you know, we need to process CRM records in Bulk based on our business requirements, so what we do, we will create an action which will have one Input Parameter String (JsonRequest) and Two Output Parameter (JsonResponse and ErrorMessage) String.  And this Action further will call a Custom Workflow DLL (which will contain Batch job logic) as shown below. Let’s name the action as  new_Batch_Flow_Calling_Action

Below is the Custom Workflow activity Class code. I tried to keep it simple so that it can be extendable by anyone based on the business requirements.

using System;

using Microsoft.Xrm.Sdk;

using Microsoft.Xrm.Sdk.Workflow;

using System.Activities;

namespace NewLearing.Workflow

{

    public class Batch_FLOW_ACTION: CodeActivity

    {

        [RequiredArgument]

        [Input("JsonRequest")]

        public InArgument<string> JsonRequest { get; set; }

        [Output("JsonResponse")]

        [Default("")]

        public OutArgument<string> JsonResponse { get; set; }

        [Output("ErrorMessage")]

        [Default("")]

        public OutArgument<string> ErrorMessage { get; set; }

        protected override void Execute(CodeActivityContext context)

        {

            ITracingService tracingService = context.GetExtension<ITracingService>();

            try

            {

                tracingService.Trace($"Batch_FLOW_ACTION : Begin({DateTime.Now})");

                string errorMessage = string.Empty;

                var logicResponse = string.Empty;

                string jsonRequest = JsonRequest.Get(context);

                //empty json request: You can pass customize Json request based on your requirement. In this example we will pass empty string to just invoke the Action and DLL

                tracingService.Trace($"JSON Request: {jsonRequest}");  

                JsonResponse.Set(context, "");                  

                ErrorMessage.Set(context, "");

                // your logic to process bulk record or any other logic will be implemented under ProcessRecord method

                var response = ProcessRecord();

                // based on response you can set logicResponse json string

                logicResponse = "Batch Successful.";

                //process output from service or you can return customize JsonResponse

                if (!string.IsNullOrEmpty(logicResponse))

                {

                    tracingService.Trace($"LogicResponse: { logicResponse }");

                    JsonResponse.Set(context, logicResponse);

                }

                if (!string.IsNullOrEmpty(errorMessage))

                {

                    tracingService.Trace("Inside Error Message");

                    ErrorMessage.Set(context, errorMessage);

                }

                else

                {

                    tracingService.Trace("No Error in batch job execution");

                    ErrorMessage.Set(context, "");

                }

                tracingService.Trace($"Batch_Flow_Action: End({DateTime.Now})");

            }

            catch (Exception ex)

            {

                tracingService.Trace(ex.Message);

            }

        }

    }

}

1)  Recurrence through Microsoft Flow:  Earlier we used to call an Action through HTTP request, but now we can directly call an Action through the Power Automate (flow) steps. For that you need to follow below steps

• Go to > flow.microsoft.com > Solutions > Create New Solution
• Inside Solution Create a new Flow
• Now in Built-In Tab select Schedule > Triggers > Recurrence
• Now you can define interval frequency in which you wanted to run your batch job logic
• In second step you need to Search for Common Data Service Trigger point in our newly created Flow & from the search result select the Common Data Service (Current Environment)
• You will get many action points in the results from that we have to select Perform an Unbound Action.
• Here you need to select the above create Dynamics CRM Action (new_Batch_Flow_Calling_Action) and pass the Empty JsonRequest string. For your logic you can pass other input parameters in JsonRequest which you catch in Custom activity workflow code. The applications are infinite its up to you how you wanted to use JsonRequest string. In our case we are passing empty string to just invoke the action.

• After configuring the flow, you can check the traces of the action in Plugin-In Trace Log and Workflow execution log in System Jobs.

Disadvantage: One disadvantage that I encountered in this approach is that, as we know Action execution timeout is 2 minutes, so your bulk processing record should execute in 2 minutes. To overcome this approach, suppose you have 2000 records to process in a batch, so what you can do you can process top 500 records in one Action hit from flow recurrence job, at last you know you can pick the rest records in next recurrence hit. Problem solved!!

2)  Recurrence through Bulk Delete Dynamics CRM System Job: This method is one of the simplest way to get a trigger event. We know that in CRM under Data Management, we have Bulk Record Deletion functionality which manages bulk record-deletion jobs for custom and OOB entities in CRM. So, the idea is to leverage this functionality to get a recurrence event. To achieve this follow below  steps.

• Browse CRM > Create a Custom entity (for ex Job Trigger)
• Create one record in this Job Trigger entity.
• Then traverse to Settings > Data Management > Bulk Record Deletion
• Schedule Bulk Delete Job that deletes all records of Job Trigger entity type.
• Add a custom workflow for that entity which starts whenever a record is deleted from this entity. There you can call this custom Workflow activity class (NewLearning.Worklflow.Batch_FLOW_ACTION) which we have created above and pass the input Argument JsonRequest as per your business requirements. Rest bulk processing logic you can extend in this workflow activity class as per requirements.
• At the end of the workflow, create a new record of the same Job Trigger entity type, so that it will be picked in the next Bulk Delete Job run and the process can repeat. The agenda is always this custom entity will create one record.

3) Recurrence through Workflow Wait- Timeout (Duration Condition): This method is the traditional method to trigger a recurring event in CRM. As we know CRM platform offers Wait timeout condition which can wait for specific duration. I will not go in deep however you can follow below steps to achieve it.

• Create a Workflow (lets name it Recurring WF) on any new custom entity.
• Create a record in this entity Run this workflow in the background and As On-Demand Process and As a child Process.
• Then in steps you can call same action new_Batch_Flow_Calling_Action which we have created initially or you can call directly custom Workflow activity class (NewLearning.Worklflow.Batch_FLOW_ACTION) and process your batch job
• At the end of the workflow execution you need to call same Workflow as a child workflow as shown below
• At last from custom entity record you need to run Workflow on demand once, so that it can trigger the Workflow first time.