web
You’re offline. This is a read only version of the page.
close
Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Community site session details

Community site session details

Session Id :
Dynamics 365 Community / Forums / Finance | Project Operations, Human Resources, ... / Apache Kafka Integrati...
Finance | Project Operations, Human Resources, ...
Unanswered

Apache Kafka Integration with D365 FO using Logic Apps

(0) ShareShare
ReportReport
Posted on by Usama Tur Razzam 115

Hello Integration Champs,
We are integrating our POS sales with D365 FO. POS sales are syched at apache Kafka and we need to fetch sales data through Apache Kafka Rest proxy using logic app.

Which means is we will be using Logic Apps as a bridge between Kafka and ERP. Kafka returns data to logic app in form of json.

We are confused how we should pass the data to ERP? Either using custom web service where we pass same json to ERP? or we go for Composite entity approach in which case we would have to convert json to XML and then add needed tags in that for composite entity and then pass this xml using Batch Data API.

All the suggestions are welcome. Please keep in mind we have around 60k orders per day with 20-40 lines each order.

Categories:
Sales and Marketing
Development / Customization / SDK
  • Martin Dráb Profile Picture
    Martin Dráb 225,155 Moderator on at
    RE: Apache Kafka Integration with D365 FO using Logic Apps

    You don't have to use standard entities. If you want to import data to your custom tables, you can create data entities for them.

  • Usama Tur Razzam Profile Picture
    Usama Tur Razzam 115 on at
    RE: Apache Kafka Integration with D365 FO using Logic Apps

    Sorry for my ambiguous statement. I will try to rewrite again.

    As a business process we won't be creating one ERP Sales Order for one sales Order in our POS. This is the reason we wont be using sales order entity (this is what i meant). We have certain parameters of consolidation which will come in play by our batch job which will run on our sales data (received from logic app) residing in our staging tables.

    Our POS data is complex having multiple objects and multiple list of objects within one sales order so using OData wont be a good choice IMHO as for each object in header and objects in lists we will be separately calling simple data entities using Odata to store the data.

    That's why we are only left with 2 options either a composite entity or a custom service.

  • Martin Dráb Profile Picture
    Martin Dráb 225,155 Moderator on at
    RE: Apache Kafka Integration with D365 FO using Logic Apps

    I'm not 100% sure what you mean by " bout using sales orders entity, we can't as we have a complex BL [...]. So the data will go to our staging tables", but your explanation doesn't sound to me like anything that would prevent from using data entities. Correct me if I'm wrong.

    I cant tell you what exactly you should do, e.g. whether you should use composite entities. I don't know enough about your situation. But if you're worried about performance (which seemed to be the case), I would strongly consider data management APIs.

    By the way, your choice isn't just "one of DM APIs" vs. custom services. You could also use data entities through OData.

  • Usama Tur Razzam Profile Picture
    Usama Tur Razzam 115 on at
    RE: Apache Kafka Integration with D365 FO using Logic Apps

    Hi Martin,

    Your reply is much appreciated always.

    For each call to kafka we get list of orders which needs to go to different legal entities. Do you recommend concating them according to each legal entity and then converting them in xml and add missing tags for composite entities and then pass them on to respective legal entitites?

    For your point about using sales orders entity, we can't as we have a complex BL to consolidate them based on factors and then posting BOMs and posting sales. So the data will go to our staging tables.

    What would you suggest now? Batch Data API or a custom service?

  • Martin Dráb Profile Picture
    Martin Dráb 225,155 Moderator on at
    RE: Apache Kafka Integration with D365 FO using Logic Apps

    Whether you should use a data entity or a contract designed by yourself depends a lot on what data it is and what you want to do with it in F&O.

    If you're talking about sales orders, using a data entity makes a good sense to me. Using a custom service would mean manually designing contracts with all fields you need and either duplicating all the logic or actually using the data entities under the hood.

    Another benefit of using data entities is that you can utilize data management APIs to import data in bulk, which may be important in your case.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Dynamics 365 Community Update – Sep 16th

Welcome to the next edition of the Community Platform Update. This is a weekly…

Announcing Our 2024 Season 2 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

Dynamics 365 Community Newsletter - September 2024

Check out the latest community news

more Community News

Leaderboard > Finance | Project Operations, Human Resources, AX, GP, SL

Last month Overall leaderboard

Featured topics

Test Comment 26-09-2024 16:14pm

Product updates

Dynamics 365 release plans