AnsweredAssumed Answered

How to prevent duplicates using the decision shape?

Question asked by johannes.mweli478832 on Oct 22, 2018
Latest reply on Oct 24, 2018 by britto.parthalomew

Dear Boomi Developers,


I have the following use case:


I get data from ServiceNow (SN) through a connector call and I cache it.

I then have another connector call which gets me data from Quickbase (QB). Basically I end up with 2 profiles - one from SN and another from QB. What I would like to do is write the QB record into SN only if that record doesn't exist in SN. In other words I am trying to prevent duplicates. I am using a decision shape to take a record from QB and compare it to record in SN and if there is no match (meaning record does not exist in SN) then I write it out to SN. The problem I am running into though is that each comparison of QB profile record and SN profile record results in a record being written out and this causes each record to be written out more than once.


For instance I have 10 records from QB profile which I want to send to SN if they don’t exist in SN. In SN profile I have 3 records which I check to make sure that they don’t match the QB ones otherwise there will be duplicates. The desired functionality is that each record I have in QB I checked it against the 3 records in SN and if it does not match all 3 records then I write it out. However if it matches any of the 3 records then I don’t write it out instead I read the next one from QB and do the same comparison with it. The problem is that each record from the QB profile ends up getting written out 3 times i.e. first record on QB is compared against first record in SN and if there is not match it gets written out and it then gets compared against the 2nd record in SN and again if there is no match it gets written out, then it gets compared against the 3rd record in SN and if there is no match the record gets written out again for the 3rd time. Therefore each of my 10 records that I have in QB end up getting written 3 times and in SN which means I end up with 30 records instead of 10 with each record repeated 3 times.


A classic example just to further illustrate the issue that I am running into here is something similar to read loops in a java / python programming language where you have the outer read loop (main one). For each record you read in the main read loop you have an inner read where you make the comparison to check if there is a match. In the inner read loop you read all the records to check if there is a match and if there is n’t you write out the record in the outer read loop. However if there is a match in the inner read loop you break out of that loop and don’t write out the main record that was matched, instead you go read the next one in the main read loop and do the same comparison on it. This is exactly the kind of processing that I am trying to achieve here.


Would you guys please advise as to what I can do to overcome this issue in my decision shape.