2

In openFDA's source code documentation, it's said that the FAERS pipeline includes a procedure for de-duping follow-up reports. I don't find in the source code where it happens.

What does it mean? Only the last follow-up of each case is kept or what?

1 Answer 1

1

This used to be handled within the source code. It is now handled by processing the quarterly reports sequentially and letting Elasticsearch handle the versioning. A basic last-one-wins approach. The openfda team added a feature to the pipelines for incremental indexing that allows this to happen without too much of a performance hit.

I will have the documentation updated to reflect this change. Thanks!

1
  • But nothing is done to detect and remove eventual duplicated cases with different case ID? (i.e., if two cases, with different case ID, represent the same "conceptual" case, they are not "merged"? )
    – Gabriele
    Jun 8, 2016 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.