duplicates dataset

This dataset holds data that was rejected as a duplicate during dispatch.

You can use duplicates as dataset type, in which case it defaults to simple (see simple dataset format). Otherwise any other dataset format will do: other useful options can be outbound dataset and discard dataset.

Example configuration

This could be an example of a standard setup:

[duplicates]
type = duplicates
step = daily
delete age = 3

This will just store the data, but won’t allow to query it or automatically delete it, and will require inspecting it manually:

[duplicates]
type = outbound
step = daily

This will throw away all data that has been interpreted as duplicates. You may want to use a simple type first, to inspect duplicates for some time to make sure your filter and unique rules in other datasets are correct, and only then switch to discard:

[duplicates]
type = discard