Dedup Command in Splunk

Dedup command in splunk, Deletes events that contain the same combination of values in the specified field.

You can use the dedup command to specify the number of duplicate events to keep for each value in a single field or for each combination of values in multiple fields. The events returned by deduplication are based on search order. The history search first searches for recent events. Real-time search finds the first event received, but it is not always the latest event.

You can specify the number of events that have duplicate values or a combination of values to keep. You can sort the fields that determine which events to keep. Other options allow you to keep events where duplicate fields have been removed, or you can keep events where the specified field does not exist in the event.

Syntax:

dedup
[<int>]
<field-list>
[keepevents=<bool>]
[keepempty=<bool>]
[consecutive=<bool>]
[sortby <sort-by-clause>]

Query before using dedup command in splunk:

source="Superstore.csv" | table Region

Result before using dedup command in splunk:

Query after using dedup command:

source="Superstore.csv" | dedup Region | table Region

Result after using dedup command:

Explanation:

After using splunk dedup command we only get the identical values that is the repeated values are neglected

More Examples:

Retain first 3 results in duplicate:

For search results that have the same source value, preserve the first 3 that occur and delete all subsequent results.

Before using dedup:

After using dedup:

Also looking for more splunk commands ?

Click here

If you are still facing issue regarding dedup command in splunk Feel free to Ask Doubts in the Comment Box Below and Don’t Forget to Follow us on social platforms, happy Splunking >