Data Model in Splunk

In this blog we are going to understand what is a data model in splunk and a overview how to create a data model let’s dive in to the topic.

What is a data model in splunk?

A data model in splunk is a hierarchically structured mapping of the time needed to search for semantic knowledge on one or more datasets. It encodes the knowledge of the necessary field.

• Encapsulate the knowledge needed to build a search
• Pivot reports are build on top of data models
• Data-independent

What are datasets?

Data model in splunk, data sets are typically categorically classified into parent and child data sets. This order makes it easier for users to search specific parts of a dataset.

What is a pivot?

Splunk uses a data model and valid searches to generate a user’s pivot report. A pivot report is a visualization, table, or graph that displays information from a search. Pivot is also the name of the tool that Splunk uses to generate pivot reports.

Types of objects in data model?

Data model in splunk consist of three types of objects:

• Events : Event datasets are broadly a type of event.
• Searches : The search dataset uses a Splunk search to define the dataset it represents.
• Transactions : Transaction dataset definitions use fields that have already been added to the model via event or search datasets. That is, you cannot create a data model that consists solely of transactional datasets and their child datasets. The event or search dataset tree must already exist in the model before you can create the transaction dataset.

Constraints in data model?

All data model datasets are defined by a set of constraints. Dataset constraints exclude events that are not related to the dataset.

Event datasets or child datasets of any type, the limit looks like a simple search without additional pipes and search commands.

Search datasets, the limit is the dataset search string.

Transaction datasets, the constraint is a transaction definition. The transaction dataset definition must identify the group dataset and one or more GroupBy fields.

Attributes & Field types in data model?

Attributes are the fields you want to include in the objects. Like constraints, attributes are inherited from parent objects.

• Auto-extracted: Fields extracted by Splunk software during indexing or searching. Auto-extracted fields can only be added to the route dataset. Child datasets can inherit them, but they cannot add their own new auto-extract fields. The automatically extracted fields are divided into three groups.
• Eval Expression: A field derived from the summary expression you enter in the field definition. In many cases, the evaluation expression contains one or more extract fields.
• Lookup: A field that is added to an event in the dataset using a search configured in the field definition. Lookup adds fields from external data sources such as CSV files and scripts. Once you have defined a lookup field, you can use any lookup object in your system to link to other fields that are already linked to the same record.
• Regular Expression: A field that is added to an event in the dataset using a search configured in the field definition. Lookup adds fields from external data sources such as CSV files and scripts. Once you have defined a lookup field, you can use any lookup object in your system to link to other fields that are already linked to the same record.
• Geo IP: A specific type of lookup that adds a geographic field such as latitude, longitude, country, or city to an event in a dataset that has a valid IP address field. Useful for map-related visualizations.

Attributes types in a data model?

• String: Field values are recognized as alpha-numeric.
• Number: Field values are recognized as numeric.
• Boolean: Field values are recognized as true/false or 1/0.
• IPV4: Field values are recognized as IP Addresses.

Attributes tags in a data model?

• Required: Only events that contain this field are returned in pivot.
• Optional: This field doesn’t have to appear in every event.
• Hidden: This field is not displayed to pivot users when they select the object in pivot – Use for fields that are only being used to define another attribute, such as an eval expression.
• Hidden & Required: Only events that contain this field are returned, and the fields are hidden from use in pivot.

Creating a data model in splunk

Overview for creating a data model:

Adding a Parent & Child: Child events inherit all attributes from the parent events (you can add more attributes to child events)

Adding Transaction: You can add a transaction to the data model. The transaction object below would equate to the search:

sourcetype=access* | transaction clientip maxpause=10s

Example:

Creating a Pivot:

Accelerating a data model:

• After you enable acceleration for a data model, pivots, reports, and dashboard panels that use that data model can return results faster than they did before.
• After the data summary is built, searches that use accelerated data model datasets run against the summary rather than the full array of _raw data.

1. Open Data Model > Select Edit Acceleration.
2. Click Add for ACCELERATION.
3. Select Edit > Edit Acceleration.
4. Select Accelerate to enable acceleration for the data model.

• After your data model is accelerated, the The data model acceleration icon ⚡ for the model on the Data Models management page is yellow instead of gray.

Searching a data model

using datamodel command:

using tstats command: tstats can basically accesses and searches on these special, DM-created tsidx files. You tell tstats which DM to use with the from datamodel=<datamodel_name> clause.

Downloading a data model

If you are still facing issue regarding data model in splunk Feel free to Ask Doubts in the Comment Box Below and Don’t Forget to Follow us on social platforms, happy Splunking >.