By Mike Zakrzewski
Director, Cyber Technologies
Elastic and Splunk — competitors, right? Sure. But did you know these tools can actually complement each other? Open-source Elastic can substantially reduce your ever-growing Splunk costs, increase performance, and boost data retention capabilities.
Cost – To understand how Elastic provides cost-savings, it is important to understand some pricing basics. Splunk’s cost is based on the amount of data ingested. This can become cost–prohibitive if your organization brings in lots of data. However, Elastic has a mix of free and paid-for services. For example, some basic Elastic security services are free, but additions like document level access control is a paid add-on. However, you don’t pay for data you ingest, allowing for huge cost savings as data increases. This is especially important for large organizations.
Performance – When data is ingested, Elastic separates and parses it into fields, turning it into structured (JSON) data, which is quicker to read than unstructured. Data like this is easy to find with what is known as a query on write. Splunk, on the other hand, brings data in as unstructured and uses schema on read, applying the schema as the data is queried. Users pay a performance tax with Splunk because every time you run a query, the schema must be applied again before it can be read.
Data Retention – When it comes to querying large datasets, Elastic makes it feasible to mine months or years of data because of schema on read. At ECS, we optimize Splunk by providing Splunk only with critical events you know you want to explore with Splunk. Elastic can handle the rest of the events, such as datasets you don’t know are important… until they are. This reduces the amount of data indexing and lessens data in Splunk to quicken searches while reducing cost.
The flowchart below demonstrates the process of using Elastic as a cost-savings tool alongside Splunk:
Use Elasticsearch to ingest large security datasets, such as authentication logs, audit events, NetFlow records and DNS traffic.
Use Elasticsearch to remove the ‘noise’ by deduplicating and filtering to only significant events.
Index the condensed/highly relevant dataset into Splunk.
Save money and increase performance of your Splunk investment.