As with any new, innovative feature in a product, it is one thing to say it is helpful for customers – it is quite another to see it in action in the wild. Case in point, I had a great discussion with a customer about using LogReduce™ in their environment. LogReduce is a groundbreaking tool for uncovering the unknown in machine data, and sifting through the inevitable noise in the sea of log data our customers put in Sumo Logic. The customer in question had some great use cases for LogReduce that I would like to share.
Daily Summaries
With massive amounts of log data flowing through modern data centers, it is very difficult to get a bird’s eye view of what is happening. More importantly, the kind of summary that provides actionable data about the day’s events is elusive at best. In our customer example, they have been using LogReduce to provide exactly that type of daily, high-level overview of the previous day’s log data. How does it work? Instead of using obvious characteristics to group log data like the source (e.g. Window’s Events) or host (e.g. server01 in data center A), LogReduce uses “fuzzy logic” to look for patterns across all of your machine data at once – letting the data itself dictate the summary. Log data with the same patterns, or signatures, are grouped together – meaning that new patterns in the data will immediately stand out, and the noise will be condensed to a manageable level.
Our customer is also able to supply context to the LogReduce results – adjusting and extending signatures, and adjusting relevance as necessary. In particular, by adjusting the signatures that LogReduce finds, the customer is to “teach” LogReduce to provide the best results in the most relevant way. This allows them to separate the critical errors out, while still acknowledging the background noise of known messages. The end-result is a daily summary that is both more relevant because of the user-supplied, business context as well as being flexible enough to find important, new patterns.
Discovering the Unknown
And finding those new patterns is the essential essence of Big Data analytics. A machine-data analytics tool should be able to find unknown patterns, not simply reinforce the well-known ones. In this use case, our customer already has alerting established for known, critical errors. The LogReduce summary provides a way to identify, and proactively address, new, unknown errors. In particular, by using LogReduce’s baseline and compare functionality, Sumo Logic customers can establish a known state for log data and then easily identify anomalies by comparing the current state to the known, baselined state.
In summary, LogReduce provides the essence of Big Machine Data analytics to our customers – reducing the the constant noise of today’s datacenter, while finding those needles in the proverbial haystack. This is good news for customers who want to leverage the true value of their machine data without the huge investments in the time and expertise required in the past.