Network operation engineers must deal with log messages generated by multiple devices and technologies. The standard procedure used so far is based on regular expressions (Regex or Grok or similar) that are used to differentiate various types of logs and classify them accordingly. This has always been an arduous process for several reasons. First, building those expressions can be complex. Second, they need to be supported and evolved which can be challenging since the engineers that created these expressions, may or may not have prepared the required documentation for new teams to reference and revise.
Imagine if there was a process in place that would accomplish the same goal. For instance, rather than explicitly defining a parsing expression, a more simplified approach would be to develop a process that could learn based on the structure of the logs themselves to determine which parts of the log are relevant and which ones are not. Therefore, once you have extracted the relevant parts, the next logical step would be to find the common patterns across the different logs and group them so similar logs are assigned a common category. Last, imagine if this process requires minimal action from the user other than giving this cluster a name. This would be the icing on the cake.
At Selector, we believe that IT teams can have their cake and eat it too. The Selector platform makes this process a reality to provide a more simplified, intuitive process that is effective and efficient. The platform can automatically normalize logs and cluster them by detecting the common patterns in their structure. What’s more, it is uniquely designed with different Natural Language Processing techniques involved. The process is entirely automated which means no manual inputs are required. Logs are automatically classified based on the existing patterns and clusters, as well as the ability to detect any new logs.
Once logs are classified into their cluster, key analytics and insights will be generated so that the Selector platform can surface multi-dimensional anomalies which can be very difficult to identify.
Most network engineers still use Regular Expressions which is commonplace, but this can be costly to maintain. The main objective of machine learning techniques is to eliminate any guesswork and simplify the mundane and manual tasks that network engineers face daily. Furthermore, machine learning allows the network engineer to focus on the outcomes, analysis of the logs, and the patterns detected to help identify any underlying issues in the infrastructure, rather than spending time working on the regular expressions that would be needed to enable this type of analysis. With the Selector platform, this process is automated allowing the network engineers to focus on what matters most: reducing the Mean Time to Detect (MTTD) and the Mean Time to Repair (MTTR).
Log messages are a key source of data for a network operation. They have been traditionally created for human consumption (human readability) and are not designed to be ‘machine friendly.’ However, the volume of log messages and their complexity don’t allow for easy human analysis. The Selector platform provides machine learning capabilities that bring back the logs to a ‘ready’ state where users can understand the meaning based on the what, the when, and the how many.
Let’s explore some of the benefits of Selector’s Analytics platform:
First, operations teams are no longer required to create and maintain complex and numerous regular expressions to parse and classify logs.
Second, by performing multi-dimensional anomaly detection, the Selector platform helps to identify unseen anomalies from the logs allowing the network operation engineers to quickly identify issues otherwise impossible to detect.
Below are several screenshots of this unique feature:
Interested in learning more about this feature? Contact us today for a free demo!