The Regex Event Breaker is the default and most flexible Event Breaker type, allowing you to define event boundaries using regular expressions (regex). It gives you more granular control over how unstructured or non-standard logs are segmented into individual events.

Event Breaker patterns use the JavaScript (ECMAScript) regex engine. If you test patterns in external tools such as Regex101, select the ECMAScript (JavaScript) flavor. For more information, see Regex Flavor.

Use this Event Breaker for:

• Any log data that doesn’t fit the structured formats of the CSV, JSON Array, or File Header breakers.
• Multi-line logs where a single event spans multiple lines and non-standard single-line logs that use custom formats.

See Event Breakers for general information about event breakers.

Considerations ​

When using the Regex Event Breaker, keep this in mind:

• Breaks are continuous: The pattern applies continuously to the data stream. Any content before the match is considered part of the current event, and the break occurs at the start of the match.

• Consuming and retaining content: By default, the matched content of the Event Breaker regex is consumed (thrown away). To define the break point without discarding the content that signifies the new event (such as a timestamp), use a positive lookahead regex, such as (?=pattern). This is the standard practice for multiline logs.

• Avoid capturing groups: Do not use capturing groups (parentheses like (pattern)) inside the Event Breaker pattern. Using them will cause further, often unintended, splitting of the stream.

Settings ​

The Regex Event Breaker has two relevant settings:

• Event Breaker: The regular expression that identifies the start of a new event. Breaking occurs at the beginning of the matched pattern.

• Event byte limit: If the accumulated event size reaches this limit before the regex pattern matches, Cribl Stream splits the data into events of up to this size. Each full-size fragment has __isBroken set to true. If a leftover fragment is smaller than the limit, Cribl Stream emits it with __isBroken set to false.

See Event Breakers for information about additional settings that are available on all event breakers.

Configuration Example ​

The following is an example of multi-line log data input before the Regex Event Breaker processes it:

2020-05-19 16:32:12 moen3628 ipsum[5213]: Use the mobile TCP feed, then you can program the auxiliary card!
   Try to connect the FTP sensor, maybe it will connect the digital bus!
   Try to navigate the AGP panel, maybe it will quantify the mobile alarm!
2020-05-19 16:32:12 moen3628 ipsum[5213]: Use the mobile TCP feed, then you can program the auxiliary card!
   Try to connect the FTP sensor, maybe it will connect the digital bus!
   Try to navigate the AGP panel, maybe it will quantify the mobile alarm!

Example Settings ​

Output ​

From the example raw data, the Regex Event Breaker would generate two output events:

{
  "_raw": "2020-05-19 16:32:12 moen3628 ipsum[5213]: Use the mobile TCP feed, then you can program the auxiliary card! \n   Try to connect the FTP sensor, maybe it will connect the digital bus!\n   Try to navigate the AGP panel, maybe it will quantify the mobile alarm!",
  "_time": 1589920332
}
{
  "_raw": "2020-05-19 16:32:12 moen3628 ipsum[5213]: Use the mobile TCP feed, then you can program the auxiliary card!\n   Try to connect the FTP sensor, maybe it will connect the digital bus!\n   Try to navigate the AGP panel, maybe it will quantify the mobile alarm!",
  "_time": 1589920332
}