On This Page

Home / Search/ Optimize/Dynamic Search Concurrency

Dynamic Search Concurrency

Automatically scale the allowed number of concurrent searches to use available capacity without overloading Cribl Search.


Autoscale the Number of Concurrent Searches Allowed

Cribl Search runs each query against available compute capacity, which limits how many searches can run at the same time. By default, the maximum number of concurrent searches is a static cap set by the Overall concurrent search limit on the system Usage Group. Admins typically set this cap conservatively to protect Cribl Search from worst-case workloads, which may leave compute capacity unused.

With dynamic search concurrency, Cribl Search continually measures its own CPU and memory load and adjusts the cap up or down to keep configurable headroom free. Throughput is higher when Cribl Search is idle, back-off is faster when it’s under pressure, and you spend less time tuning by hand.

While dynamic search concurrency is on, it overrides the Overall concurrent search limit on the system Usage Group, and that field becomes read-only in the UI. Per-group concurrent limits on other Usage Groups still apply.

Before enabling this feature, contact your Cribl account team. They can help you choose the best configuration for your workload.

Enable Dynamic Search Concurrency

You need to be a Search Admin or higher.

  1. From Search Home, go to Settings > Limits.
  2. Expand the Dynamic Concurrency section.
  3. Turn on Enable dynamic concurrency.
  4. Adjust the settings listed below to match your needs, then save.

Minimum Concurrent Searches

The lowest cap Cribl Search will use, even under sustained pressure. Increase this if you need to guarantee a baseline level of search concurrency. Defaults to 2.

Maximum Concurrent Searches

The highest cap Cribl Search will use, even when there’s plenty of headroom. Acts as a hard ceiling that protects against runaway concurrency. The percentage is relative to CPU count, so on a 16-CPU system 400% means a ceiling of 64 concurrent searches.

  • Default: 400%.
  • Maximum: 1000%.

Initial Concurrent Searches

The cap Cribl Search uses immediately after you enable the feature or after a restart, until it has gathered enough measurements to make the first adjustment. Tune this to gradually ramp up concurrency without sudden jumps or drops. Defaults to 100%.

Target CPU Headroom

The fraction of total CPU capacity Cribl Search aims to keep idle. For example, 0.20 means Cribl Search tries to keep at least 20% of CPU free. Lower the value to push harder, raise it to be more conservative.

  • Accepts: number between 0 and 1.
  • Default: 0.20.

Target Memory Headroom

The fraction of total memory Cribl Search aims to keep idle.

  • Accepts: number between 0 and 1.
  • Default: 0.10.

Sample Interval (Seconds)

How often Cribl Search measures its CPU and memory load.

  • Accepts: integer of 1 or higher.
  • Default: 1.

Window (Seconds)

The length of the rolling window Cribl Search uses to summarize measurements before each adjustment. Longer windows smooth out brief spikes and dips, at the cost of slower reaction time.

  • Accepts: integer of 1 or higher.
  • Default: 60.

Shrink Interval (Seconds)

How long sustained pressure must persist before Cribl Search lowers the cap. Use a longer interval to make decreases more conservative.

  • Accepts: integer of 1 or higher.
  • Default: 5.

Grow Interval (Seconds)

How long sustained headroom must persist before Cribl Search raises the cap. Typically longer than the shrink interval, so that Cribl Search backs off quickly under pressure but expands cautiously.

  • Accepts: integer of 1 or higher.
  • Default: 10.

Current Limit

A read-only display of the cap Cribl Search is currently enforcing. Use it to confirm the feature is responding to load.

See Also

  • Limits describes general system limits that apply alongside dynamic search concurrency.
  • Usage Groups lets you set per-group concurrent search limits and other usage limits.