On This Page

Home / Stream/ Secure Your Deployment/Centralize and Reuse Global Secrets

Centralize and Reuse Global Secrets ​

Use global secrets to define credentials once and reuse them across Cribl Stream and Edge by referencing them in Source, Destination, and Collector configurations. Teams that span products can use the same credentials instead of maintaining duplicate credentials that can drift out of sync. When you rotate credentials, you can update the values for global secrets in one place.

Cribl organizes global secrets in secret folders that are backed by secret stores. Worker Group and Edge Fleet access to global secrets is controlled at the folder level. Manage secret stores, secret folders, and global secrets in the UI under Global Settings > Security > Secrets or with the Cribl API.

Any legacy secrets that you use in existing resource configurations will continue to work as you migrate to global secrets.

Global secrets are not supported when GitOps is enabled in Cribl Stream, and you cannot enable GitOps on a Leader that already has global secrets configured. Cribl excludes global secret values from Git, so those values never reach a Git remote and cannot promote across environments. If GitOps is enabled, use legacy secrets instead.

A typical workflow for global secrets is:

  1. Connect an external provider to add a secret store, or use the built-in cribl secret store.
  2. Add a secret folder and make it accessible to Worker Groups or Edge Fleets. You can also clone an existing secret folder to create a new secret folder that uses the same secret store.
  3. Add global secrets to the secret folder.
  4. Reference global secrets in Cribl resources that support them.
  5. Commit and deploy your configuration.
  6. Rotate credentials as needed in the built-in cribl secret store or an externally backed secret store.

Deployment Scope ​

In Cribl.Cloud, global secrets are scoped to a single Workspace. Workspaces in the same Organization cannot access each other’s global secrets.

In on-prem deployments, global secrets are scoped to one Leader. Unrelated Leader installations cannot access global secrets configured on a different Leader.

The Leader persists global secrets configuration and resolves secret values. In Cribl.Cloud, this means the Leader for the Workspace. In on-prem deployments, this means the Leader Node. Cribl adds global secrets to deployment bundles for Worker Groups and Edge Fleets through the bundle push path. Cribl adds a global secret when its secret folder is accessible to the Group or Fleet, even if no resource configuration in the Group or Fleet references the secret.

Global secrets are available only in Distributed deployments. If you run Cribl Stream in single-instance mode, use legacy secrets instead.

Secret Stores ​

Secret stores are the backing stores for the secret folders that contain global secrets. You can use the built-in cribl secret store or connect an externally backed secret store.

The built-in cribl secret store cannot be deleted, and it persists encrypted global secret values on the Leader. To delete a secret store that is backed by an external provider, you must first delete all of its secret folders.

Cribl Stream uses cribl.secret as the primary encryption key to protect sensitive values in configuration files. It is created when the Leader starts and is stored by default at $CRIBL_HOME/local/cribl/auth/cribl.secret. Global secrets in the built-in cribl secret store are encrypted with this key.

When you configure an external KMS provider (HashiCorp Vault or AWS KMS) to manage cribl.secret, the key is removed from disk and managed by the KMS provider instead. This protection extends to global secrets in the built-in cribl secret store. The KMS provider configuration is separate from the global secret store. When using an external KMS, ensure that IAM permissions are correctly scoped (for example, kms:Decrypt for AWS KMS). Misconfigured permissions can prevent the Leader from starting.

Back up cribl.secret before major changes such as enabling an external KMS or rotating keys.

Secret Folders ​

Within secret stores, global secrets are organized into secret folders. Each secret folder can only belong to one secret store.

For secret folders backed by the built-in cribl secret store, Cribl persists encrypted secret values on the Leader in per-folder storage. For secret folders that are backed by external secret stores, Cribl stores metadata for folders and global secrets on the Leader and resolves secret values from the external provider at runtime.

Access Model ​

Global secrets are defined once on the Leader within the deployment scope. Cribl controls global secrets at the folder level, not per secret.

When you add a secret folder, the Accessible to setting determines which Worker Groups and Edge Fleets can use the folder’s global secrets in resource configurations. Adding a Worker Group or Edge Fleet to the folder’s Accessible to list makes every global secret in that folder available to use throughout the Group or Fleet. If you make a secret folder accessible to an Edge Fleet or Subfleet, its Subfleets inherit the ability to use the global secrets in the folder.

Deployment bundles include every global secret in a folder that is accessible to the Worker Group or Edge Fleet. Take care to make folders accessible only to the Groups and Fleets that should be able to use the secrets that the folders contain.

After a secret folder is accessible to a Worker Group or Edge Fleet, supported resources in that Group or Fleet expose references to the global secrets in the folder. Within one Workspace (Cribl.Cloud) or Leader (on-prem), any Worker Group or Edge Fleet that can use the folder can reuse the same global secret definitions.

To remove a Worker Group or Edge Fleet from a secret folder’s Accessible to list, first remove all references to the global secrets in the folder from resources in the Group or Fleet. This includes references used in Subfleets that will lose the ability to use those secrets if you remove the parent Fleet or Subfleet from the list.

Making a secret folder accessible to a Worker Group or Edge Fleet does not grant Member Permissions on the folder. Members with the Admin Permission on a higher-level object can view and manage the folder’s secrets. For other Members and Teams, an Admin must share the folder and assign the Shared Folder Secrets Viewer or Shared Folder Secrets Maintainer Permission.

Leader Restart and Externally Backed Secret Folders ​

When the Leader restarts, it reinitializes each secret store that is backed by an external provider and loads global secrets from the external provider. The Leader keeps externally backed secret values in memory only (they are not persisted on disk).

If credential values changed in the external store since the last commit and deploy, the Leader might resolve the updated values after it restarts. This does not distribute the new credentials to Worker Groups or Edge Fleets. Commit and deploy still controls when updated global secrets reach nodes. To rotate credentials through the normal workflow, see Rotate Global Secrets in Externally Backed Secret Stores.

If the Leader cannot reach the external provider when it restarts, store initialization or secret resolution can fail because it has not fetched and cached global secret values. Cribl surfaces the failure through secret store health checks as system bulletin messages. Deploy is blocked for Worker Groups and Edge Fleets with access to secret folders that are backed by the unhealthy store until the Leader successfully resolves those secrets. You can still commit configuration changes.