Add a Cribl Search Pack and Create a Lake Dataset with the Cribl SDK
Preview Feature
The Cribl SDKs are Preview features that are still being developed. We do not recommend using them in a production environment, because the features might not be fully tested or optimized for performance, and related documentation could be incomplete.
Please continue to submit feedback through normal Cribl support channels, but assistance might be limited while the features remain in Preview.
This code example demonstrates how to use the Python SDK for the control plane to:
Add the Search Pack named Cribl Search AWS VPC Flow Logs from the Cribl Packs Dispensary.
Create a Lake Dataset with a basic configuration: Dataset ID, Cribl Lake storage location, retention period of 30 days, and Direct Access (HTTP) disabled.
This example includes the Pack URL for Cribl Search AWS VPC Flow Logs, which is used as the value of the source parameter. To get the URL for a different Pack, see Get the URL for a Search Pack.
About the Code Examples
The code examples use Bearer token authentication. Read the SDK authentication documentation to learn how to configure authentication. The Permissions granted to your Bearer token must include creating and managing Packs and Lake Datasets.
Replace the variables in the examples with the corresponding information for your Cribl deployment.
The configurations in the examples do not include all available body parameters. For a complete list of body parameters for each endpoint, refer to the documentation in the API Reference.
Cribl Search and Cribl Lake are available only on Cribl.Cloud, so this page does not include examples for on-prem deployments.
"""
Replace the placeholder values for ORG_ID, CLIENT_ID, CLIENT_SECRET, and
WORKSPACE_NAME with your Organization ID, Client ID and Secret, and
Workspace name. To get your CLIENT_ID and CLIENT_SECRET values, follow the steps
at https://docs.cribl.io/cribl-as-code/sdks-auth/#sdks-auth-cloud.
This example demonstrates how to install a Search Pack and create a Lake Dataset
using the Control Plane SDK with Cribl.Cloud authentication.
The example:
1. Installs the AWS VPC Flow Logs Search Pack from Cribl Packs Dispensary
2. Creates a Lake Dataset with basic configuration
Prerequisites: Replace the placeholder values for ORG_ID, CLIENT_ID,
CLIENT_SECRET, and WORKSPACE_NAME with your Organization ID, Client ID and
Secret, and Workspace name. To get your CLIENT_ID and CLIENT_SECRET values,
follow the steps at https://docs.cribl.io/api/#criblcloud.
Your Client ID and Secret are sensitive information and should be kept private.
NOTE: This example is for Cribl.Cloud deployments only.
"""
import asyncio
from cribl_control_plane import CriblControlPlane
from cribl_control_plane.models import Security, SchemeClientOauth
# Cribl.Cloud configuration: Replace the placeholder values
ORG_ID = "your-org-id" # Replace with your Organization ID
CLIENT_ID = "your-client-id" # Replace with your OAuth2 Client ID
CLIENT_SECRET = "your-client-secret" # Replace with your OAuth2 Client Secret
WORKSPACE_NAME = "your-workspace-name" # Replace with your Workspace name
# AWS VPC Flow Logs Search Pack from Cribl Packs Dispensary
PACK_URL = "https://packs.cribl.io/dl/cribl-search-aws-vpc-flow-logs/0.1.1/cribl-search-aws-vpc-flow-logs-0.1.1.crbl"
PACK_ID = "cribl-search-aws-vpc-flow-logs"
LAKE_ID = "default"
DATASET_ID = "aws-vpc-flow-logs-dataset"
base_url = f"https://{WORKSPACE_NAME}-{ORG_ID}.cribl.cloud/api/v1"
# Create authenticated SDK client
async def main():
client_oauth = SchemeClientOauth(
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
token_url="https://login.cribl.cloud/oauth/token",
audience="https://api.cribl.cloud",
)
security = Security(client_oauth=client_oauth)
cribl = CriblControlPlane(server_url=base_url, security=security)
# Construct URLs for pack installation
search_group_url = f"{base_url}/m/default_search"
# Install AWS VPC Flow Logs Search Pack
cribl.packs.install(
request={
"source": PACK_URL,
"id": PACK_ID,
},
server_url=search_group_url,
)
print(f"✅ Installed Search Pack {PACK_ID} from Cribl Packs Dispensary")
# Create lake dataset
cribl.lakes.datasets.create(
lake_id=LAKE_ID,
id=DATASET_ID,
retention_period_in_days=30,
http_da_used=False,
storage_location_id="cribl_lake",
)
print(f"✅ Created Lake Dataset: {DATASET_ID}")
if __name__ == "__main__":
try:
asyncio.run(main())
except Exception as error:
print(f"❌ Something went wrong: {error}")
Get the URL for a Search Pack
To add a Search Pack from the Cribl Packs Dispensary or the Dispensary GitHub Repository, provide the Pack URL as the value for the source parameter in your request. The URL must be the direct URL location of the .crbl file for the Pack.
Read Pack Repositories to learn about the differences between the Cribl Packs Dispensary and the Dispensary GitHub Repository.
Here’s how to get a Pack’s URL from the Cribl Packs Dispensary:
Search the Cribl Packs Dispensary to find the Pack that you want to add. Select the Pack’s tile to open the Pack sidebar.
Open the developer tools for your browser and select the Network tab.
At the upper-right of the Pack sidebar, select the download icon.
In your browser’s developer tools, in the Name column, select the entry for the Pack’s
.crblfile and the Headers tab. Copy the Request URL: the copied URL is the value to provide for thesourceparameter in your request.
To get a Pack’s URL from the Dispensary GitHub Repository:
Search the Dispensary GitHub Repository to find the repository for the Pack that you want to add. Select the Pack’s repository to open it.
In the right sidebar, select Releases.
On the Releases page, find the release that you want to use and expand the Assets section.
In the Assets section, find the
.crblfile for the Pack. Right-click the.crblfile and select the option to copy the link. The copied link URL is the value to provide for thesourceparameter in your request.
.crbl File Link to Use as the Pack URL