> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sazabi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Buildkite

> Turn on OpenTelemetry job log export on your self-hosted Buildkite agents to stream every job log line to Sazabi.

[All Buildkite integration capabilities](https://docs.sazabi.com/catalogs/integrations/buildkite)

## About

From [v3.135.0](https://github.com/buildkite/agent/releases/tag/v3.135.0), the Buildkite agent can [export job output as OpenTelemetry log records](https://buildkite.com/docs/agent/self-hosted/monitoring-and-observability/tracing#exporting-job-logs-as-opentelemetry-logs). Turn that on with the agent's `job-logs-otlp` setting and point its OTLP/HTTP log exporter at your Sazabi intake URL. Each line of job output then reaches Sazabi as one record, with the same `[REDACTED]` masking as the Buildkite job log. The job log in the Buildkite dashboard is unchanged, and no collector, Buildkite API token, or pipeline change is needed.

Records carry `buildkite.organization.slug`, `buildkite.pipeline.slug`, `buildkite.branch`, `buildkite.build.number`, `buildkite.build.id`, `buildkite.job.id`, `buildkite.job.label`, `buildkite.job.key`, `buildkite.queue`, and the agent name and ID. The resource `service.name` is `buildkite-agent`.

> [!NOTE]
>
> Job log export is an agent operator setting: a pipeline can't turn it on, and only self-hosted agents are covered. Buildkite documents OTLP export from [hosted agents](https://buildkite.com/docs/pipelines/integrations/observability/opentelemetry#opentelemetry-tracing-from-buildkite-agent-buildkite-hosted-agents) for traces only, not job logs.

## Prerequisites

* Self-hosted Buildkite agents on v3.135.0 or later (every v4 release qualifies), installed from the Linux packages, the [Elastic CI Stack for AWS](https://buildkite.com/docs/agent/self-hosted/aws/elastic-ci-stack), the [Agent Stack for Kubernetes](https://buildkite.com/docs/agent/self-hosted/agent-stack-k8s), the [`buildkite/agent` Docker image](https://buildkite.com/docs/agent/self-hosted/install/docker), or any other way you start `buildkite-agent`
* Outbound HTTPS from the agents to your Sazabi intake host
* Your Sazabi intake URL, shown above — open the log source and copy the **Intake URL** from its setup screen (the public key is embedded in the URL)

## Set up in the dashboard

**Step 1**

### Turn on job log export

Every agent needs the same three settings. Replace `<your intake host>` with the intake URL shown above, without `https://`. No `Authorization` header is needed because the public key is in the hostname.

| Setting | Value |
| - | - |
| `BUILDKITE_JOB_LOGS_OTLP` (or `job-logs-otlp` in `buildkite-agent.cfg`) | `true` |
| `OTEL_EXPORTER_OTLP_LOGS_PROTOCOL` | `http/protobuf` (the agent defaults to gRPC) |
| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | `https://<your intake host>/v1/logs` |

The logs-specific endpoint is used as-is, so it must end in `/v1/logs`. Using the `OTEL_EXPORTER_OTLP_LOGS_*` variables leaves any tracing setup that uses `OTEL_EXPORTER_OTLP_ENDPOINT` untouched. The exporter variables can't go in `buildkite-agent.cfg`; they must be in the agent process's environment.

**Linux packages.** Add `job-logs-otlp=true` to the [agent config file](https://buildkite.com/docs/agent/self-hosted/configure) and the exporter variables to a [systemd drop-in](https://buildkite.com/docs/agent/self-hosted/install/ubuntu#systemd-modifications), then restart the agent.

```bash title="Linux" theme={"system"}
echo 'job-logs-otlp=true' | sudo tee -a /etc/buildkite-agent/buildkite-agent.cfg

sudo mkdir -p /etc/systemd/system/buildkite-agent.service.d
sudo tee /etc/systemd/system/buildkite-agent.service.d/sazabi.conf <<'CONF'
[Service]
Environment="OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=http/protobuf"
Environment="OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://<your intake host>/v1/logs"
CONF

sudo systemctl daemon-reload
sudo systemctl restart buildkite-agent
```

**Elastic CI Stack for AWS.** Save this environment file in a private S3 bucket and set the stack's [`AgentEnvFileUrl`](https://buildkite.com/docs/agent/self-hosted/aws/elastic-ci-stack/ec2-linux-and-windows/managing-elastic-ci-stack#configuring-agent-environment-variables) parameter to its `s3://` URI. Grant the instances `s3:GetObject` on it through `ManagedPolicyARNs`, then update the stack.

```bash title="agent.env" theme={"system"}
BUILDKITE_JOB_LOGS_OTLP="true"
OTEL_EXPORTER_OTLP_LOGS_PROTOCOL="http/protobuf"
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT="https://<your intake host>/v1/logs"
```

**Agent Stack for Kubernetes.** The controller adds an `agent` container to every job pod and applies [`pod-spec-patch`](https://buildkite.com/docs/agent/self-hosted/agent-stack-k8s/podspec) to it. Merge this into your Helm values and run `helm upgrade`. If you already have a `pod-spec-patch`, add the `agent` container entry to it.

```yaml title="values.yaml" theme={"system"}
config:
  pod-spec-patch:
    containers:
      - name: agent
        env:
          - name: BUILDKITE_JOB_LOGS_OTLP
            value: "true"
          - name: OTEL_EXPORTER_OTLP_LOGS_PROTOCOL
            value: http/protobuf
          - name: OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
            value: "https://<your intake host>/v1/logs"
```

**Docker.** Add the three `-e` flags to your existing `docker run` command (or the `environment` of your Compose service) and recreate the container.

```bash title="Docker" theme={"system"}
docker run -d -t --name buildkite-agent \
  -e BUILDKITE_JOB_LOGS_OTLP=true \
  -e OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=http/protobuf \
  -e OTEL_EXPORTER_OTLP_LOGS_ENDPOINT="https://<your intake host>/v1/logs" \
  buildkite/agent:4 start --token "<your-agent-token>"
```

**Anything else.** Export the three variables in the environment of the process that runs `buildkite-agent start`.

**Step 2**

### Restart the agents

The agent reads these settings when it starts. Restart every agent, or roll the stack or Helm release. Jobs that start afterwards export their logs; jobs already running don't.

**Step 3**

### Hand off the change

If your agent configuration is version-controlled, commit it on a new branch, push it, and open a pull request. Otherwise apply it directly. Keep the intake URL in a secret store where your tooling supports it, such as an S3 object with restricted access or a Kubernetes Secret.

## Set up with the CLI

You can also register the Buildkite log source with the [Sazabi CLI](https://docs.sazabi.com/reference/cli.md) (installed and authenticated — see [CLI reference](https://docs.sazabi.com/reference/cli/log-sources.md)).

Registering the source mints the same intake URL the dashboard shows above (the public key is embedded in its hostname):

```bash theme={"system"}
sazabi log-sources create buildkite --mode manual
```

The command prints the intake URL for the new source — copy the whole URL and use its host in the configuration shown above. Run `sazabi log-sources get <log-source-id>` at any time to reprint the endpoint, or `sazabi log-sources list` to see every log source in the project.

## Verify

Run any build on a queue served by an updated agent. Open the [Intake page](https://docs.sazabi.com/guides/instrument/intake.md) in the Sazabi dashboard and confirm records from the **Buildkite** source appear within a minute or two, then search for the pipeline's `buildkite.pipeline.slug` or a `buildkite.job.id`.

## Troubleshooting

**No records after restarting** — Confirm the agent is v3.135.0 or later and was restarted after the change. If the exporter can't start, the job log shows `Failed to initialize OTLP job log exporter`.

**Still nothing** — Check that `OTEL_EXPORTER_OTLP_LOGS_PROTOCOL` is `http/protobuf`; without it the agent uses gRPC, which the intake doesn't accept. Check that the logs endpoint ends in `/v1/logs`.

**Kubernetes jobs don't export** — The patch must target the container named `agent`. Inspect a job pod with `kubectl get pod <job-pod> -o yaml` and confirm the three variables are on that container.

**Sensitive output** — Redaction only masks values the agent recognizes as secrets (redacted environment variables and secrets registered through the Job API). Anything else a build prints is sent as-is.

## Further reading

> [card] [Intake](https://docs.sazabi.com/guides/instrument/intake.md)
>
> See per-source ingestion volume and health.

> [card] [Log sources](https://docs.sazabi.com/catalogs/log-sources.md)
>
> Browse the full log source catalog.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.