• Engineering

Building the Trestle Debugger for AI-Powered, Human-in-the-Loop Root Cause Identification

Sentry already gives us a strong signal when something goes wrong. It groups related events, captures stack traces, records tags, and gives engineers a place to begin. But in practice, the alert itself is rarely the whole story.

When an issue appears in Slack, the engineer still has to connect several pieces of context before deciding what to do next:

• Which service owns this issue? 
• Which repository should we inspect? 
• Is there a Helm chart tied to the service? 
• Is this user-impacting, or is it noisy observability? 
• Should this become a Jira ticket? 
• Is there enough confidence to open a pull request (PR)?

That work is repetitive, but valuable. Misunderstanding the context of an issue can send the team to the wrong repository or generate a fix that doesn’t match how the service actually runs. 

We didn’t want to automate that blindly. So we built Trestle Debugger: a Slack-triggered, human-in-the-loop Root Cause Analysis (RCA) workflow for Sentry issues. It uses Claude as a collaborator and stores the investigation context so that engineers can decide when to move from RCA to Jira or PR. We needed a workflow that could reduce the manual work while keeping engineers in control of every meaningful action.

The Problem We Were Solving

Our Sentry alerts already landed in Slack, which is where engineers discuss production issues. That part worked well. The gap was everything that happened after the alert.

Sentry can tell us what exception occurred, how often it occurred, which environment it came from, and what stack trace or tags were available. What it doesn’t know by default is how our internal engineering workflow is stitched together.

That missing operational context usually looks like this:

• Which GitHub repository owns the failing service.
• Which branch should be inspected for production RCA.
• Whether a Helm chart is relevant for the workload.
• Which Jira project should track the follow-up.
• Which Slack users or channels are allowed to trigger automation.
• Which repo-specific instructions should guide Claude.

Before we created the Trestle Debugger, engineers had to perform that mapping manually. For a straightforward issue that may have taken just a few minutes, but for a noisy production warning, confusing stack trace, or service with multiple deployment artifacts, the work became slower and more repetitive.

The goal was never to replace the engineer. We wanted to make the first investigation faster, more consistent, and easier to turn into a useful follow-up artifact.

Why We Didn’t Use Sentry’s AI

Sentry has useful issue context, and Trestle Debugger relies on that context. But what we needed was a system that could start from a Sentry alert, move through our internal service mapping, inspect the right repository context, and then preserve the result for Jira and PR workflows. The important part was giving the AI model the right operational context and making every step explicit.

The most useful workflow is closer to this:

Sentry issue → Slack thread → service mapping → repository context → RCA → Jira or PR

Slack is where the alert is discussed. The mapping layer tells us which codebase owns the issue. The repository gives Claude service-specific context and instructions. The workspace saves the investigation so later commands avoid rerunning the same work. Jira and GitHub turn the analysis into reviewable engineering artifacts.

This workflow is more specific to how Trestle engineers work, versus a generic issue summary.

Human-in-the-Loop by Design

We’ve established an AI philosophy at Trestle to help guide how we responsibly utilize AI in our work. One of the core principles of this philosophy states that AI is a collaborator, not an author. Engineers still own the output. A good AI workflow should remove repetition, surface context, and accelerate decision-making, but cannot remove accountability. You can read more about our AI Philosophy in this LinkedIn post by our Co-founder and CRO, Jordan Reynolds. 

That principle shaped the design of the Trestle Debugger. It doesn’t process every Sentry issue automatically. An engineer starts the workflow from Slack:

If the command is posted inside a Sentry alert thread, the bot reads the thread and extracts the Sentry URL. An engineer can also provide the URL directly:

That first command runs read-only RCA. It doesn’t edit code, create branches, deploy changes, run kubectl, or modify production.

Follow-up actions are intentionally separate:

This separation is crucial. Diagnosis, ticket creation, and code modification are different levels of action. Each should require a deliberate human decision.

PR creation is also gated by configuration. If PR_CREATION_ENABLED is not enabled, the bot will not create a PR. Even when it is enabled, the output is a pull request for review. The app does not merge or deploy the change.

Architecture

Trestle Debugger is a Java 21 Spring Boot service running in Kubernetes. It receives Slack Events API requests, validates them, fetches incident context, runs Claude locally in a controlled workspace, and writes results back to Slack, Jira, and GitHub.

At a high level, the flow starts with a Sentry alert in Slack. An engineer mentions the bot in that thread. Slack sends an app_mention event to POST /slack/events. The application validates the Slack request, checks authorization, fetches Sentry issue context, resolves the service mapping, creates a per-thread RCA workspace, runs Claude in read-only RCA mode, saves the full RCA, and posts a concise summary back to Slack. Later Slack commands can reuse that saved RCA to suggest a fix, create a Jira ticket, or create a PR.

The key architectural split is:

• Slack is the human trigger and collaboration surface.
• Spring Boot is the control plane.
• Sentry provides issue and event context.
• Repository mapping connects observability signals to code ownership.
• GitHub is the source and PR destination.
• Jira is the follow-up tracking system.
• Claude is the repo-aware RCA and optional fix engine.
• The request workspace is the durable handoff between commands.

Service Mapping: The Bridge From Alert to Ownership

The most important part of the workflow is mapping a Sentry issue to the correct engineering context.

Trestle Debugger keeps that mapping in application configuration under trestle.services. Each service entry can define the service name, GitHub repository, and optional Helm chart location. When a Sentry issue is loaded, Trestle Debugger checks tags such as service, app, and application, then falls back to the Sentry project slug.

That allows a Sentry project name to differ from the repository name. The alert doesn’t need to contain every internal detail. The mapping layer turns that alert into repository context.

This also lets us support services that do not have Helm configured. In that case, the workflow still performs repository-aware RCA and simply skips Helm context instead of failing.

The Per-Thread Workspace

Every accepted debug request gets a workspace scoped to the Slack channel and thread timestamp. The workspace contains the application checkout, optional Helm chart checkout, context files, the saved RCA, and any later Jira or PR metadata.

RCA_WORKSPACE_ROOT/
  requests/
    <channel-id>/
      <thread-ts>/
        app/
        helm-charts/
        context/
        rca.md
        saved-at.txt
        jira-issue-key.txt
        pr-url.txt

RCA_WORKSPACE_ROOT/
  requests/
    <channel-id>/
      <thread-ts>/
        app/
        helm-charts/
        context/
        rca.md
        saved-at.txt
        jira-issue-key.txt
        pr-url.txt

This workspace is what lets the workflow continue across multiple Slack commands.

During debug this, the service clones the application repository, optionally clones the Helm charts repository, writes Sentry and mapping context into context/, and runs Claude from the application checkout. The complete RCA is saved to rca.md.

Slack gets a concise response with the parts engineers need immediately:

• Summary
• Impact
• Root cause
• Confidence
• Relevant evidence
• Suggested next steps

The longer RCA remains available for follow-up workflows. When an engineer later runs suggest fix, the bot reads the saved RCA and formats remediation details. It doesn’t rerun Claude just because the engineer asked for a different presentation of the same investigation.

This was intentionally designed. Iteration matters, but repeated first-pass analysis is wasteful. Once the RCA is saved, the rest of the workflow should reuse it.

How Claude Is Used

Claude is used in two different modes:

Read-Only RCA

The first mode runs from the cloned application repository, receives Sentry context and repository mapping, reads repo-specific instructions when present, and blocks write or edit tools by default. The output is a full RCA artifact that is saved to rca.md.

Guarded PR Creation

The second mode only runs when an engineer explicitly asks for a pull request, and PR creation is enabled. In this mode, Claude is allowed to edit files, but the workflow still runs a configured validation command. If validation fails, Claude gets one repair attempt. The service then commits, pushes, and opens a GitHub PR for human review.

The two modes have different trust boundaries:

• RCA mode is read-only and advisory.
• Suggest-fix mode reformats the saved RCA.
• Jira mode creates or reuses a tracking issue.
• PR mode can edit code, but must pass validation, and still ends at human review.

The app stops at the PR. It does not merge or deploy.

This is the practical balance we wanted. AI helps remove manual investigation and boilerplate, but the review boundary stays where engineering teams already expect it.

From RCA to Jira

Once RCA is complete, an engineer can create a Jira ticket from the same Slack thread:

The Jira workflow reads rca.md, creates or reuses an issue, saves the Jira key back to the workspace, and links the Slack thread. It also attempts to assign the issue to the engineer who invoked the command by resolving Slack identity to Jira identity.

This avoids a common incident-follow-up problem: useful context exists in Slack, but the Jira ticket is created later with only a vague one-line summary. With Trestle Debugger, the RCA becomes the source material for the ticket.

The engineer still owns the ticket. The system just handles the repetitive conversion from investigation to tracking artifacts.

From RCA to Pull Request

The PR workflow is intentionally more guarded because it can modify code.

When an engineer runs:

The Trestle Debugger:

1. Reads the saved RCA.
2. Uses the configured source branch.
3. Creates a Trestle Debugger branch.
4. Runs Claude with edit tools enabled.
5. Runs the configured validation command.
6. Gives Claude one repair attempt if validation fails.
7. Commits and pushes the branch.
8. Opens a GitHub PR against the configured base branch.
9. Comments on Jira when a Jira issue is linked.
10. Posts the PR link back to Slack.

Failed PR attempts keep the workspace for debugging. Successful PR flows clean up the thread workspace after Slack and Jira have been updated.

Again, the workflow creates a reviewable artifact. It does not merge the PR or deploy the change.

Security and Guardrails

The value of this system depends on keeping it controlled.

Slack requests are validated using Slack’s signature headers and request timestamp. Old requests are rejected to reduce replay risk. The service can restrict access by Slack team, channel, user, and user group.

Credential exposure is treated as a workflow failure mode. Sensitive values are redacted from workflow logs where errors are handled.

Claude’s RCA mode is read-only. Write mode is separate and used only for PR creation. PR creation is disabled unless explicitly enabled. The application does not run Helm upgrades or perform Kubernetes mutations.

The guardrails are what make the workflow viable:

• Slack signing prevents unauthenticated event spoofing.
• Timestamp validation reduces replay risk.
• Channel and user checks keep the bot scoped to approved spaces.
• Read-only RCA limits the blast radius of investigation.
• Write-enabled PR mode requires an explicit command and still creates a reviewable artifact.
• Credential material is kept out of source code.
• Redaction keeps sensitive values out of logs and Slack error messages.

These boundaries reflect multiple tenets of our AI philosophy. We’re encouraged to experiment aggressively and to look for places where AI can remove repetitive work. But data boundaries are non-negotiable, and customer trust and production safety come first without exception.

What We Learned

AI Needs a Control Plane, Not Just a Prompt

The prompt is only one part of the system. The more important engineering work is the control plane around it: who can trigger the workflow, which context is allowed in, which tools are enabled, where artifacts are stored, and what action requires another explicit human command. Without that control plane, AI output becomes a one-off answer. With it, AI becomes part of a repeatable engineering workflow.

Context Selection Is the Real Product Surface

The quality of the RCA depends less on asking a clever question and more on selecting the right context. A Sentry issue alone can produce a generic explanation. A Sentry issue plus service mapping, repository files, local instructions, and optional deployment artifacts can produce an application-aware investigation. The mapping layer is therefore not just configuration. It is the product surface that decides what the AI is allowed to understand.

Saved Artifacts Matter

The first RCA should not be thrown away after a Slack response. Saving rca.md lets Jira and PR workflows reuse the same analysis. That makes follow-up commands faster and more consistent, but it also reduces repeated AI calls, token usage, and the overall AI footprint of the workflow. The saved artifact becomes a durable handoff between humans, systems, and later automation.

Explicit Steps Are Safer Than One Big Automation

The safest workflow is not “AI does everything.” The safer pattern is explicit commands with clear permissions:

• Read-only RCA.
• Explicit fix suggestion.
• Explicit Jira creation.
• Explicit PR creation.
• Human review before merge or deploy.

Good AI Workflows Are Team Leverage

One engineer could manually inspect Sentry, search the repo, write a Jira ticket, and prepare a PR. The point of Trestle Debugger is to make that easily repeatable for the team. The win is a shared workflow.

Roadmap

Trestle Debugger is useful today, but there are several areas where we want to keep improving it.

Service Mapping Layer

The service mapping layer should become easier to validate and maintain. Slack output can become richer while still keeping the thread readable. Deployment context is part of the roadmap, including future correlation with ArgoCD, GitHub commits, PRs, and release timing. PR routing can become CODEOWNERS-aware. We also want better metrics around RCA success rate, PR success rate, validation failures, and common workflow errors.

Runtime Monitoring Integrations

We want to research runtime monitoring integrations that can interact with the Trestle Debugger during an investigation. Today, the workflow starts from Sentry and the repository context. A future version could ask observability systems for relevant telemetry at runtime, such as logs, traces, latency changes, error-rate movement, or resource signals around the incident window. That would help the RCA move from “what does the code suggest?” toward “what actually happened in the running system?”

Longer term, the same pattern can apply beyond Sentry. Any alert source that can be mapped to code ownership and operational context could use a similar workflow.

We also want stronger isolation for write-mode Claude. Read-only RCA is lower risk. Code modification deserves tighter boundaries as the workflow matures.

Conclusion

Trestle Debugger started from a simple observation: Sentry alerts were easy to see but still slow to turn into confident engineering action.

The solution was never to replace engineers with AI. Instead, we solved this gap by connecting the systems engineers already used, giving Claude the right context and keeping each action reviewable.

Slack gives us the human-controlled entry point. Sentry gives us the issue. Repository mapping gives us ownership. Claude helps analyze and prepare changes. Jira and GitHub turn that work into trackable artifacts.

The result is a workflow that reflects how we want to use AI at Trestle: as a collaborator that reduces repetition, increases team leverage, and keeps humans accountable for the final output.