Menu

View traces from a running app.

Tracing usually means running a collector, a backend and opening a browser tab. So they only get looked at when something is already on fire.

Praytel puts the traces in the terminal you started the app from. Make that part of how you run things and you get two payoffs: It’s easier to understand the apps you use, and the traces are already good when something does go wrong.

Run it

praytel --tui -- ./server

Everything after -- is your command, run unchanged. Praytel forwards signals to it and exits with its exit code, so it can sit in front of anything you already run.

Try it yourself

You can try praytel before instrumenting anything of your own: run it against shop, a small app that ships with praytel, or against a tool you may already have installed.

The shop app

shop is a small Go program in demo/workloads/shop that places four orders across four in-process services (web, checkout, inventory, payments). It exports OTLP traces and writes JSON logs tagged with trace and span IDs. The recording above is of this app.

Two of its orders are there to be found:

OrderCardWhat happens
A-1002amexGateway.Bank.Authorize takes 450ms and logs a retry.
A-1003giftGateway.Bank.Authorize refuses the card; the error travels up to Orders.Create.

Install it and praytel from the repository root, with Go installed:

go install ./cmd/praytel
(cd demo/workloads/shop && go install .)

Both binaries land in $(go env GOPATH)/bin. shop also runs without praytel, exporting nothing, so you can compare the two. Then:

praytel --tui -- shop

Four traces fill in, one per order, each rooted at Orders.Create. Type gift to narrow to the order that failed, and follow the × marks down to Gateway.Bank.Authorize. Esc clears the search; Ctrl-C quits.

A tool you already have

Some tools export traces themselves. Neither command below changes anything: one lints, the other lists.

praytel --tui -- docker buildx build --call=check .   # lint the Dockerfile in .
praytel --tui -- dagger functions                     # in a Dagger module
ToolWhat the trace shows
docker buildxA build span over the gRPC calls buildx makes to BuildKit, such as moby.buildkit.v1.Control/Solve.
daggerThe CLI starting and connecting to its engine, then the engine loading the module.

This shows two things:

  1. we can ‘pop the hood’ on even the tools we already have and just have a look - fun!
  2. the overview will only be as good as the traces you wire up your app with. e.g. docker buildx build is traced much more clearly than docker ps. Perfectly understandable, but the more you trace your own tools the more you’ll get out of all this.

What the app needs

The app should export OpenTelemetry traces over OTLP. Praytel starts a receiver and points the child at it through the standard OTEL_EXPORTER_OTLP_* environment variables, so an OpenTelemetry SDK that configures its exporter from the environment needs no code change.

There is no collector to run. If one is already listening on the default ports (4317 for gRPC, 4318 for HTTP), move praytel to other ports:

praytel --tui --grpc-port 14317 --http-port 14318 -- ./server

An app with no instrumentation still runs normally; you will see its output but no spans.

Reading the tree

Each trace is a tree of spans. Every span shows its state and how long it took.

MarkMeaning
spinnerThe span is still open.
●The span closed without error.
×The span closed with an error status.

A duration prefixed with ~ is an estimate for an open span whose start time has not been reported yet.

Finding things

Type to search. The match is loose on purpose: letters only need to appear in order across span names and attributes, and one matching span surfaces its whole trace. The characters that matched are highlighted so you can see why a trace showed up. Esc clears the search, the arrow keys scroll, and Ctrl-C quits. The full key list is in praytel(1).

The TUI stays open after the app exits, so you can keep reading.

Long-running and chatty apps

Praytel holds everything in memory. --max-spans caps what it keeps (one million by default). At the cap it evicts the oldest closed traces first and says so on stderr.

Next

View logs correlated to traces.