Collecting data with the tracing crate and printing them out in the console is a great start. But what if, instead of a single application, we had 10 or 20 instances deployed?
How would we follow along with which application crashed and has latency issues?

That’s where Jaeger comes into play. Jaeger allows us to collect traces from various applications and analyze them later. You can download Jaeger from jaegertracing.io. For development and testing purposes, make sure to download Jaeger All in One.

Install the necessary crates

Before we start to make code changes, let’s install the necessary Jaeger crates in Cargo.toml:

[package]
name = "tokio-tracing-demo"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
# Install these crates
tracing = "0.1.29"
tracing-subscriber = {version = "0.3.3", features = ["std", "env-filter"]}
tracing-opentelemetry = "0.16.0"
opentelemetry-jaeger = "0.15.0"
opentelemetry = "0.16"

We’re using tracing to collect spans from our application. In addition, we’re also installing crates such as tracing-subscriber, tracing-opentelemetry, opentelemetry-jaeger, as well as opentelemetry. With these installed, we’re able to send spans off to Jaeger (and other OpenTelemetry backends).

Initialize a Jaeger Pipeline

Now that we have all necessary crates installed, let’s have a look at how to initialize an OpenTelemetry Jaeger pipeline. Open src/main.rs and add:

use opentelemetry::sdk::trace::Tracer;
use opentelemetry::trace::TraceError;
use tracing::info;
use tracing_subscriber::layer::SubscriberExt;
use tracing_subscriber::prelude::*;


//add this function
fn init_tracer() -> Result<Tracer, TraceError> {
    opentelemetry_jaeger::new_pipeline()
        .with_service_name("jaeger_example")
        .install_simple()
}

//
// Make sure your main function's return type matches the one below
//
fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
    //...
}

init_tracer creates a new Jaeger pipeline, configuring the service name ("jaeger_example") under which the traces will show up in the Jaeger UI. install_simple installs a Jaeger pipeline with a simple span processor. It’s also possible to have the tracer export traces in batches via Tokio.

Now that we have init_tracer in place, let’s use it:

fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
    let tracer = init_tracer().expect("Failed to initialize tracer"); //calling our new init_tracer function

    tracing_subscriber::registry() //(1)
        .with(tracing_subscriber::EnvFilter::new("TRACE")) //(2)
        .with(tracing_opentelemetry::layer().with_tracer(tracer)) //(3)
        .try_init()
        .expect("Failed to register tracer with registry");
    //...
}

We want to ensure that whenever we apply the tracing::instrument attribute or use info!/error!, spans are created using the tracer instance we instantiated by init_tracer.
We ensure that by calling tracing_subscriber::registry (1). We’d also like to ensure that everything equal or above the Trace level (Debug, info, warn, error) will get sent to Jaeger (2).
In step (3), we’re configuring our tracer struct so that all traces will end up in Jaeger.

Once that’s done, you can start instrumenting your code using the instrument attribute or custom-built spans.

Also, make sure to add this call at the end of your main function:

    //...
    opentelemetry::global::shutdown_tracer_provider(); //add this line
    Ok(())
}

shutdown_tracer_provider ensures that all collected traces will be sent to Jaeger.

Within Jaeger

Once your application is sending spans, open the Jaeger Web UI, running at http://localhost:16686/.

Select your application from the dropdown and click the “Search” button on the left-hand side. Now you’ll see traces populating. Click on the first one to open the detail view.

In the detail view, you see a timeline view as well as further details in the expanded view. The code generating these spans looks like this:

#[tracing::instrument]
pub(crate) fn shave_all(number_of_yaks: i32) -> i32 {
    for yak_index in 0..number_of_yaks {
        info!(current_yak=yak_index+1, "Shaving in progress");
    }

    number_of_yaks
}

The screenshot below shows some general information about this function call; what’s more interesting, though, is the information under Logs. We have one log entry for each iteration of the for loop, indicating the current yak being shaved.

Conclusion

With only a few lines of code, you can upgrade your already existing instrumentation to a new level. We can collect and analyze telemetry data from dozens of application instances with Jaeger without breaking a sweat.

Find the source code for this blog post here: link. Did you find this blog post helpful? Make sure to tell me on Twitter.