Guide

Table of contents

1. Prerequisites
   1. Get a Lightstep Account
2. Instrument Code
   1. Get Lightstep Access Token
   2. Create Lightstep configmap
   3. Enable Instrumentation
3. Deploy Instrumented Application
   1. Re-Deploy Application using Helm Pipeline
   2. Monitor Services

Prerequisites

Get a Lightstep Account

Lightstep is the leading Open Telemetry platform that natively, naturally, and scalably combines metrics and tracing. Query and investigate issues and automatically correlate root causes across Traces, Metrics, and Logs anywhere in the system. You will need a Lightstep account to collect Telemetry from your servics. Procedure is as follows:

1. You probably allready have a Lightstep Account, or you can register for one as follows:

   • Browse to the Lightstep App and Create a Free Account.

Instrument Code

Get Lightstep Access Token

1. Browse to Lightstep App

2. Select your Project (e.g. dev, prod, or other)

3. Navigate to Settings > Access Tokens

4. Safeguard your Access Token

Create Lightstep configmap

1. Start a Bash Shell

2. Configure kubectl environment variable

    export KUBECONFIG=$HOME/.kube/config-aks
   

3. Set kubectl context for AKS (See Example below)

    kubectl config use-context olympus
   

4. Test AKS Cluster Connection

    kubectl cluster-info
    kubectl get nodes -A
   

   

5. Set Lightstep Access Token Environment Variable

    export LS_ACCESS_TOKEN=YOUR LIGHTSTEP ACCESS TOKEN
   

6. Create Lightstep configmap

    kubectl apply -n cassandra -f - <<EOF
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: lightstep-configmap
    data:
      collector-config: |
        receivers:
          otlp:
            protocols:
              grpc:
              http:
        exporters:
          otlp:
            endpoint: ingest.lightstep.com:443
            headers: 
              "lightstep-access-token": "${LS_ACCESS_TOKEN}"
          logging:
            loglevel: debug
        service:
          pipelines:
            traces:
              receivers: [otlp]
              processors: []
              exporters: [logging,otlp]             
    EOF
   

Enable Instrumentation

1. Sign In to GitHub

2. Select the Boutique Project (e.g. cassandra) you configured earlier.

3. Locate the helm/templates/kubernetes-manifests.yaml file

4. Un-Comment the Deployment section for the otelcollector

5. Un-Comment the image section as follows:

    image: otel/opentelemetry-collector-contrib:0.49.0 
    # image: ghcr.io/lightstep/lightstep-partner-toolkit-collector:latest
   

6. Un-Comment the Service section for the otelcollector

7. Un-Comment the #Lightstep Instrumentation sections for all the services to expose the OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and OTEL_RESOURCE_ATTRIBUTES to Lightstep as follows:

    # Lightstep Instrumentation
    - name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
      value: "http://otelcollector:4317"
    - name: OTEL_RESOURCE_ATTRIBUTES
      value: "service.name=productcatalogservice,service.version=1.0.0"
   

8. Commit changes

Deploy Instrumented Application

Re-Deploy Application using Helm Pipeline

After the Lightstep instrumentation, we need to re-deploy the 10 Boutique application Pods along with the new Lightstep OTEL collector Pod to your Azure Kubernetes using the same provided kubernetes-deploy-all Helm Pipeline. Procedure is as follows:

1. Sign In to Azure DevOps

2. Select the Boutique Project (e.g. cassandra) you configured earlier.

3. Navigate to Pipelines > All > azure-pipelines

4. Select the kubernetes-deploy-all Pipeline

5. Press Run Pipeline

6. Review Variables and press Run as follows:

   Name	Value	Keep this value secret	Let users override this value when running this pipeline
   SERVICE_NAMESPACE	cassandra	False	True
   REPO_PREFIX	YOUR DOCKER HUB ID	False	True

7. Press Run

8. Monitor the Pipeline and make sure it runs successfully

Monitor Services

1. Browse to Lightstep App

2. Select your Project (e.g. dev, prod, or other)

3. Navigate to the Service Directory, and you should see all 10 services emitting traces

   

4. Navigate the Explorer and query for non-health check operations, and sort by descending Duration as follows:

    operation NOT IN ("/_healthz", "/grpc.health.v1.Health/Check", "grpc.grpc.health.v1.Health/Check", "grpc.health.v1.Health/Check", "PING")
   

   

5. Click on one of the longer span durations to examine where the transaction is spending the most time as follows:

   

6. Navigate back to the Explorer and examine the live Service Diagram detected by Lightstep from the Traces as follows:

   

---