Link Search Menu Expand Document

Guide

Table of contents

  1. Prerequisites
    1. Microservice Observability
    2. Install the Lightstep Service Graph Connector
  2. Configure Lightstep
    1. Create Lightstep Stream
    2. Get Lightstep Information
  3. Update the OTEL Collector to use the Service Graph compatible version
    1. Update the Lightstep configmap
    2. Update Helm Chart
  4. Deploy Helm Chart
    1. Re-Deploy Application using Helm Pipeline
    2. Check OTEL Collector Deployment
  5. Configure the Lightstep Service Graph Connector
  6. View Services & Relationships

Prerequisites

Microservice Observability

Implement the Microservice Observability Solution.

Install the Lightstep Service Graph Connector

Lightstep Service Graph Connector enables teams to automatically discover services and relationships across platforms and send them to ServiceNow to ensure a complete view of your environment. Installation is as follows:

  1. Login to your SN Instance as Administrator

  2. Navigate to the System Definition > Plugins and install or activate the following plugins:

    Plugin Name Plugin ID
    Service Graph Connector for Observability - Lightstep v1.0.1 sn_sg_lightstep

Configure Lightstep

Create Lightstep Stream

  1. Browse to Lightstep App

  2. Select your Project (e.g. dev, prod, or other)

  3. Navigate to Streams

  4. Create a Stream (e.g. mystream)

  5. Click on your Stream

  6. Note and safeguard your Stream ID from the URL as shown below:

    Stream ID

Get Lightstep Information

  1. Browse to Lightstep App

  2. Select your Project (e.g. dev, prod, or other)

  3. Navigate to Settings

  4. Note and safeguard your Organization and your Project identifiers.

  5. Navigate to Settings > Access Tokens

  6. Safeguard your Access Token

  7. Navigate to Account > Account Settings > API Keys

  8. Create and safeguard an API Key as follows:

    Field Value
    Description Any memorable description (e.g. srecon22)
    Role Member

Update the OTEL Collector to use the Service Graph compatible version

Update the Lightstep configmap

  1. Start a Bash Shell

  2. Configure kubectl environment variable

     export KUBECONFIG=$HOME/.kube/config-aks

  3. Set kubectl context for AKS (See Example below)

     kubectl config use-context olympus

  4. Test AKS Cluster Connection

     kubectl cluster-info
 kubectl get nodes -A

    Boutique Frontend

  5. Set Lightstep Access Token Environment Variable

     export LS_ACCESS_TOKEN=YOUR LIGHTSTEP ACCESS TOKEN
 export LS_STREAM_ID=YOUR LIGHTSTEP STREAM ID
 export LS_API_KEY=YOUR LIGHTSTEP API KEY
 export LS_ORG_ID=YOUR LIGHTSTEP ORG ID
 export LS_PROJECT_ID=YOUR LIGHTSTEP PROJECT ID

  6. Update the Lightstep configmap

     kubectl apply -n cassandra -f - <<EOF
 kind: ConfigMap
 apiVersion: v1
 metadata:
   name: lightstep-configmap
 data:
   collector-config: |
     receivers:
       otlp:
         protocols:
           grpc:
           http:
       lightstep-streams/mystream: 
         organization: "${LS_ORG_ID}" 
         project: "${LS_PROJECT_ID}" 
         api_key: "${LS_API_KEY}"
         window_size: 5m 
         stream_id: "${LS_STREAM_ID}"
     processors:
       resourcedetection/azure:
         detectors: [env, azure]
         timeout: 2s
         override: false
     exporters:
       otlp:
         endpoint: ingest.lightstep.com:443
         headers: 
           "lightstep-access-token": "${LS_ACCESS_TOKEN}"
       logging:
         loglevel: debug
       service: 
         scraper: 
           endpoint: 0.0.0.0:55688 
     service:
       pipelines:
         traces:
           receivers: [otlp]
           processors: [resourcedetection/azure]
           exporters: [logging,otlp]
         traces/sgc:
           receivers: [lightstep-streams/mystream]
           processors: []
           exporters: [logging,service]
 EOF

Update Helm Chart

  1. Sign In to GitHub

  2. Select the Boutique Project (e.g. cassandra) you configured earlier.

  3. Locate the helm/templates/kubernetes-manifests.yaml file

  4. Locate the Deployment section for the otelcollector

  5. Un-Comment the image section as follows:

     # image: otel/opentelemetry-collector-contrib:0.49.0 
 image: ghcr.io/lightstep/lightstep-partner-toolkit-collector:latest

  6. Expose Port 55688

     ports:
   - containerPort: 55688 # Add this entry

  7. Locate the Service section for the otelcollector

  8. Expose Port 55688

     # Add this section
 - name: "55688"
   port: 55688
   targetPort: 55688

  9. Locate the Service section for the otelcollector-external

  10. Un-Comment the Service as follows:

     apiVersion: v1
 kind: Service
 metadata:
   name: otelcollector-external
 spec:
   type: LoadBalancer
   selector:
     app: otelcollector
   ports:
   - name: "55688"
     port: 55688
     targetPort: 55688

  11. Commit changes

Deploy Helm Chart

Re-Deploy Application using Helm Pipeline

After the Lightstep instrumentation, we need to re-deploy the 10 Boutique application Pods along with the new Lightstep OTEL collector Pod to your Azure Kubernetes using the same provided kubernetes-deploy-all Helm Pipeline. Procedure is as follows:

  1. Sign In to Azure DevOps

  2. Select the Boutique Project (e.g. cassandra) you configured earlier.

  3. Navigate to Pipelines > All > azure-pipelines

  4. Select the kubernetes-deploy-all Pipeline

  5. Press Run Pipeline

  6. Review Variables and press Run as follows:

    Name Value Keep this value secret Let users override this value when running this pipeline
    SERVICE_NAMESPACE cassandra False True
    REPO_PREFIX YOUR DOCKER HUB ID False True

  7. Press Run

  8. Monitor the Pipeline and make sure it runs successfully

    Check OTEL Collector Deployment

  9. Start a Bash Shell

  10. Start a Bash Shell

  11. Configure kubectl environment variable

     export KUBECONFIG=$HOME/.kube/config-aks

  12. Set kubectl context for AKS (See Example below)

     kubectl config use-context olympus

  13. Test AKS Cluster Connection

     kubectl cluster-info
 kubectl get nodes -A

    Boutique Frontend

  14. Retrieve the otelcollector-external service EXTERNAL-IP

     kubectl get service otelcollector-external -n cassandra

  15. Browse to the http://EXTERNAL-IP:55688 to view the OTEL Collector JSON

    Otel Json

Configure the Lightstep Service Graph Connector

  1. Login to your SN Instance as Administrator

  2. Navigate to the SG Connector for Observability Lightstep > Setup

  3. Configure the Setup > Set your Organization and Project Task fields as follows:

    Field Value
    u_lightstep_project YOUR LIGHTSTEP PROJECT ID
    u_lightstep_organization YOUR LIGHTSTEP ORG ID

  4. Configure the Setup > Set your API Key Task fields as follows:

    Field Value
    Name Some memorable name (e.g. srecon22)
    srecon22 YOUR LIGHTSTEP API KEY

  5. Configure the Setup > Configure OpenTelemetry Task fields as follows:

    Field Value
    Host YOUR OTEL COLLECTOR EXTERNAL-IP
    Override default port 55688
    Protocol http

  6. Configure the Setup > Test the Connection to test connection.

  7. Configure the Setup > Set up Scheduled Import Jobs Task fields as follows:

    Lightstep Jobs

    NOTE: Set all Jobs Active=True

  8. Navigate to the SG Connector for Observability Lightstep > Scheduled Data Imports

  9. Select the SG-OpenTelemetry Resources Scheduled Import Job.

  10. Manually trigger the Job by pressing the Execute Now buton.

View Services & Relationships

  1. Login to your SN Instance as Administrator

  2. Navigate to the SG Connector for Observability Lightstep > Lightstep Services

  3. Click on View map for your Lightstep Project (e.g. LS-servicenow-dev-e95fa366)

    SGC Services 1`

  4. Right-Click on the LS-frontend service and select Open in Dependency Views.

    SGC Services 2`