Integrating customer-provided Grafana instance with NSP Data Sources

The purpose of this tutorial is to guide users through the process of building customized Grafana dashboards using NSP data, by deploying an independent and external Grafana instance separate from the default NSP integration.

What is Grafana?

Grafana is an open-source data visualization and monitoring platform widely used to analyze metrics, logs, and traces collected from a variety of sources. It transforms raw data into accessible insights through highly customizable dashboards and alerting mechanisms. NSP relies on an integrated Grafana instance to gain visibility into system health, performance trends, and operational events. This tutorial explains how that functionality can be extended by deploying an external Grafana instance that connects to NSP data, enabling the creation of customized dashboards beyond the default integration.

Key Capabilities

Data Ingestion: Grafana can connect to NSP’s northbound interfaces using supported APIs (such as REST) or intermediary data stores like InfluxDB, Prometheus, or a custom connector pipeline.
Dashboarding: Real-time network KPIs from NSP (e.g., NSP Pod Status, CPU usage, link utilization, telemetry metrics) can be visualized using Grafana’s panel types—graphs, heatmaps, gauges, and tables.
Alerting: Custom alerts based on NSP-derived metrics can be defined using threshold conditions, with notifications routed via email, Microsoft Teams, Slack, or external incident systems.
Query Customization: Grafana supports domain-specific query languages (e.g., UQL) and templates, enabling filtered views by site, service, or device group as defined in the NSP topology.
User Access: Role-based permissions ensure that dashboard visibility and edit rights align with operational policies.

Note

NSP ships with an integrated Grafana instance that includes a set of predefined dashboards and a limited set of pre-installed plugins. This tutorial does not interact with that integrated environment. Instead, it provides step-by-step guidance for deploying a separate, standalone Grafana instance, installed on a system with direct connectivity to the NSP server - to build custom dashboards using NSP data. This approach facilitates enhanced customization by enabling the use of plugins that exceed the limitations of the default NSP integrated Grafana environment. Cloud-hosted deployments, such as Grafana Cloud, are not compatible due to connectivity and plugin limitations. For installation instructions of Grafana, refer to Grafana Installation Page.

Use Case 1: Card Inventory Dashboard using Infinity Data Source Plugin

This use case tends to develop a dynamic card inventory dashboard that displays total slot assignments per Network Element (NE) and NE type. The dashboard will integrate with the Network Inventory API using Infinity data source plugin, to retrieve real-time data, enabling accurate visualization and monitoring of card distribution across the network.

Infinity data source plugin is a universal connector for Grafana, offering seamless integration with data from non-traditional and custom formats. It enables visualization of external datasets such as REST APIs, JSON, CSV, XML, and similar sources that may not be supported natively within Grafana. Designed to bridge gaps in observability and reporting, the plugin accommodates use cases ranging from static file ingestion to dynamic API responses, without the need for a dedicated backend.

Installation of the Infinity plugin is flexible and straightforward. Users may choose to install it directly via the Grafana interface under the Plugins section or opt for command-line installation methods using grafana-cli. For detailed installation procedures, configuration guidance, and compatibility notes, refer to the official documentation provided by Grafana Labs: Installing Infinity data source plugin.

Step 1:

Following successful installation of the Infinity data source plugin, navigate to the Configuration section within Grafana and proceed to Data sources. Select and add Infinity as the designated data source.

Within the Infinity data source settings, navigate to the Authentication section, select option OAuth2 to enable secure communication. Configure the auth details using Client Credentials method, and set auth style to 'In Header'. Enter the username and password in fields 'Client ID' and 'Client Secret' respectively. The Token URL should be populated with below "NSP Gateway REST API". These APIs can be accessed from API documentation page under "Access & Authentication".

In the Allowed Hosts configuration, specify the NSP server endpoint using the following format: https://<NSP-server-ip/. This ensures that outbound API requests are restricted to approved network locations as per Grafana's security model.

https://{{server}}/rest-gateway/rest/api/v1/auth/token

Step 2:

To configure the card inventory dashboard using the Infinity data source plugin in Grafana, navigate to the Dashboards section and initiate the creation of a new panel. Within the panel editor, switch to the Query tab and select the Infinity data source that was previously configured.

Since Network Inventory NSP API is being used, the Source should be set to URL, allowing Grafana to retrieve data directly from the specified API endpoint. "Format" can be configured as Table, which ensures the returned JSON structure is appropriately mapped into rows and columns for tabular visualization. Infinity also offers additional format options which can be used according to user's choice.

Populate the query fields with appropriate endpoint URL and corresponding HTTP method. In this case, card information is retrieved based on a specific neType, so the payload has been provided accordingly by selecting the "Header, Body, Request params" option within the query configuration. The request body should then be entered, and the content type updated to "JSON". This payload itself can be customized to reflect user-defined criteria or operational preferences.

{
    "nsp-inventory:input" : {
            "xpath-filter": "/nsp-equipment:network/network-element[contains('type','${neType}')]/hardware-component/card",
            "fields": "ne-name;name;ne-id;card-details(card-type;available-slots;installed-slots)",
            "depth": "3"
        }
}

Step 3:

To configure '${neType}' as a variable within the dashboard, navigate to Dashboard Settings and open the Variables section. Create a new variable and set the Query Type to Infinity, ensuring that appropriate data source and query parameters are defined based on the target API. Use the relevant endpoint to retrieve a list of neType values. The below used Network Inventory API retrieves all network elements discovered in NSP. All such APIs can be accessed from 'Network Inventory' section of API documentation page.

https://{{server}}/restconf/data/nsp-equipment:network/network-element?depth=3&fields=ne-id;ne-name;type

Under the Parsing Options & Result Fields, define a column with the selector set to type and assign the title as neType. This allows the response values to be used dynamically. To support multiple values selection, enable the Multi-value option and save the configuration. Once this variable configuration is complete, panels referencing ${neType} will dynamically adjust based on the values retrieved from the Network Inventory API. This allows the dashboard to visualize card inventory per selected network element type.

In the panel view, navigate to the Query tab and set the parser to UQL, which enables handling and parsing of JSON-formatted responses. Within the UQL query field, append the expression parse json | jsonata "<below-filter>", where <below-filter> represents the actual JSONata filter used to analyze card slot assignments across Network Elements (NE), identifying the card type present in each slot. For instance, in scenarios involving five card slots on an NE, if two are populated with cards of type 'IOM', the query will yield five entries, two reflecting a count of one for the 'IOM' type, and the remaining three indicating a zero value due to unassigned slots. It is important to note that the query does not aggregate total counts of unassigned slots, focusing instead on assigned card types only.

The JSONata filter can be customized to include any card type not mentioned in the filter. Upon applying the filter and confirming query configuration, the parsed results are visualized in a tabular format, providing a structured overview of card distribution within the NE infrastructure as shown below. This visualization supports real-time analysis and can be refined further according to user's choice.

$map('nsp-inventory:output'.data,function($v){
  { 
	'ne-id': $v.'ne-id', 
	'name': $v.'ne-name', 
	'IOM/XCM': ($substring($v.'card-details'[0].'card-type', 0, 3) in ['iom','xcm'] or $substring($v.'card-details'[0].'card-type', 0, 8) = 'card_iom')? 1 : 0,
	'IMM': $substring($v.'card-details'[0].'card-type', 0, 3) = 'imm' ? 1 : 0,
	'Free_IOM_IMM_XCM': ($substring($v.'card-details'[0].'card-type', 0, 3) in ['iom','xcm','imm'] or $substring($v.'card-details'[0].'card-type', 0, 8) = 'card_iom') ? ($v.'card-details'[0].'available-slots' - $v.'card-details'[0].'installed-slots') : 0,
	'XIOM': ($substring($v.'card-details'[0].'card-type', 0, 5) = 'iom-s') or ($substring($v.'card-details'[0].'card-type', 0, 7) = 'iom2-se') or ($substring($v.'card-details'[0].'card-type', 0, 2) = 'x2') ? 1 : 0,
	'Free_XIOM': (($substring($v.'card-details'[0].'card-type', 0, 5) = 'iom-s') or ($substring($v.'card-details'[0].'card-type', 0, 7) = 'iom2-se') or ($substring($v.'card-details'[0].'card-type', 0, 2) = 'x2')) ? ($v.'card-details'[0].'available-slots' - $v.'card-details'[0].'installed-slots') : 0,
	'MDA/XMA': $not($substring($v.'card-details'[0].'card-type', 0, 3) in ['iom','xcm','imm'] or $substring($v.'card-details'[0].'card-type', 0, 8) = 'card_iom') and ($v.'card-details'[0].'card-type' != 'unassigned') ? 1 : 0 
  }
})

Step 4:

To enhance the visualization and refine the dataset, navigate to the Transform tab within the panel editor. From the available options, select Group by, which allows aggregation of data based on defined key fields. To achieve the goal that displays the total count of assigned card types per Network Element (NE), include all card type columns in the aggregation logic by selecting category 'Calculate' and stats 'Total'. Assign ne-id and name as the grouping criteria, ensuring that rows are grouped per NE and the card type totals are computed accordingly.

To improve readability and layout, apply the Organize fields transformation. This enables renaming of column headers to reflect domain-specific terminology and provides control over the ordering of fields within the panel. Columns can be repositioned to prioritize identifiers, counts, or any other attributes, depending on the presentation requirements.

Step 5:

Grafana's panel editor provides several ways to refine and present data clearly within the panel. From the right menu under the Visualizations tab, select Table to format the data into a structured view. In the Overrides tab, apply column-specific enhancements for all fields representing total card slots. To refine the panel further, apply the LCD Gauge display mode under cell settings. This mode replaces plain numeric output with a compact gauge visual, making slot totals more immediately digestible. Adjust the color scheme to enhance contrast or align with thematic branding. These changes elevate readability and deliver instant visual cues alongside the data.

The final representation displays a structured table panel where card slot assignments are grouped per Network Element, with each card type column reflecting the total number of assigned slots. Identifiers such as ne-id and name are positioned at the beginning of each row for clear reference. Card type columns are enhanced using the LCD Gauge display mode, providing a compact visual indicator of slot utilization. Customized color schemes applied to these gauges offer immediate visual differentiation, highlighting distribution patterns and drawing attention to key metrics. The overall layout presents a clean, informative view of card inventory across the selected NE types, optimized for operational monitoring and analysis.

Note

The image above displays the card inventory across all Network Element (NE) types. To view inventory for a specific NE type, use the 'Chassis Type' dropdown located in the top-left corner. Upon selecting the desired NE type, the panel dynamically updates to show only the relevant Network Elements.

Use Case 2: Real-Time Telemetry Data Streaming with Telegraf

The objective of this usecase is to integrate Telegraf with Grafana for real-time visualization of telemetry data. Telegraf is configured to listen to Kafka notifications and transmit the data via WebSocket, enabling seamless streaming into Grafana dashboards. This use case focuses on two distinct telemetry subscriptions: one for gNMI-based telemetry and another for accounting-related telemetry, each contributing to a comprehensive monitoring solution.

Telegraf is a lightweight, plugin-driven agent designed for collecting, processing, and sending metrics and events from a wide range of sources. It supports various input and output plugins, making it highly adaptable for monitoring and observability tasks. In this tutorial, the open-source version of Telegraf is installed on the same system where the standalone Grafana instance is deployed, ensuring direct connectivity between the two components. Refer to InfluxData’s official Telegraf page for the installation of Telegraf.

Step 1:

Once Telegraf is successfully installed on the Grafana host system, ensure that its service status is active and running. Within the Grafana user interface, navigate to the Service Accounts section and create a new service account. After creation, generate a service account token by selecting the option to add a token. This token is essential and should be mentioned in telegraf configuration, so it should be securely stored for future use.

Step 2:

Refer to Telemetry tutorial for detailed guidance on creating telemetry subscriptions. When configuring a subscription, ensure that the "notification" attribute is enabled, either via API or by selecting "Enable notifications and notification counters" in the NSP UI. This will automatically generate a "notif-topic"/ "Notification Topic" which serves as Kafka topic associated with that subscription. For this use case, the telemetry types to be consumed are "system-info" and "cfm-dmm-session-acc-stats".

Step 3:

Telegraf uses a configuration file to define its behavior, including which plugins to enable and how data flows between them. By default, Telegraf creates this file at - /etc/telegraf/telegraf.conf. If the file is not present, for detailed setup instructions, refer to Telegraf’s official Get Started guide.

Within this configuration file, the following plugins are used:

Input Plugin - source from which Telegraf collects data. In this configuration, Telegraf is set up to consume messages from a Kafka topic which is the generated 'Notification Topic' or 'notif-topic' of the subscription, functioning as a Kafka consumer. The authentication method used for this Kafka integration is TLS, ensuring secure communication between Telegraf and the Kafka broker. Required certificates can be obtained by contacting NSP team. Brokers should be populated with 'NSP-IP' and port used, in this case it's 9192. Also, kafka-version must match the version installed in the NSP environment. Contact NSP team to confirm the correct version.
Output Plugin - defines where Telegraf sends the processed data. Here, the output is a WebSocket channel, meaning the data is forwarded to a WebSocket endpoint for real-time streaming or integration.
Processor Plugin - A component that transforms or filters data between input and output. The regex processor is used to apply regular expression-based transformations—such as extracting, modifying, or discarding specific fields or values within the data payload. In this context, the regex is used to extract the telemetry type substring that succeeds "telemetry" from "kpiType":"telemetry:/base/interfaces/interface"

[[inputs.kafka_consumer]]
  brokers = ["<NSP-IP>":<port>"]
  kafka_version = "<Kafka-version>"
  topics = ["<Kafka-topics>"]
  enable_tls = true
  tls_ca = "/etc/ssl/ca-cert.pem"
  tls_cert = "/etc/ssl/client-cert.pem"
  tls_key = "/etc/ssl/client-key.pem"

  data_format = "json"
  json_query = "data.ietf-restconf:notification.nsp-kpi:real_time_kpi-event"
  json_time_key = "time-captured"
  json_time_format = "unix_ms"
  json_name_key = "kpiType"
  tag_keys = [
    "neId",
    "name"
  ]

[[processors.regex]]
  [[processors.regex.metric_rename]]
    pattern = "telemetry\\:\\/(.*)$"
    replacement = "${1}"

[[outputs.websocket]]
  url = "ws://localhost:3000/api/live/push/nsp"
  data_format = "influx"
  [outputs.websocket.headers]
    Authorization = "Bearer <service acc token>"

After the configuration, restart Telegraf service using this command,

systemctl restart telegraf

and the status of the Telegraf service can be verified using the following command,

systemctl status telegraf

A correctly configured setup will display the service as active (running). To monitor runtime activity and confirm successful data ingestion, the following command can be used to stream logs,

journalctl -u telegraf -f

Log entries indicating successful consumption of Kafka messages and transmission to the WebSocket endpoint confirm that Telegraf is operating as expected.

Step 4:

Navigate to the Grafana UI and create a new dashboard. Choose '-- Grafana --' as the data source, then proceed to the Query section. From there, select Live Measurements, and under Channel, choose the Telemetry type corresponding to the subscriptions that were previously configured. This ensures the dashboard reflects real-time data for the intended telemetry stream.

The final representation displays gNMI telemetry data, where the selected telemetry type is "system-info". From the available counters, CPU usage has been specifically chosen for monitoring. This setup provides focused visibility into system performance by tracking CPU metrics in real time.

This is the time series visualization for accounting telemetry data, specifically displaying a graph of the 'delay-dmm-2wy-avg' counter for 4 nodes discovered in NSP. The data is sourced from CFM DMM session accounting statistics, offering insight into average two-way delay measurements over time. This representation helps track latency trends and supports performance analysis for service assurance.

Integrating customer-provided Grafana instance with NSP Data Sources

What is Grafana?

Key Capabilities

Use Case 1: Card Inventory Dashboard using Infinity Data Source Plugin

Step 1:

Step 2:

Step 3:

Step 4:

Step 5:

Use Case 2: Real-Time Telemetry Data Streaming with Telegraf

Step 1:

Step 2:

Step 3:

Step 4:

On this page

Network solutions