This tutorial is for performing inventory requests in order to visualize the NSP managed network in an OSS.
The basic steps are:
This tutorial requires a network managed by NSP. Inventory requests will be made for the following objects:
This tutorial has been tested with and is supported in NSP 23.4.
All steps in this tutorial are provided in the Postman Collection. Example responses for each request are also included.
The collection can be run using "Postman Runner" to perform the complete configuration. The following environment variables must be defined in order to run the collection:
The following artifacts from ALED are required to run this tutorial:
This tutorial is meant as a proof of concept and as an example method for configuring a service. It is not designed to be implemented, and must not be used as is, in a production network. It is intended to guide development of an OSS.
This tutorial contains sample code that has been written in Golang and is intended to show how data can be retrieved from NSP and parsed to suit particular purposes. For readability purposes, it has not been optimized, and omits most error handling. No warranty is available, either expressed or implied, for this sample code.
As will be described below, this tutorial makes use of Kafka for ensuring that data retrieved from NSP is kept up to date.
For further details on using Kafka, see Kafka Notification Service
The initial inventory requests will be made using the NSP RESTCONF framework. It is recommended to make use of the "depth" and "fields" query parameters in order to limit the amount of information that is returned from NSP with a single request.
For simple requests, the use of RESTCONF "GET" is recommended. For more complicated requests, such as those using multi-level filtering, the NSP "nsp-inventory:find" method can be used. This tutorial makes use of both methods.
By default, inventory requests will retrieve up to 300 entries, starting at index 0. This can be modified using the "limit" and "offset" query parameters. The number of results is set using limit and can be a maximum of 1000. The offset is used to specify the index to start retrieving from. Responses include a start and end index which can be used to determine what value to use for the offset. There is also a total count included that can be used to determine when the end of the results has been reached.
Although the network element inventory can be retrieved in a single request, it is recommended to use multiple requests to retrieve different pieces of the hierarchy. For this tutorial, the network elements, cards, and ports will be retrieved in separate calls.
Network Elements will be filtered to retrieve only the type, administrative and operational states, NE id, management IP address, version and name. Using a depth of '2' limits the response to only the network element without traversing to cards and ports.
GET /restconf/data/nsp-equipment:network/network-element?depth=2&fields=type;admin-state;ne-id;ip-address;oper-state;version;name HTTP/1.1Cards will be retrieved with no filtering.
GET /restconf/data/nsp-equipment:network/network-element/hardware-component/card HTTP/1.1Ports will be retrieved using the "nsp-inventory:find" method. This will easily allow the fields that are returned in the results to be filtered even though at different levels. The NE id, administrative and operational states, and port name are from the port level. Under the port, there is a port-details that includes the mode, type, rate, mtu, and encapsulation type for the port. Finally, the lag-member-details includes the lag ID for any lags that the port may be a member of.
POST /restconf/operations/nsp-inventory:find HTTP/1.1
{
"nsp-inventory:input": {
"xpath-filter": "/nsp-equipment:network/network-element/hardware-component/port",
"depth": "2",
"fields": "ne-id;admin-state;oper-state;name;lag-member-details/lag-id;port-details(port-mode;port-type;actual-rate;mtu-value;encap-type)"
}
}The physical links are the last piece of the physical inventory that will be retrieved
GET /restconf/data/nsp-service:services/physical-layer/cable HTTP/1.1The logical inventory that this tutorial will retrieve contains the LAGs, and the IETF modelled IGP topology.
GET /restconf/data/nsp-equipment:/network/network-element/lag HTTP/1.1The IGP topology requires that a VSR be integrated with the NSP. See the NSP documentation for details on configuring a VSR for use with NSP.
For the IETF model to be available, the IETF mapping files must be installed in NSP. The files are available from ALED and can be installed using the RESTCONF uploadFile API.
Uploading the mapping files will automatically trigger a resync of IGP data from the network. Depending on the size of the network, this may take several minutes.
POST /restconf/operations/nsp-yang-mapping-converter:nsp-yang-mapping-converter/uploadFile?nsp-plugin-id=yang HTTP/1.1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="IetfNodeHandlerForNSPNetworkElement.java"
Content-Type: <Content-Type header here>
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="IetfSapTopology.json"
Content-Type: application/json
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="IetfTopologies.json"
Content-Type: application/json
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="IetfTopologyDeleteObjectFilter.java"
Content-Type: <Content-Type header here>
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="L2TopologyHandler.java"
Content-Type: <Content-Type header here>
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="L3UnicastTopologyHandler.java"
Content-Type: <Content-Type header here>
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="SAPTopologyHandler.java"
Content-Type: <Content-Type header here>
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gWSimilar to the physical inventory, it is recommended to retrieve the IGP topology using separate requests. The first request is to retrieve all of the available topologies.
GET /restconf/data/ietf-network:networks/network?depth=2 HTTP/1.1Once the IGP topologies have been identified, requests can be made to retrieve the nodes and links that make up the topology.
GET /restconf/data/ietf-network:networks/network=TopologyId-0:0:0-ospf/node?fields=node-id HTTP/1.1GET /restconf/data/ietf-network:networks/network=TopologyId-0:0:0-ospf/ietf-network-topology:link HTTP/1.1With the inventory that has been retrieved, it is necessary to parse the objects and possibly correlate them in order to generate a useful visualization. The tutorial will show how to create:
The sample source code at the end of this section shows one way of performing the parsing in a programmatic way.
Connected endpoints physical cables are modelled as connected endpoints. The endpoints can be pulled from the model in order to determine a mapping between network elements and ports that are physically connected.
for each cable
get endpoint a network element and port
get endpoint b network element and port
add two endpoints to map
end forLAG membership There are two ways to determine the members of a LAG:
for each port
if lag id is populated
create lag on site if not already created
add port as member of lag
end if
end for2. iterate over LAGs to determine what ports they contain
Each LAG has a list of FDNs for ports that are members of the LAG.
for each LAG
look at list of members
look up details of ports, based on FDN, as needed
end forEach method of looking at LAGs is defined in the sample code below.
Connected LAGs Individual members of LAGs are likely connected via physical cable to a LAG on another network element. In order to determine which LAGs are connected the physical links can be used. The pseudo code assumes that a map of connected physical links has been created as above, and that LAG membership has been determined.
for lag
for lag member
look up member in physical link map to find other end
look up other end in lag table to see which lag it is a member of
show the two lags as being connected
end for lag member
end for lagIGP Connections
The following sample code shows a possible implementation of the above concepts. Resultant structures are simply printed to the console in this sample, but can be visualized in any manner as deemed appropriate by the application.
package main
import (
"crypto/tls"
"crypto/x509"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"regexp"
"strconv"
"strings"
)
const (
server = "https://<<server>>"
authUrl = server + "/rest-gateway/rest/api/v1/auth/token"
revUrl = server + "/rest-gateway/rest/api/v1/auth/revocation"
findUrl = server + ":8545/restconf/operations/nsp-inventory:find"
lagUrl = server + ":8545/restconf/data/nsp-equipment:/network/network-element/lag"
linkUrl = server + ":8545/restconf/data/nsp-service:services/physical-layer/cable"
ietfNetwork = server + ":8545/restconf/data/ietf-network:networks/network"
user = "admin"
pass = "NokiaNsp1!"
certFile = "ssl/nsp_external_combined.pem"
)
// set up various structs for JSON Data
type Token struct {
Access_Token string `json:"access_token"`
Refresh_Token string `json:"refresh_token"`
Token_Type string `json:"token_type"`
Expires_In int64 `json:"expires_in"`
}
type Ports struct {
NspInventoryOutput struct {
Data []struct {
AdminState string `json:"admin-state"`
EquipmentExtensionPortExtension struct {
} `json:"equipment-extension-port:extension"`
LagMemberDetails []struct {
LagId string `json:"lag-id"`
} `json:"lag-member-details"`
Name string `json:"name"`
NeID string `json:"ne-id"`
OperState string `json:"oper-state"`
PortDetails []struct {
ActualRate float64 `json:"actual-rate"`
EncapType string `json:"encap-type"`
MtuValue int `json:"mtu-value"`
PortMode string `json:"port-mode"`
PortType string `json:"port-type"`
} `json:"port-details"`
} `json:"data"`
EndIndex int `json:"end-index"`
StartIndex int `json:"start-index"`
TotalCount int `json:"total-count"`
} `json:"nsp-inventory:output"`
}
type Lag struct {
NspEquipmentLag []NspEquipmentLag `json:"nsp-equipment:lag"`
}
type NspEquipmentLag struct {
LagIndex int `json:"lag-index"`
ConfiguredAddress string `json:"configured-address"`
Parent interface{} `json:"parent"`
LagID string `json:"lag-id"`
AvailabilityState []interface{} `json:"availability-state"`
OperationalSpeed float64 `json:"operational-speed"`
OperState string `json:"oper-state"`
Description interface{} `json:"description"`
OperationalSpeedUnits string `json:"operational-speed-units"`
AdminState string `json:"admin-state"`
NeID string `json:"ne-id"`
NeName string `json:"ne-name"`
LagMode string `json:"lag-mode"`
StandbyState string `json:"standby-state"`
Members []string `json:"members"`
Name string `json:"name"`
StateReason []interface{} `json:"state-reason"`
EncapType string `json:"encap-type"`
ParentNe string `json:"parent-ne"`
}
type Cable struct {
NspServiceCable []NspServiceCable `json:"nsp-service:cable"`
}
type PortBindings struct {
Resource string `json:"resource"`
Name interface{} `json:"name"`
OperState string `json:"oper-state"`
Description interface{} `json:"description"`
AdminState string `json:"admin-state"`
ClassName string `json:"class-name"`
}
type LinkEndpoint struct {
ServiceName interface{} `json:"service-name"`
SiteName interface{} `json:"site-name"`
Site string `json:"site"`
PortBindings []PortBindings `json:"port-bindings"`
Service string `json:"service"`
Name string `json:"name"`
OperState string `json:"oper-state"`
Description interface{} `json:"description"`
SiteID string `json:"site-id"`
AdminState string `json:"admin-state"`
Type string `json:"type"`
EndpointID string `json:"endpoint-id"`
}
type NetworkElementBinding struct {
Resource string `json:"resource"`
Name interface{} `json:"name"`
OperState string `json:"oper-state"`
Description interface{} `json:"description"`
AdminState string `json:"admin-state"`
ClassName string `json:"class-name"`
}
type LinkSite struct {
ServiceName interface{} `json:"service-name"`
Endpoint []string `json:"endpoint"`
TunnelBinding []interface{} `json:"tunnel-binding"`
Service string `json:"service"`
Name interface{} `json:"name"`
OperState string `json:"oper-state"`
Description interface{} `json:"description"`
SiteID string `json:"site-id"`
AdminState string `json:"admin-state"`
NetworkElementBinding []NetworkElementBinding `json:"network-element-binding"`
}
type NspServiceCable struct {
LinkEndpoint []LinkEndpoint `json:"link-endpoint"`
ServiceID interface{} `json:"service-id"`
Latency interface{} `json:"latency"`
Origin string `json:"origin"`
OperState string `json:"oper-state"`
Description string `json:"description"`
AdminState string `json:"admin-state"`
NeServiceID interface{} `json:"ne-service-id"`
IsNfmpIPService bool `json:"is-nfmp-ip-service"`
LinkSite []LinkSite `json:"link-site"`
Name string `json:"name"`
ID string `json:"id"`
Direction string `json:"direction"`
}
type LinkEndpointKey struct {
neID string
portID string
}
type Ietf struct {
IetfNetworkNetwork []IetfNetworkNetwork `json:"ietf-network:network"`
}
type IetfNetworkNode struct {
Node []Node `json:"ietf-network:node"`
}
type IetfL3UnicastTopologyL3UnicastTopology struct {
}
type NetworkTypes struct {
IetfL3UnicastTopologyL3UnicastTopology IetfL3UnicastTopologyL3UnicastTopology `json:"ietf-l3-unicast-topology:l3-unicast-topology"`
}
type SupportingNetwork struct {
NetworkRef string `json:"network-ref"`
}
type SupportingNode struct {
NetworkRef string `json:"network-ref"`
NodeRef string `json:"node-ref"`
}
type IetfL3TeTopologyL3TeTpAttributes struct {
TpRef string `json:"tp-ref"`
NodeRef string `json:"node-ref"`
NetworkRef string `json:"network-ref"`
}
type IetfL3UnicastTopologyL3TerminationPointAttributes struct {
IPAddress []string `json:"ip-address"`
IetfL3TeTopologyL3TeTpAttributes IetfL3TeTopologyL3TeTpAttributes `json:"ietf-l3-te-topology:l3-te-tp-attributes"`
}
type IetfNetworkTopologyTerminationPoint struct {
TpID string `json:"tp-id"`
IetfTeTopologyTeTpID interface{} `json:"ietf-te-topology:te-tp-id"`
IetfL3UnicastTopologyL3TerminationPointAttributes IetfL3UnicastTopologyL3TerminationPointAttributes `json:"ietf-l3-unicast-topology:l3-termination-point-attributes"`
}
type Prefix struct {
Prefix string `json:"prefix"`
Metric int `json:"metric"`
}
type IetfL3TeTopologyL3TeNodeAttributes struct {
NodeRef string `json:"node-ref"`
NetworkRef string `json:"network-ref"`
}
type IetfL3UnicastTopologyL3NodeAttributes struct {
Name string `json:"name"`
Flag []string `json:"flag"`
RouterID []string `json:"router-id"`
NspIetfNetworkTopologyNspAttributes []interface{} `json:"nsp-ietf-network-topology:nsp-attributes"`
Prefix []Prefix `json:"prefix"`
IetfL3TeTopologyL3TeNodeAttributes IetfL3TeTopologyL3TeNodeAttributes `json:"ietf-l3-te-topology:l3-te-node-attributes"`
}
type Node struct {
NodeID string `json:"node-id"`
IetfTeTopologyTeNodeID interface{} `json:"ietf-te-topology:te-node-id"`
SupportingNode []SupportingNode `json:"supporting-node,omitempty"`
IetfNetworkTopologyTerminationPoint []IetfNetworkTopologyTerminationPoint `json:"ietf-network-topology:termination-point"`
IetfL3UnicastTopologyL3NodeAttributes IetfL3UnicastTopologyL3NodeAttributes `json:"ietf-l3-unicast-topology:l3-node-attributes"`
}
type IetfL3TeTopologyL3TeTopologyAttributes struct {
NetworkRef string `json:"network-ref"`
}
type IetfSrMplsTopologySrMpls struct {
}
type IetfL3UnicastTopologyL3TopologyAttributes struct {
Name string `json:"name"`
IetfL3TeTopologyL3TeTopologyAttributes IetfL3TeTopologyL3TeTopologyAttributes `json:"ietf-l3-te-topology:l3-te-topology-attributes"`
IetfSrMplsTopologySrMpls IetfSrMplsTopologySrMpls `json:"ietf-sr-mpls-topology:sr-mpls"`
}
type Source struct {
SourceNode string `json:"source-node"`
SourceTp string `json:"source-tp"`
}
type Destination struct {
DestNode string `json:"dest-node"`
DestTp string `json:"dest-tp"`
}
type IetfL3TeTopologyL3TeLinkAttributes struct {
LinkRef string `json:"link-ref"`
NetworkRef string `json:"network-ref"`
}
type IetfL3UnicastTopologyL3LinkAttributes struct {
Name string `json:"name"`
Metric1 int `json:"metric1"`
Metric2 int `json:"metric2"`
Flag []string `json:"flag"`
NspIetfNetworkTopologyNspAttributes []interface{} `json:"nsp-ietf-network-topology:nsp-attributes"`
IetfL3TeTopologyL3TeLinkAttributes IetfL3TeTopologyL3TeLinkAttributes `json:"ietf-l3-te-topology:l3-te-link-attributes"`
}
type IetfNetworkTopologyLink struct {
LinkID string `json:"link-id"`
Source Source `json:"source"`
Destination Destination `json:"destination"`
IetfL3UnicastTopologyL3LinkAttributes IetfL3UnicastTopologyL3LinkAttributes `json:"ietf-l3-unicast-topology:l3-link-attributes"`
}
type IetfTeTopologyPacketPacket struct {
}
type IetfTeTopologyTeTopology struct {
IetfTeTopologyPacketPacket IetfTeTopologyPacketPacket `json:"ietf-te-topology-packet:packet"`
}
type NetworkTypes0 struct {
IetfTeTopologyTeTopology IetfTeTopologyTeTopology `json:"ietf-te-topology:te-topology"`
}
type IetfTeTopologyTeTopologyIdentifier struct {
ProviderID int `json:"provider-id"`
ClientID int `json:"client-id"`
TopologyID string `json:"topology-id"`
}
type IetfNetworkNetwork struct {
NetworkID string `json:"network-id"`
NetworkTypes NetworkTypes `json:"network-types,omitempty"`
SupportingNetwork []SupportingNetwork `json:"supporting-network,omitempty"`
Node []Node `json:":node"`
IetfL3UnicastTopologyL3TopologyAttributes IetfL3UnicastTopologyL3TopologyAttributes `json:"ietf-l3-unicast-topology:l3-topology-attributes,omitempty"`
IetfNetworkTopologyLink []IetfNetworkTopologyLink `json:"ietf-network-topology:link"`
IetfTeTopologyTeTopologyIdentifier IetfTeTopologyTeTopologyIdentifier `json:"ietf-te-topology:te-topology-identifier,omitempty"`
}
func main() {
tlsConfig, err := LoadTlsConfig(certFile)
if err != nil {
panic(err)
}
tr := &http.Transport{
TLSClientConfig: tlsConfig,
}
client := &http.Client{Transport: tr}
// get auth token
payload := strings.NewReader(`{ "grant_type": "client_credentials" }`)
cred := base64.StdEncoding.EncodeToString([]byte(user + ":" + pass))
req, _ := http.NewRequest("POST", authUrl, payload)
req.Header.Add("Content-Type", "application/json")
req.Header.Add("Authorization", "Basic "+cred)
body, _ := sendReq(client, req)
tokenRes := Token{}
parseJSON([]byte(body), &tokenRes)
token := tokenRes.Access_Token
// get ports
portOffset := 0
portTotal := 1
var portRes Ports
// Keep reading until all pages have been retrieved
for portOffset < portTotal {
portBody := portReader(portOffset)
req, _ = http.NewRequest("POST", findUrl, portBody)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
body, _ = sendReq(client, req)
var p Ports
parseJSON([]byte(body), &p)
portRes.NspInventoryOutput.Data = append(portRes.NspInventoryOutput.Data, p.NspInventoryOutput.Data...)
portOffset = p.NspInventoryOutput.EndIndex + 1
portTotal = p.NspInventoryOutput.TotalCount
}
// get lags from port information to determine membership
var lagDetails = make(map[string]map[string][]string)
for i := 0; i < len(portRes.NspInventoryOutput.Data); i++ {
if len(portRes.NspInventoryOutput.Data[i].LagMemberDetails) > 0 {
for j := 0; j < len(portRes.NspInventoryOutput.Data[i].LagMemberDetails); j++ {
n := portRes.NspInventoryOutput.Data[i].NeID
l := portRes.NspInventoryOutput.Data[i].LagMemberDetails[j].LagId
p := portRes.NspInventoryOutput.Data[i].Name
_, ok := lagDetails[n]
if ok {
lagDetails[n][l] = append(lagDetails[n][l], p)
} else {
lagDetails[n] = map[string][]string{l: {p}}
}
}
}
}
fmt.Println("Lag map: ", lagDetails)
// Get LAGs
req, _ = http.NewRequest("GET", lagUrl, nil)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
body, _ = sendReq(client, req)
var lagRes Lag
parseJSON([]byte(body), &lagRes)
for i := 0; i < len(lagRes.NspEquipmentLag); i++ {
fmt.Print("NeID: ", lagRes.NspEquipmentLag[i].NeID, " Name: ", lagRes.NspEquipmentLag[i].LagID, "\n")
}
// Get Cables
req, _ = http.NewRequest("GET", linkUrl, nil)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
body, _ = sendReq(client, req)
var cableRes Cable
parseJSON([]byte(body), &cableRes)
cableAssoc := linkEndpointAssoc(cableRes)
fmt.Println(cableAssoc)
// GET IETF L3 topology
req, _ = http.NewRequest("GET", ietfNetwork, nil)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
req.URL.RawQuery = url.Values{"depth": {"2"}}.Encode()
body, _ = sendReq(client, req)
var ietfRes Ietf
parseJSON([]byte(body), &ietfRes)
// declare structures that will contain topology information
topoNode := make(map[string][]string)
topoLink := make(map[string][]map[string]map[string]string)
for i := 0; i < len(ietfRes.IetfNetworkNetwork); i++ {
// ignore SAP and L2 topologies
if ietfRes.IetfNetworkNetwork[i].NetworkID == "SAPTopology" || ietfRes.IetfNetworkNetwork[i].NetworkID == "L2Topology" {
continue
}
// Get nodes participating in the IGP
u := ietfNetwork + "=" + ietfRes.IetfNetworkNetwork[i].NetworkID
req, _ = http.NewRequest("GET", u+"/node", nil)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
req.URL.RawQuery = url.Values{"fields": {"node-id"}}.Encode()
var nodes IetfNetworkNode
body, _ = sendReq(client, req)
parseJSON([]byte(body), &nodes)
// add nodes to node structure, grouped by topology ID
for j := 0; j < len(nodes.Node); j++ {
topoNode[ietfRes.IetfNetworkNetwork[i].NetworkID] = append(topoNode[ietfRes.IetfNetworkNetwork[i].NetworkID], nodes.Node[j].NodeID)
}
fmt.Println(topoNode)
// Get links in the IGP
req, _ = http.NewRequest("GET", u+"/ietf-network-topology:link", nil)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
var links IetfNetworkNetwork
body, _ = sendReq(client, req)
parseJSON([]byte(body), &links)
re := regexp.MustCompile(`\[node-id=(.*)\]`)
// add links with source and destination to link structure, grouped by topology ID
for j := 0; j < len(links.IetfNetworkTopologyLink); j++ {
srcFDN := links.IetfNetworkTopologyLink[i].Source.SourceNode
src := re.FindStringSubmatch(srcFDN)[1]
destFDN := links.IetfNetworkTopologyLink[i].Source.SourceNode
dest := re.FindStringSubmatch(destFDN)[1]
tl := map[string]map[string]string{links.IetfNetworkTopologyLink[j].LinkID: {"source": src, "dest": dest}}
topoLink[ietfRes.IetfNetworkNetwork[i].NetworkID] = append(topoLink[ietfRes.IetfNetworkNetwork[i].NetworkID], tl)
}
}
// print out node topology structure
for key, val := range topoNode {
fmt.Println("Topology: ", key, "contains nodes: ", val)
}
// print out link topology structure
b, _ := json.MarshalIndent(topoLink, "", " ")
fmt.Println(string(b))
// Revoke client token
payload = strings.NewReader("token=" + token + "&token_type_hint=token")
req, _ = http.NewRequest("POST", revUrl, payload)
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
req.Header.Add("Authorization", "Basic "+cred)
_, _ = sendReq(client, req)
}
// send HTTP request
func sendReq(c *http.Client, req *http.Request) ([]byte, error) {
res, err := c.Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
body, err := io.ReadAll(res.Body)
if err != nil {
return nil, err
}
return body, nil
}
// parse JSON into relevant struct
func parseJSON(body []byte, res any) {
err := json.Unmarshal(body, &res)
if err != nil {
panic(err)
}
}
// create map of endpoints that are cabled together
func linkEndpointAssoc(c Cable) map[LinkEndpointKey]LinkEndpointKey {
var list = make(map[LinkEndpointKey]LinkEndpointKey)
for i := 0; i < len(c.NspServiceCable); i++ {
siteA := c.NspServiceCable[i].LinkEndpoint[0].SiteID
siteB := c.NspServiceCable[i].LinkEndpoint[1].SiteID
portA := c.NspServiceCable[i].LinkEndpoint[0].EndpointID
portB := c.NspServiceCable[i].LinkEndpoint[1].EndpointID
a := LinkEndpointKey{neID: siteA, portID: portA}
b := LinkEndpointKey{neID: siteB, portID: portB}
list[a] = b
list[b] = a
}
return list
}
// generate payload for next page of ports to retrieve
func portReader(o int) io.Reader {
return strings.NewReader(`{"nsp-inventory:input": {
"xpath-filter": "/nsp-equipment:network/network-element/hardware-component/port",
"depth": "2",
"limit": 10,
"offset": ` + strconv.Itoa(o) + `,
"fields": "ne-id;admin-state;oper-state;name;lag-member-details/lag-id;port-details(port-mode;port-type;actual-rate;mtu-value;encap-type)"
} } `)
}
func LoadTlsConfig(caCertFilePath string) (*tls.Config, error) {
// load the TLS certs from the specified file and return a TLS config
tlsConfig := tls.Config{}
caCert, err := os.ReadFile(caCertFilePath)
if err != nil {
return &tlsConfig, err
}
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
tlsConfig.RootCAs = caCertPool
tlsConfig.BuildNameToCertificate()
return &tlsConfig, err
}In order to keep up to date with changes in the network, a client must consume Kafka messages. The "nsp-yang-model.change-notif" topic can be used to monitor for relevant changes.
Each message on this topic contains a header that indicates the class for which the message is applicable. In this way, a consumer can ignore messages that are not of interest without parsing the entire payload.
For this tutorial, the following classes are used:
By comparing the "class-id" header to this list, irrelevant messages can be dropped.
The following source code shows an example of how to connect to Kafka and to filter messages based on the "class-id" header.
package main
import (
"bytes"
"context"
"crypto/tls"
"crypto/x509"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"os"
"os/signal"
"regexp"
"syscall"
"time"
"github.com/segmentio/kafka-go"
)
const (
topic = "nsp-yang-model.change-notif"
brokerAddress = "<<server>>:9192"
certFile = "ssl/nsp_external_combined.pem"
)
var r *kafka.Reader
func Classes() map[string]bool {
// list of classes that are of interest
// other classes, as identified in the Kafka header, will be ignored
return map[string]bool{
"/nsp-equipment:network/network-element/hardware-component/port": true,
"/nsp-equipment:network/network-element/hardware-component/port/port-details": true,
"/nsp-equipment:network/network-element/hardware-component/card": true,
"/nsp-equipment:network/network-element/lag": true,
"/nsp-equipment:network/network-element/lag/multi-chassis-lag-member": true,
"/nsp-service:services/physical-layer/cable": true,
"/ietf-network:networks/network/ietf-network-topology": true,
}
}
func handler(signal os.Signal) {
// clean up the Kafka consumer when exiting
fmt.Println("Closing reader")
if err := r.Close(); err != nil {
log.Fatal("failed to close reader:", err)
}
os.Exit(0)
}
func main() {
// create a new context
ctx := context.Background()
sigchnl := make(chan os.Signal, 1)
signal.Notify(sigchnl, syscall.SIGINT, syscall.SIGTERM)
go func() {
for {
s := <-sigchnl
handler(s)
}
}()
consume(ctx)
}
func consume(ctx context.Context) {
// open file for log messages. Note, no cleanup or file maintenance is performed
readerLogFile, err := os.OpenFile("log.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if err != nil {
panic(err)
}
l := log.New(readerLogFile, "kafka reader: ", 0)
// load TLS certificates
tlsConfig, err := LoadTlsConfig(certFile)
if err != nil {
log.Fatal(err)
}
dialer := &kafka.Dialer{
Timeout: 10 * time.Second,
DualStack: true,
TLS: tlsConfig,
}
// initialize a new reader with the brokers and topic
r = kafka.NewReader(kafka.ReaderConfig{
Brokers: []string{brokerAddress},
Topic: topic,
GroupID: "my-group",
// assign the logger to the reader
Logger: l,
Dialer: dialer,
MinBytes: 10,
MaxBytes: 1e9,
MaxWait: 60 * time.Second,
})
classes := Classes()
re := regexp.MustCompile(`\/ietf-network:networks\/network\/ietf-network-topology`)
for {
// the `ReadMessage` method blocks until we receive the next event
msg, err := r.ReadMessage(ctx)
if err != nil {
panic("could not read message " + err.Error())
}
// look at headers of message and check the class-id
// if a class of interest then print out the msg value
for _, header := range msg.Headers {
if string(header.Key) == "class-id" {
if classes[string(header.Value)] || re.Match([]byte(header.Value)) {
var pJSON bytes.Buffer
fmt.Println(string(header.Value))
err := json.Indent(&pJSON, msg.Value, "", " ")
if err != nil {
log.Fatal(err)
}
fmt.Println(&pJSON)
}
}
}
}
}
func LoadTlsConfig(caCertFilePath string) (*tls.Config, error) {
// load the TLS certs from the specified file and return a TLS config
tlsConfig := tls.Config{}
caCertFileDir := caCertFilePath
caCert, err := ioutil.ReadFile(caCertFileDir)
if err != nil {
return &tlsConfig, err
}
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
tlsConfig.RootCAs = caCertPool
tlsConfig.BuildNameToCertificate()
return &tlsConfig, err
}