Introduction to Segment Routing

Segment Routing in IOS-XE

Introduction

Segment Routing is a new architecture that leverages source routing based tunneling technique and allows an edge (Ingress) router to encode a list of segments as packet header. Each Segment can be considered as an instruction on how devices should process the packet. Segments could refer to the forwarding instruction of sending a packet towards a node, over a specific link, or towards any service application. The list of segments becomes the path the Ingress router instructs the network on how to treat/forward the packet. Because the information of the path that the packet has to traverse is included in the packet, intermediate routers do not have to maintain state for all possible paths that the network offers.

In this document, we provide an overview of the segment routing technology followed by the configuration and PI troubleshooting details in IOS-XE platforms.

Segment Routing Architecture

Figure 1. Segment Routing Network

In this Section, we describe the main components of the Segment Routing Architecture. The topology above depicts a network that offers different path that satisfies traffic constraints. For example, large file transfer traffic from CE1 to CE2 requires High Bandwidth path while voice traffic from CE1 to CE2 require low latency path. Accordingly, the network allows the file transfer traffic to flow over CE1-R2-R3-R5-R7-CE2 while voice traffic will flow over CE1-R2-R3-R4-R6-R5-R7-CE2. In a traditional MPLS network, this requires pre instantiated TE tunnel from R2 to R7 over different paths creating state entries in all transit nodes along the path.

With Segment Routing, R2 can define the path a packet should traverse by inserting a list of segments to be executed by the network. The state entry is carried in the packet itself eliminating the need for intermittent transit nodes to maintain the state entries. The control plane of Segment Routing defines different Segment Identifiers (Segment ID) and how the segment information is communicated to all devices in the network. Segment Routing can be applied on MPLS dataplane without any changes in dataplane. The segments are equivalent to labels. The main two types of Segments are as below:

Prefix Segment ID (aka Node Segment ID) – The forwarding semantic associated with Node Segment ID is to forward the packet on the shortest path towards the Node to which the Segment ID is assigned. Each node in Segment Routing network will be assigned with a minimum of one domain wide Prefix Segment. This can be done manually (like assigning IP address) or using a centralized controller.

Adjacency Segment ID – The forwarding semantic associated with an Adjacency Segment ID is to POP the segment and forward the packet over the corresponding adjacency. Each router will assign a locally significant segment ID for each of its IGP adjacencies.

Assigning Segment Identifier

To maintain domain wide uniqueness for Prefix Segment ID, it is the Operators responsibility to assign the Prefix SID without overlapping with other nodes in the domain. Since each node will dynamically assign Adj-SID, it should not overlap with Prefix SID. To achieve this, a block of label will be carved out on each Segment Routing router from platform label space. This block is known as Segment Routing Global Block (SRGB).  The default range of this block in Cisco platform will be 16000 to 23999 (a range of 8000 labels).

Figure 2. SR network Segment ID

In the above topology, each routers are assigned with a domain wide unique Prefix Segment ID from SRGB. There are 2 ways of configuring Prefix Segment ID as below:

·      Absolute Value based Prefix SID

·      Index based Prefix SID

In case of Absolute Value, the Prefix SID will be defined as a value. For example, on R2, the Prefix SID for loopback address will be configured as “16002” and advertise via IGP (with V flag set). Any other node will use 16002 as the label to reach R2 loopback address. Currently this is not supported in IOS-XE.

In case of Index based approach, the Prefix SID will be defined as Index and this needs to be unique for each node. For example on R2, the Index for loopback address will be configured as “2” and will be advertised via IGP (with V flag unset). Any other node will perform the below to identify the Prefix Segment ID of R2 loopback address:

In our topology, Prefix SID for R2 will be 16000 + 2 making it as 16002. Other nodes will use this value as label to reach R2 loopback address. From above topology, it could be observed that the Adj-SID {2003, 2004, 3004 ..} are assigned outside SRGB block and will not overlap with Prefix SID. Since Adj-SID have node level uniqueness, it could be observed that same Adj-SID can be assigned by different node for different IGP adjacency. For example, R2 assigned 2003 for R3 while R3 assigned 2003 for R5.

Segment Routing relies on link state IGP protocol to advertise the Segments to other nodes eliminate the need to manage multiple control plane protocols (like LDP, RSVP etc). ISIS and OSPF, the most popular IGP protocols in services provider networks, were extended to support the advertisement of segment identifiers.

Note:

Support for Segment Routing ISIS extension is introduced in XE3.16. Support for OSPF is introduced in XE3.17

ISIS Segment Routing Extension

Draft-ietf-isis-segment-routing-extensions defines a new ISIS Router capability as “SR Capabilities sub-TLV” to advertise the SR data plane capabilities and the SRGB block from where the Prefix SID will be assigned. The format is as below:

Each router will include Prefix SID Sub-TLV while advertising the prefix via LSP. The format is as below:

Ø  R-Flag (Re-advertisement Flag): Set when the Prefix is propagated from other level or redistributed into ISIS.

Ø  N-Flag (Node SID): Set when this prefix is assigned with a segment from SRGB.

Ø  P-Flag (non-PHP Flag): When set, directly connected (or the PHP router) will not POP the label. By default, this flag will be unset and so will be treated as if the prefix is received with imp-null label.

Ø  E-Flag (Explicit-null Flag): If set, the PHP router will swap the label with Exp-null label.

Ø  V-Flag (Value Flag): Set when the SID is configured as Absolute Value. When Prefix-SID is configured as Index, this flag is set to 0.

Ø  L-Flag (Local Flag): This flag will be set for Adj-SID or any other Segment ID with local significance. Prefix SID that have global significance will have this flag unset.

Each router will also assign Adj-SID for each IGP adjacency and advertise via Adj-SID Sub-TLV. The format is as below:

Ø  F-Flag (Address-Family Flag): Set to 0, when Adj-SID refers to IPv4 IGP adjacency and set to 1, when Adj-SID refers to IPv6 IGP adjacency.

Ø  B-Flag (Backup Flag): Set to 1, when Adj-SID is eligible for FRR protection using LFA-FRR or MPLS-FRR.

Ø  V-Flag (Value Flag): Set when the SID is configured as Absolute Value. For Adj-SID, this flag will always be set.

Ø   L-Flag (Local Flag): This flag will be set for Adj-SID or any other Segment ID with local significance. Adj-SID has local significance and so Adj-SID will have this flag always set.

Ø  S-Flag (Set Flag):

Configuring Segment Routing

 

Programming the forwarding table

Each router should derive the Incoming and Outgoing label for each Prefix to populate the Label forwarding table. Each router will follow the below procedure to derive the Incoming and Outgoing label for Prefix SID:

Incoming Label

Ø  If the Prefix SID is assigned as “Absolute Value”, use the same as Incoming Label. Else,

Ø  Get the Index advertised by Originating router for Prefix P from IGP database.

Ø  Add the first label of local SRGB.

Ø  Use the resulting value as Incoming label for prefix P in Label forwarding table.

Outgoing Label

Ø  If the Prefix SID is assigned as “Absolute Value”, use the same as “Potential Outgoing Label”. Else,

Ø  Get the Index advertised by Originating router for Prefix P from IGP database.

Ø  Identify the next hop of the best path to P. Get the first label in SRGB range advertised by NextHop.

Ø  Add the first label of SRGB to Index.  The resulting value will be the “Potential Outgoing Label”.

Ø  If the NextHop address and Prefix P belong to different router, set Outgoing Label as “Potential Outgoing Label” value. The action will be “SWAP”.

Ø  If the NextHop address and Prefix P belongs to same router, set Outgoing Label as “imp-null”. The action will be “POP”.

Using the same range of SRGB block in all routers within the SR domain helps with ease of Operation and troubleshooting and is considered as a best practice. So it is common to see same value used as Incoming and Outgoing label.

While any SR node within IGP domain have the Adj-SID assigned/advertised by all other nodes, it only installs the locally assigned Adj-SID in the forwarding table. Each router will follow the below procedure to derive the Incoming and Outgoing label for Adj-SID:

Incoming Label

Ø  Use the locally assigned Adj-SID as Incoming label in Label forwarding table.

Outgoing Label

Ø  Set the action as “POP”.

Ø  Populate the outgoing interface as interface over which IGP adjacency is established.

Verification

·      Check the basic Segment Routing configuration and state on local router

R2#show segment-routing mpls state

 Segment Routing MPLS State : ENABLED

R2#

R2#show segment-routing mpls connected-prefix-sid-map ipv4

Prefix/masklen       Index Range in-SRGB

       10.1.2.2/32       2     1    Y

R2#

R2#show isis segment-routing

 ISIS protocol is registered with MFI

 ISIS MFI Client ID:0x63

 Tag Null - Segment-Routing:

   SR State:SR_ENABLED

   Number of SRGB:1

   SRGB Start:16000, Range:8000, srgb_handle:0xF5DABB24, srgb_state: created

   Address-family IPv4 unicast SR is configured

     Operational state:Enabled

     Receive is enabled

     Advertise local is enabled

     Explicit null is disabled

     SR label preferred is disabled

R2#

Use “show isis database <> details” to check if Router capabilities are advertised with Segment Routing.

R2#show isis database R2.00-00 detail

Tag null:

IS-IS Level-1 LSP R2.00-00

LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL

R2.00-00            * 0x00000085   0x28D7        658               0/0/0

  Area Address: 49

  NLPID:        0xCC

  Router CAP:   10.1.2.2, D:0, S:0

    Segment Routing: I:1 V:0, SRGB Base: 16000 Range: 8000

  Hostname: R2

  Metric: 10         IS-Extended R4.01

  Metric: 10         IS-Extended R3.01

  IP Address:   10.1.2.2

  Metric: 10         IP 10.1.2.2/32

  Metric: 10         IP 10.1.23.0/24

  Metric: 10         IP 10.1.24.0/24

R2#

It could be observed that SR Capabilities Sub-TLV is advertised by R2 with IPv4 MPLS flag (I flag) set IPv6 MPLS (V flag) flag unset.

·      Check if the Prefix SID advertised/received with the right value.

R2#show isis database R2.00-00 verbose

Tag null:

IS-IS Level-1 LSP R2.00-00

LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL

R2.00-00            * 0x0000000E   0x3D23        868               0/0/0

  Area Address: 49

  NLPID:        0xCC

  Router CAP:   10.1.2.2, D:0, S:0

    Segment Routing: I:1 V:0, SRGB Base: 16000 Range: 8000

  Hostname: R2

  Metric: 10         IS-Extended R3.01

    Lan Adjacency SID:

      SID Value:30, R3.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  Metric: 10         IS-Extended R4.01

    Lan Adjacency SID:

      SID Value:31, R4.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  IP Address:   10.1.2.2

  Metric: 10         IP 10.1.2.2/32

    Prefix-SID Index: 2, Algorithm:0, R:0 N:1 P:0 E:0 V:0 L:0

  Metric: 10         IP 10.1.23.0/24

  Metric: 10         IP 10.1.24.0/24

R2#

It could be observed that R2 advertises prefix 10.1.2.2/32 with Index of 2. The Node-SID Flag is set and rest other flags are unset in the prefix SID. Also it could be observed that R2 assign label 30 as Adj-SID for R3 and label 31 as Adj-SID for R4. The Value Flag and Local Flag are set and other flags are unset in the Adj-SID.

·      Check if the forwarding table is installed with right label forwarding information

R2#show isis database R3.00-00 verbose

Tag null:

IS-IS Level-1 LSP R3.00-00

LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL

R3.00-00              0x0000000D   0x33CD        1194              0/0/0

  Area Address: 49

  NLPID:        0xCC

  Router CAP:   10.1.3.3, D:0, S:0

    Segment Routing: I:1 V:0, SRGB Base: 16000 Range: 8000

  Hostname: R3

  Metric: 10         IS-Extended R3.01

    Lan Adjacency SID:

      SID Value:26, R2.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  Metric: 10         IS-Extended R4.02

    Lan Adjacency SID:

      SID Value:27, R4.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  Metric: 10         IS-Extended R5.01

    Lan Adjacency SID:

      SID Value:28, R5.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  IP Address:   10.1.3.3

  Metric: 10         IP 10.1.3.3/32

    Prefix-SID Index: 3, Algorithm:0, R:0 N:1 P:0 E:0 V:0 L:0

  Metric: 10         IP 10.1.23.0/24

  Metric: 10         IP 10.1.34.0/24

  Metric: 10         IP 10.1.35.0/24

R2#

R2 fetches the Index advertised by R3 (Originator) for 10.1.3.3/32 as 3, add the value to first label from local SRGB which is 16000. 16003 will be used as Incoming label for 10.1.3.3. Now R2 identifies the NextHop to reach 10.1.3.3 as 10.1.23.3 (R3) from SPF calculation. Since 10.1.3.3/32 and 10.1.23.3 belongs to same router, the Outgoing will be set to imp-null. Altogether on R2, the forwarding table for 10.1.3.3 should be populated with Incoming label as 16003, Outgoing label as imp-null and action as POP.

R2#show mpls forwarding-table 10.1.3.3

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop

Label      Label      or Tunnel Id     Switched      interface

16003      Pop Label  10.1.3.3/32      0             Et0/0.23   10.1.23.3

R2#

Below is another example:

R2#show isis database R6.00-00 verbose

Tag null:

IS-IS Level-1 LSP R6.00-00

LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL

R6.00-00              0x0000000F   0x4171        1190              0/0/0

  Area Address: 49

  NLPID:        0xCC

  Router CAP:   10.1.6.6, D:0, S:0

    Segment Routing: I:1 V:0, SRGB Base: 16000 Range: 8000

  Hostname: R6

  Metric: 10         IS-Extended R7.02

    Lan Adjacency SID:

      SID Value:28, R7.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  Metric: 10         IS-Extended R6.03

    Lan Adjacency SID:

      SID Value:27, R4.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  IP Address:   10.1.6.6

  Metric: 10         IP 10.1.6.6/32

    Prefix-SID Index: 6, Algorithm:0, R:0 N:1 P:0 E:0 V:0 L:0

  Metric: 10         IP 10.1.46.0/24

  Metric: 10         IP 10.1.56.0/24

  Metric: 10         IP 10.1.67.0/24

R2#

On the same line, R2 fetches the Index advertised by R6 (Originator) for 10.1.6.6/32 as 6, add the value to first label from local SRGB which is 16000. 16006 will be used as Incoming label for 10.1.7.7. Now R2 identifies the NextHop to reach 10.1.6.6 as 10.1.34.4 (R4) from SPF calculation. Since 10.1.6.6/32 and 10.1.34.4 belongs to different router, R2 will get the first label of SRGB advertised by R4 (NextHop), add Index 6. The resulting value 16006 will be used as Outgoing label and the action will be marked as “SWAP”.

R2#show mpls forwarding-table 10.1.6.6

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop

Label      Label      or Tunnel Id     Switched      interface

16006      16006      10.1.6.6/32      0             Et0/0.24   10.1.24.4

R2#

·      Check the Adj-SID programming in forwarding table

R2 assign locally unique Adj-SID (outside SRGB) for each ISIS adjacency. In our topology, R2 have 2 neighbors as R3 and R4. We can either check the Adj-SID in LSP or using neighborship output.

Below is an example to check the Adj-SID from LSP:

R2#show isis database R2.00-00 verbose

Tag null:

IS-IS Level-1 LSP R2.00-00

LSPID                 LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL

R2.00-00            * 0x00000120   0x1637        696               0/0/0

  Area Address: 49

  NLPID:        0xCC

  Router CAP:   10.1.2.2, D:0, S:0

    Segment Routing: I:1 V:0, SRGB Base: 16000 Range: 8000

  Hostname: R2

  Metric: 10         IS-Extended R3.01

    Lan Adjacency SID:

      SID Value:30, R3.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  Metric: 10         IS-Extended R4.01

    Lan Adjacency SID:

      SID Value:31, R4.00, F:0 B:0 V:1 L:1 S:0 Weight:0

  IP Address:   10.1.2.2

  Metric: 10         IP 10.1.2.2/32

    Prefix-SID Index: 2, Algorithm:0, R:0 N:1 P:0 E:0 V:0 L:0

  Metric: 10         IP 10.1.23.0/24

  Metric: 10         IP 10.1.24.0/24

R2#

Alternately, we could check ISIS neighborship to get the respective Adj-SID:

R2#show isis neighbors detail

Tag null:

System Id       Type Interface     IP Address      State Holdtime Circuit Id

R3              L1   Et0/0.23      10.1.23.3       UP    8        R3.01

  Area Address(es): 49

  SNPA: aabb.cc03.eb00

  State Changed: 2d14h

  LAN Priority: 64

  Format: Phase V

  Remote TID: 0

  Local TID:  0

  Interface name: Ethernet0/0.23

  Adjacency SID Value: 30 f:0 b:0 v:1 l:1 s:0 weight:0

R4              L1   Et0/0.24      10.1.24.4       UP    7        R4.01

  Area Address(es): 49

  SNPA: aabb.cc03.ec00

  State Changed: 2d14h

  LAN Priority: 64

  Format: Phase V

  Remote TID: 0

  Local TID:  0

System Id       Type Interface     IP Address      State Holdtime Circuit Id

  Interface name: Ethernet0/0.24

  Adjacency SID Value: 31 f:0 b:0 v:1 l:1 s:0 weight:0

R2#

From above outputs, Incoming label 31 will be programmed with imp-null as Outgoing label and nexthop as R4. Similarly, incoming label 30 will be programmed with imp-null as Outgoing label and nexthop as R3.

R2#show mpls forwarding-table labels 30

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop

Label      Label      or Tunnel Id     Switched      interface

30         Pop Label  0000.0000.3333-Et0/0.23-10.1.23.3   \

                                       0             Et0/0.23   10.1.23.3

R2#show mpls forwarding-table labels 31

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop

Label      Label      or Tunnel Id     Switched      interface

31         Pop Label  0000.0000.4444-Et0/0.24-10.1.24.4   \

                                       0             Et0/0.24   10.1.24.4

R2#

LSP Ping is the powerful OAM tool used for LSP path validation. MPLS Echo Request carries FEC details in the payload that will be used by egress for validation. Draft-kumarkini-mpls-spring-lsp-ping defines the LSP Ping extension for Segment Routing. This is not yet deployed and so currently we have to use generic FEC type while using LSP Ping. When the FEC type is not defined in CLI, it tries to query the FEC and throws error message as “no FEC mapping” as below:

R2#ping mpls ipv4 10.1.7.7 255.255.255.255

Sending 5, 72-byte MPLS Echos to 10.1.7.7/32,

     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,

  'L' - labeled output interface, 'B' - unlabeled output interface,

  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,

  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,

  'P' - no rx intf label prot, 'p' - premature termination of LSP,

  'R' - transit router, 'I' - unknown upstream index,

  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,

  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.

FFFFF

Success rate is 0 percent (0/5)

 Total Time Elapsed 10 ms

By using generic FEC type, it carries the IP address in TLV and validates the path as below:

R2#ping mpls ipv4 10.1.7.7 255.255.255.255 fec-type generic

Sending 5, 72-byte MPLS Echos to Generic FEC 10.1.7.7/32,

     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,

  'L' - labeled output interface, 'B' - unlabeled output interface,

  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,

  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,

  'P' - no rx intf label prot, 'p' - premature termination of LSP,

  'R' - transit router, 'I' - unknown upstream index,

  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,

  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms

 Total Time Elapsed 7 ms

R2#

Currently for testing purpose, we can use nil-FEC based validation by statically defining the stack of label to be imposed to the LSP Ping packet as below:

R2#ping mpls nil-fec labels 16006 output interface e0/0.23 nexthop 10.1.23.3

Sending 5, 72-byte MPLS Echos with Nil FEC labels 16006,

     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,

  'L' - labeled output interface, 'B' - unlabeled output interface,

  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,

  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,

  'P' - no rx intf label prot, 'p' - premature termination of LSP,

  'R' - transit router, 'I' - unknown upstream index,

  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,

  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms

 Total Time Elapsed 7 ms

R2#

-->