Saturday, July 10, 2010

MPLS LSP Ping

MPLS LSP Ping is one of the powerful troubleshooting mechanisms to detect data plane failure in MPLS Label Switched Path.

Consider the below topology for further reading,






With the above topology, Normal ICMP Ping tool works as below,

When normal ICMP echo is originated from R5 to 150.1.6.6, the ICMP packet will have the following details,



With the above details, even if the MPLS LSP is not established, we can reach the destination, as all intermittent routers will be aware of the internal network addresses.

If normal ICMP ping is originated when LSP is broken between R4 and R6, R4 will have to look into the destination address of the packet and perform RIB lookup and will be IP Forwarded. So with normal ICMP ping, we don’t have any way to detect the data plane failure in end to end LSP.

How MPLS Ping works?

MPLS Ping doesn’t rely on ICMP echo messages. Instead it uses UDP protocol with both source and destination port as 3503 and relies on MPLS Echo request and MPLS Echo reply.

When MPLS ping is triggered from any MPLS router, it will generate UDP segment with source/destination port as 3503. The source address will be selected as usual while the destination address will be 127.0.0.1. The IP TTL will be set to 1.

Below is a sample IP format when MPLS Ping is originated from R5 to 150.1.6.6/32,




Now the originating LSR will look into the LFIB and populate the label header with respective labels to reach the FEC, in our case 150.1.6.6/32.

R4 on receiving the MPLS packet will be able to send to the actual destination only if the LSP is end to end. If the LSP is broken between R4 and R6, R4 will look into the destination IP address which will be 127.0.0.1 and won’t be able to perform IP forwarding. As per RFC 1812, a router should not forward any packet that has destination address of 127.0.0.0/8

Below is few simple testcase examples which will help us better understand how MPLS Ping works.


Testcase 1:

End to end LSP is established from R5 to R6 and is capable of Label switching packets. Now when MPLS Ping is originated from R5 to 150.1.6.6/32, it will be sent out with respective label(s) as per LFIB. R6 on receiving it, will respond back with MPLS echo reply (Message Type=02) with UDP source/destination port as 3503.

R5#show mpls forwarding-table 150.1.6.6
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or VC or Tunnel Id Switched interface
19 18 150.1.6.6/32 0 Et0/0.45 150.1.45.4
R5#ping mpls ipv4 150.1.6.6 255.255.255.255
Sending 5, 100-byte MPLS Echos to 150.1.6.6/32,
timeout is 2 seconds, send interval is 0 msec:
Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'L' - labeled output interface, 'B' - unlabeled output interface,
'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,
'P' - no rx intf label prot, 'p' - premature termination of LSP,
'R' - transit router, 'I' - unknown upstream index,
'X' - unknown return code, 'x' - return code 0
Type escape sequence to abort.
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/21/32 ms
R5#

Testcase 2:
LSP is fine between R5 and R4, But LDP is disabled in R4 resulting in broken LSP. Now R4 will have to perform IP lookup as the LFIB will not have any outgoing label. Assuming R4 is loaded with IOS version which supports MPLS ping, it will understand that the packet is MPLS ping which needs to be label switched while the outgoing interface is not enabled with LDP. So it will send an error message to ultimate source with Return Code=09 and subcode=01 (See error code details in RFC 4379).
R5#show mpls forwarding-table 150.1.6.6
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or VC or Tunnel Id Switched interface
19 18 150.1.6.6/32 0 Et0/0.45 150.1.45.4
R5#ping mpls ipv4 150.1.6.6 255.255.255.255
Sending 5, 100-byte MPLS Echos to 150.1.6.6/32,
timeout is 2 seconds, send interval is 0 msec:
Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'L' - labeled output interface, 'B' - unlabeled output interface,
'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,
'P' - no rx intf label prot, 'p' - premature termination of LSP,
'R' - transit router, 'I' - unknown upstream index,
'X' - unknown return code, 'x' - return code 0
Type escape sequence to abort.
BBBBB
Success rate is 0 percent (0/5)
R5#

Testcase 3:
Above same testcase is tried with R4 loaded with IOS version which doesn’t support MPLS Ping. So R4 will not understand the MPLS Ping traffic and will drop on looking into the destination IP address. So no error messages will be received on R5,

R5#show mpls forwarding-table 150.1.6.6
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or VC or Tunnel Id Switched interface
19 17 150.1.6.6/32 0 Et0/0.45 150.1.45.4
R5#ping mpls ipv4 150.1.6.6 255.255.255.255
Sending 5, 100-byte MPLS Echos to 150.1.6.6/32,
timeout is 2 seconds, send interval is 0 msec:
Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'L' - labeled output interface, 'B' - unlabeled output interface,
'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,
'P' - no rx intf label prot, 'p' - premature termination of LSP,
'R' - transit router, 'I' - unknown upstream index,
'X' - unknown return code, 'x' - return code 0
Type escape sequence to abort.
.....
Success rate is 0 percent (0/5)

Points to remember:
  • MPLS Ping will use UDP protocol of source and destination as 3503. It will not use the normal ICMP messages.
  • Destination IP address of the MPLS ping packet will be from 127.0.0.0/8 network
  • IP TTL will be set to 1.
  • It is better to have all routers loaded with IOS which supports MPLS ping. If the destination doesn’t support MPLS ping, we may not receive the MPLS Echo reply and may assume that the LSP is broken.

15 comments:

  1. Thanks. I am working on a bug in LSP ping and your blog is surely helping me.

    ReplyDelete
  2. Thank you very much. This is surely a classic case of ingenuity.

    ReplyDelete
  3. Really Good work, nice explanation

    ReplyDelete
  4. Can I test a PW using LSP ping with packet size more than 1500 bytes. According to my understanding the packet size limit is from 64 to 1500 Bytes. I am not sure why this limit exists. Any comments?

    ReplyDelete
    Replies
    1. Masham,

      PW Ping is to validate the VC label and perform CV. So it is relevant only for the end points while MTU issue can be validated by transport LSP Ping.

      Delete
  5. nice explaination.you may want to mention about router-alert option in the ip packet.

    ReplyDelete
    Replies
    1. Why does the Router Alert Option matter for us? From my understanding if the "Router-Alert" option is set within "IP Options" it informs the router that it needs to look further into the IP Packet (i.e. into the MPLS Ping) for example

      Delete
    2. Right. The intention is to make sure that the packet is not IP forwarded and not leaked outside MPLS domain. This is achieved by setting the destination address as 127.0.0.x and Router Alert option set.

      Delete
  6. The source port is not 3503, it's randomly chosen. From RFC4379:

    The source UDP port is chosen by the sender;
    the destination UDP port is set to 3503 (assigned by IANA for MPLS
    echo requests). The Router Alert option MUST be set in the IP
    header.

    ReplyDelete
    Replies
    1. Thanks, that is right per RFC. While MPLS tree trace chooses different source ports, some implementation uses 3503 as source and destination ports by default.

      Delete
  7. Can you pls tell the Packet processing steps @R4 when Router Receive with Label 17 and no outgoing label for a FEC & How R4 know its MPLS ping ?

    ReplyDelete
  8. When R4 receives the packet with top label as 17 and if it does not have the same in the forwarding table, it extract and punt to CPU for error generation. While doing so, the CPU will recognize the packet as LSP Ping based on the UDP port number 3503.

    ReplyDelete