NX-OS and Cisco Nexus Switching (2^nd Ed.) Next-Generation Data Center Architectures Networking Technology Series

NX-OS and Cisco Nexus Switching

Next-Generation Data Center Architectures

Second Edition

The complete guide to planning, configuring, managing, and troubleshooting NX-OS in the enterprise?updated with new technologies and examples

Using Cisco Nexus switches and the NX-OS operating system, data center professionals can build unified core networks that deliver unprecedented scalability, resilience, operational continuity, flexibility, and performance. NX-OS and Cisco Nexus Switching, Second Edition, is the definitive guide to applying these breakthrough technologies in real-world environments. This extensively updated edition contains five new chapters addressing a wide range of new technologies, including FabricPath, OTV, IPv6, QoS, VSG, Multi-Hop FCoE, LISP, MPLS, Layer 3 on Nexus 5000, and Config sync. It also presents a start-to-finish, step-by-step case study of an enterprise customer who migrated from Cisco Catalyst to a Nexus-based architecture, illuminated with insights that are applicable in virtually any enterprise data center. Drawing on decades of experience with enterprise customers, the authors cover every facet of deploying, configuring, operating, and troubleshooting NX-OS in today?s data center. You?ll find updated best practices for high availability, virtualization, security,
L2/L3 protocol and network support, multicast, serviceability, provision of networking and storage services, and more. Best of all, the authors present all the proven commands, sample configurations, and tips you need to apply these best practices in your data center.

Ron Fuller, CCIE No. 5851 (Routing and Switching/Storage Networking), Technical Marketing Engineer on Cisco?s Nexus 7000 team, specializes in helping customers design end-to-end data center architectures. Ron has 21 years of industry experience, including 7 at Cisco. He has spoken at Cisco Live on VDCs, NX-OS multicast, and general design.

David Jansen, CCIE No. 5952 (Routing/Switching), is a Cisco Technical Solutions Architect specializing in enterprise data center architecture. He has 20 years of industry experience, 15 of them at Cisco (6 as a solution architect); and has delivered several Cisco Live presentations on NX-OS and data center solutions.

Matthew McPherson, senior systems engineer and solutions architect for the Cisco Central Select Operation, specializes in data center architectures. He has 12 years of experience working with service providers and large finance and manufacturing enterprises, and possesses deep technical knowledge of routing, switching, and security.

• Understand the NX-OS command line, virtualization features, and file system
• Utilize the NX-OS comprehensive Layer 2/Layer 3 support: vPC, Spanning Tree Protocol, Cisco FabricPath, EIGRP, OSPF, BGP, HSRP, GLBP, and VRRP
• Configure IP multicast with PIM, Auto-RP, and MSDP
• Secure your network with CTS, SGTs, ACLs, CoPP, and DAI
• Establish a trusted set of network devices with Cisco TrustSec
• Maximize availability with ISSU, stateful process restart/switchover, and non-stop forwarding
• Improve serviceability with SPAN, ERSPAN, configuration checkpoints/rollback, packet analysis, Smart Call Home, Python, and PoAP
• Unify storage and Ethernet fabrics with FCoE, NPV, and NPIV
• Take full advantage of Nexus 1000V in a virtualized environment
• Achieve superior QoS with MQ CLI, queuing, and marking
• Extend L2 networks across L3 infrastructure with Overlay Transport Virtualization (OTV)
• Deliver on SLAs by integrating MPLS application components such as L3 VPNs, traffic engineering, QoS, and mVPN
• Support mobility via the new Locator ID Separation Protocol (LISP)
• Walk step-by-step through a realistic Nexus and NX-OS data center migration

Foreword xxiii

Introduction xxiv

Chapter 1 Introduction to Cisco NX-OS 1

NX-OS Overview 1

NX-OS Supported Platforms 3

NX-OS Licensing 7

Nexus 7000 7

Nexus 5500 8

Nexus 3000 8

Nexus 2000 9

Nexus 1000v 9

Installing the NX-OS License File 9

Cisco NX-OS and Cisco IOS Comparison 10

NX-OS User Modes 12

EXEC Command Mode 12

Global Configuration Command Mode 13

Interface Configuration Command Mode 13

Management Interfaces 14

Controller Processor (Supervisor Module) 15

Connectivity Management Processor (CMP) 16

Telnet 18

SSH 19

SNMP 23

DCNM 26

Managing System Files 28

File Systems 28

Configuration Files: Configuration Rollback 33

Operating System Files 35

Virtual Device Contexts 37

VDC Configuration 43

VDC Interface Allocation 46

Interface Allocation: N7K-M132XP-12 and L 46

Interface Allocation: N7K-F132XP-15 47

Interface Allocation: N7K-M108X2-12L 48

Interface Allocation: 10/100/1000 Modules 48

Interface Allocation on M2 Modules 52

Troubleshooting 54

show Commands 54

debug Commands 55

Topology 56

Further Reading 57

Chapter 2 Layer 2 Support and Configurations 59

Layer 2 Overview 59

Store-and-Forward Switching 60

Cut-Through Switching 60

Fabric Extension via the Nexus 2000 60

Configuring Nexus 2000 Using Static Pinning 61

Nexus 2000 Static Pinning Verification 62

Configuring Nexus 2000 Using Port-Channels 66

Nexus 2000 Static Pinning Verification 67

Layer 2 Forwarding on a Nexus 7000 69

L2 Forwarding Verification 70

VLANs 71

Configuring VLANs 72

VLAN Trunking Protocol 72

Assigning VLAN Membership 73

Verifying VLAN Configuration 74

Private VLANs 76

Configuring PVLANs 77

Verifying PVLAN Configuration 80

Spanning Tree Protocol 80

Rapid-PVST+ Configuration 82

Verifying Spanning Tree State for a VLAN 83

Spanning Tree Timers 84

MST Configuration 87

Additional Spanning Tree Configuration 91

Port Cost 91

Port Priority 94

Spanning Tree Toolkit 94

BPDUGuard 94

BPDUFilter 95

RootGuard 96

LoopGuard 97

Dispute Mechanism 98

Bridge Assurance 98

Spanning Tree Port Types 99

Virtualization Hosts 100

Configuring Layer 2 Interfaces 100

Trunk Ports 100

Standard Host 101

Link to Virtualization Host 101

Port-Profiles 102

Port-Channels 103

Assigning Physical Ports to a Port-Channel 104

Port-Channel Flow Control 107

Verifying Load Distribution Across a Port-Channel 108

Virtual Port-Channels 109

vPC Peer-Gateway 116

vPC Peer-Switch 116

ARP Synchronization 117

Unidirectional Link Detection 118

Cisco FabricPath 119

vPC+ 127

Configuring vPC+ 127

Summary 133

Chapter 3 Layer 3 Support and Configurations 135

EIGRP 135

EIGRP Operation 136

Configuring EIGRP 137

EIGRP Summarization 142

EIGRP Stub Routing 145

Securing EIGRP 147

EIGRP Redistribution 149

OSPF 154

OSPFv2 Configuration 154

OSPF Summarization 160

OSPF Stub Routing 163

Securing OSPF 167

OSPF Redistribution 169

OSPFv3 Configuration 177

IS-IS 178

IS-IS Configuration 178

BGP 183

BGP Configuration 184

BGP Neighbors 187

Securing BGP 190

BGP Peer Templates 192

Advertising BGP Networks 194

Modifying BGP Routing Metrics 197

Verifying BGP-Specific Configuration 198

First Hop Redundancy Protocols 198

HSRP 199

HSRP Configuration 199

HSRP Priority and Preempt 200

Verifying the HSRP Configuration 201

Securing HSRP 202

HSRP Secondary Support 204

HSRP Support for IPv6 204

VRRP 205

VRRP Configuration 205

VRRP Priority and Preempt 207

Verifying VRRP Configuration 208

Securing VRRP 208

VRRP Secondary Support 209

HSRP, VRRP, and vPC Interactions 210

GLBP 212

GLBP Configuration 212

GLBP Priority and Preempt 214

Verifying GLBP Configuration 214

Securing GLBP 215

GLBP Secondary Support 218

Summary 220

Chapter 4 IP Multicast Configuration 221

Multicast Operation 221

Multicast Distribution Trees 222

Reverse Path Forwarding 225

Protocol Independent Multicast (PIM) 225

RPs 226

PIM Configuration on Nexus 7000 and Nexus 5500 227

Configuring Static RPs 230

Configuring BSRs 232

Configuring Auto-RP 235

Configuring Anycast-RP 237

Configuring SSM and Static RPF 239

IGMP Operation 241

IGMP Configuration on Nexus 7000 242

IGMP Configuration on Nexus 5000 245

IGMP Configuration on Nexus 1000V 246

MSDP Configuration on Nexus 7000 248

Administrative Scoping of Multicast RPs in PIM 250

Configuring PIM Join and Prune Policies 252

Multicast and Control Plane Policing (CoPP) 253

Summary 253

Chapter 5 Security 255

Configuring RADIUS 256

RADIUS Configuration Distribution 259

Configuring TACACS+ 266

Enabling TACACS+ 266

TACACS+ Configuration Distribution 267

Configuring the Global TACACS+ Keys 268

Configuring the TACACS+ Server Hosts 268

Configuring TACACS+ Server Groups 269

Configuring TACACS+ Source Interface 270

Configuring SSH 275

Cisco TrustSec 278

Configuring AAA for Cisco TrustSec 281

Defining Network Device Admission Control 282

Configuring the Nexus 7000 for 802.1x and SGA Features 285

SGT Assignment via ISE Server 288

Policy Component: IP to SGT Mapping 290

Policy Component: SGACL Creation 292

Configuring Cisco TrustSec: IEEE 802.1AE LinkSec 294

Layer 2 Solutions Between Data Centers 301

Configuring IP ACLs 302

Configuring MAC ACLs 305

Configuring VLAN ACLs 307

Configuring Port Security 308

Security Violations and Actions 311

Configuring DHCP Snooping 313

Configuring Dynamic ARP Inspection 316

Dynamic ARP Inspection Trust State 317

Configuring IP Source Guard 321

Configuring Keychain Management 322

Configuring Traffic Storm Control 323

Configuring Unicast RPF 325

Configuring Control Plane Policing 327

Configuring Rate Limits 335

SNMPv3 340

Summary 347

Chapter 6 High Availability 349

Physical Redundancy 349

Redundant Power Supplies 350

Redundant Cooling System 352

Redundant Supervisors 355

Redundant Ethernet Out-of-Band (EOBC) 357

Redundant Fabric Modules 357

Generic Online Diagnostics 358

Bootup Diagnostics 359

Runtime Diagnostics 360

On-Demand Diagnostics 365

NX-OS High-Availability Architecture 365

Process Modularity 366

Process Restart 368

Stateful Switchover 369

Nonstop Forwarding 370

In-Service Software Upgrades 370

Summary 383

Chapter 7 Embedded Serviceability Features 385

SPAN 386

SPAN on Nexus 7000 386

Configuring SPAN on Nexus 7000 387

SPAN on Nexus 5x00 392

Configuring SPAN on Nexus 5x00 393

SPAN on Nexus 1000V 397

Configuring SPAN on Nexus 1000V 398

ERSPAN on Nexus 1000V 400

ERSPAN on Nexus 7000 406

ERSPAN on Nexus 5x00 412

Embedded Analyzer 414

Smart Call Home 424

Smart Call Home Configuration 428

Configuration Checkpoint and Rollback on Nexus 7000 431

Checkpoint Creation and Rollback 432

Configuration Checkpoint and Rollback on Nexus 5x00 434

Checkpoint Creation and Rollback 435

NetFlow 437

Configuring NetFlow on Nexus 7000 438

Configuring NetFlow on Nexus 1000V 442

Network Time Protocol 444

Precision Time Protocol 445

IEEE 802.3az (Energy Efficient Ethernet) 447

Power On Auto-Provisioning 448

Python 449

Summary 454

Chapter 8 Unified Fabric 455

Unified Fabric Overview 455

Enabling Technologies 456

10-Gigabit Ethernet 456

Fibre Channel over Ethernet 458

Single-Hop Fibre Channel over Ethernet 461

Multhop Fibre Channel over Ethernet 462

Storage VDC on Nexus 7000 463

N-Port Virtualization 465

N-Port Identification Virtualization 466

FCoE NPV Mode 466

Nexus 5x00 Unified Fabric Configuration 467

Single-Hop FCoE Configuration: Nexus 5x00 469

FCoE-NPV on Nexus 5x00 473

Nexus 7000 Unified Fabric Configuration 477

Summary 488

Chapter 9 Nexus 1000V 489

Hypervisor and vSphere Introduction 489

Nexus 1000V System Overview 490

Nexus 1000V Switching Overview 494

Nexus 1000V VSM Installation 496

Nexus 1000V Deployed on Nexus 1010 Virtual Services Blade 497

Registering the Nexus 1000V Plug-In to VMware Virtual Center Management Application 502

Configuring the SVS Domain and Networking Characteristics 507

Connecting the Nexus 1000V VSM to the vCenter Server 508

Nexus 1000V Installation Management Center 510

VEM Installation Option on the Nexus 1000V Management Installation Center 519

vCenter Connection Option on the Nexus 1000V Management Installation Center 523

Creating the Uplink Profile 526

Adding the VEM to a ESX vSphere Host 528

Enabling the Telnet Server Process 536

Changing the VSM Hostname 536

Layer 3 Control 536

1000V Port Profiles 542

Virtual Network Management Center 552

Installing Virtual Network Management Center Software from OVA Downloaded from Cisco.com 553

Adding the VM-Manager for vCenter Connectivity in VNMC Management Application 564

Configuring the Cisco VNMC Policy-Agent on the 1000v VSM 570

Virtual Security Gateway 571

Install Virtual Security Gateway on the Nexus 1010 574

Configuring the Cisco VNMC Policy-Agent on the VSG 577

Verify That the VSG and VSM Are Registered Clients in VNMC 578

Creating a Tenant in VMMC 579

Virtual Extensible LAN 602

Deploying Virtual Extensible LAN 604

Nexus 1000v Network Analysis Module 629

Installing Nexus 1000v Network Analysis Module 630

Deploying the Nexus 1000v NAM as a Virtual Services Blade on the Nexus 1010 641

Summary 642

Chapter 10 Quality of Service (QoS) 643

QoS on Nexus 7000 646

Forwarding Architecture 646

Network-QoS Policies 648

Queuing Policies 650

QoS and Nexus 2000 Fabric Extenders 661

Ron Fuller

David Jansen

Matthew McPherson

is a senior systems engineer and solutions architect for Cisco in the Central Select Operation, specializing in data center architectures. Matt has been with Cisco for more than 2 1/2 years and has more than 12 years of experience in the industry working for service providers and large enterprise customers in the financial and manufacturing verticals. He has held certifications from Juniper, Netscreen, and Cisco, and possesses a deep technical background in the areas of routing, switching, and security. His primary foc

The complete guide to planning, configuring, managing, and troubleshooting NX-OS in enterprise environments

• Thorougly explains how to configure and troubleshoot NX-OS in the data center
• Offers deep and practical coverage of high availability, virtualization, and security
• Contains new and updated coverage of FabricPath, OTV, IPv6, QoS, VSG, Multi-Hop FCoE, LISP, MPLS, Layer 3 on Nexus 5000, Config sync, and more
• Includes realistic case studies from Cisco data center experts