The daily life of a Network Architect(Blog)

Last Chance to sign up for todays NEXUS Dashboard Webinar

I will be co-hosting a webinar with Cisco on the Cisco NEXUS Dashboard and day 2 operations

Cisco is offering a 1 hour Cisco Nexus Dashboard deep dive and demo in their Hybrid Cloud Demo Series on April 13th 10:00 AM PST, 1PM EST

Cisco has invited me to speak on their Hybrid Cloud Series webinar on April 13 at 10:00 AM PST so mark your Calendars!

Please use this link to sign up and I’ll see you April 13th! https://engage2demand.cisco.com/LP=26112

I have been working with the Cisco BU on EFTs of the NEXUS Dashboard since it was APIC-X, the ASE, the CASE, and now NEXUS Dashboard. The same holds for the other day 2 ops as I worked on NAE when it was Candid and NIR and NIA before combining into NEXUS Insights.

Cisco Nexus Dashboard: Achieve full stack operations and observability in the multicloud

Tuesday, April 13, 2021
10:00 AM Pacific Time

Experience the ease of comprehensive visibility and scale in the multicloud

The Cisco Data Center Nexus® Dashboard delivers unprecedented simplicity across Cisco ACI® and Cisco NX-OS in multicloud networks. It brings together unified proactive operations, including actionable insights, that are open to any fabric or domain, thus seamlessly empowering both native and fabric-agnostic solutions.

In this demo webinar, seasoned WWT technical architect Mike Witte, Cisco® product management technical leader Soumitra Mukherji, and Cisco product manager Navjyoti Sharma, Sr., will unpack powerful use cases for the Cisco Nexus Dashboard.

New Cisco NEXUS Dashboard article on WWT.com

Given the customer interest in NEXUS Dashboard and day 2 operations suite I have written an article to explain the NEXUS Dashboard and day 2 ops. The article discusses high level design considerations for deployment and what the day 2 operations suite of NAE and NEXUS Insights can offer customers to maintain their fabric infrastructures.

https://www.wwt.com/article/understanding-the-cisco-nexus-dashboard-platform

One thing to note is that Cisco is consolidating the applications that run on top of NEXUS Dashboard with new names to give a consistent naming under the NEXUS Dashboard platform name

Cisco NEXUS Dashboard is the cluster platform for the day 2 operations tools run on top of. Today it is physical servers, in upcoming releases there will be a Virtual NEXUS Dashboard and a Cloud NEXUS Dashboard.

MSO will become NEXUS Dashboard Orchestrator

NEXUS Insights and Network Assurance Engine will combine to be NEXUS Dashboard Insights

NEXUS Data Broker will become NEXUS Dashboard Data Broker.

We have also developed on-demand labs for NEXUS Dashboard and day 2 operations, look for them to be live on WWT.com in the coming weeks.

Co-Hosting on Cisco NEXUS Dashboard Demo Series

Cisco is offering a 1 hour Cisco Nexus Dashboard deep dive and demo in their Hybrid Cloud Demo Series on April 13th 10:00 AM PST

Cisco has invited me to speak on their Hybrid Cloud Series webinar on April 13 at 10:00 AM PST so mark your Calendars!

Please use this link to sign up and I’ll see you April 13th! https://engage2demand.cisco.com/LP=26112

I have been working with the Cisco BU on EFTs of the NEXUS Dashboard since it was APIC-X, the ASE, the CASE, and now NEXUS Dashboard. The same holds for the other day 2 ops as I worked on NAE when it was Candid and NIR and NIA before combining into NEXUS Insights.

Cisco Nexus Dashboard: Achieve full stack operations and observability in the multicloud

Tuesday, April 13, 2021
10:00 AM Pacific Time

Experience the ease of comprehensive visibility and scale in the multicloud

The Cisco Data Center Nexus® Dashboard delivers unprecedented simplicity across Cisco ACI® and Cisco NX-OS in multicloud networks. It brings together unified proactive operations, including actionable insights, that are open to any fabric or domain, thus seamlessly empowering both native and fabric-agnostic solutions.

In this demo webinar, seasoned WWT technical architect Mike Witte, Cisco® product management technical leader Soumitra Mukherji, and Cisco product manager Navjyoti Sharma, Sr., will unpack powerful use cases for the Cisco Nexus Dashboard.

I am presenting a deep dive into the NEXUS dashboard and how it enables onboarding Cisco Data Center Day2 operations tools such as NEXUS Insights, NAE, and MSO. The NEXUS Dashboard also allows third-party integrations to UCS Intersight, AppDynamics, Splunk, and in the future other applications using the standards-based KAFKA Bus. The KAFKA bus allows bi-direction communications for data ingestion into the NEXUS dashboard and data export to Splunk and Service now for integration.

By ingesting data from various sources allows us to correlate Application and user issues into networking issues seen by the data center fabrics such as ACI or VXLAN EVPN using DCNM. With this correlated data, we can now holistically view business-critical issues and see the network anomalies causing these mission-critical applications to have errors and timeouts. The view of correlated business and network traffic data allows a faster MTTR and reducing cost from outages.

Doing a deep dive into NAE and NEXUS Insights under the covers and use cases to use NAE’s assurance engine to validate the state of the fabrics and NEXUS Insights to look at flows and traffic errors. The combination gives us correlated data in a single place to review events. As more applications integrate into these products via the KAFKA bus, we have a more correlated business, application, and network data to proactively view business problems and repair the network issues causing them

Hope to see you on April 13th 10:00 AM PST!

Upgrading NEXUS Dashboard, MSO,NAE and NI

Just a quick walk through of the upgrade process of the NEXUS Dashboard(ND) and the day 2 operations applications that run on it. For those not familiar with the NEXUS Dashboard, it is a cluster of high powered UCS compute infrastructure that the NEXUS Dashboard software runs on. This cluster can be from 3 to 7 servers in size and is connected to ACI fabrics to gather telemetry and health data from the APICs and switches via InBand ACI management. This data is then ingested and processed by either the Network Assurance Engine(NAE) or Network Insights(NI) to give a single pane of glass view of the day 2 operations of the ACI fabric(s). Also the NEXUS Dashboard can be used to host the Multi-Site Orchestrator(MSO) when running ACI multisite. The upgrade will be in two parts; first we shall upgrade the NEXUS dashboard cluster and let the cluster become healthy, then we shall upgrade the NAE, NI and MSO applications that reside on the NEXUS Dashboard

When first login into ND we have our dashboard as shown below We see we have two sites East and West, MSO, NAE and NI applications

First we will navigate to the operations menu on the left and go to backup and restore as its always best practice to back anything up before a upgrade. We will give the backup an encyrption key and a file name and download it to our jumpbox.

And we see a successful backup

Next we will go up to the Firware Management section under the Operations menu. We see the firmware version, and there is no firmware updates.

Next we click on the Images tab and we can load a .iso file for our upgrade either from a remote site or local jumpbox.

Going back to the Updates tab we will launch the Update wizard

We will choose the .iso 2.0.1d firmwae and click next

Click next and it will show the service nodes and target firmware. Click Begin Install the image will be downloaded to the 3 ND devices.

We can see here the image is being downloaded and is at 33%

Now that we are done downloading the iso to each device in the cluster we can now “Activate” the software.

This will reboot each ND device just like when we do a APIC upgrade and the cluster will reconverge with the new code.

One thing to note is the extreme amount of time for the upgrade

Once the ND is upgraded its time for upgrading the MSO, NAE and NI app. Navigate to Service Catalog menu, and click on the app store. As you can see each of the applications can be updated

This will prompmt you for a license accept screen click agree and download

Update the apps one at a time, and going to Installed Services you will see the apps updating and initializing. These do take a bit to upgrade given the size and data stores so plan a sizable outage window for the upgrades.

NEXUS Dashboard 2.0 first time bring up

For first time bring up or if you want to reload a new image(like going from EFT to GA code), we need to image the ND nodes to the proper code and do a factory reset. On any of the three nodes we will run the first boot script, then run through the wizard to configure the other nodes.

Once the ND comes up connect to the OOB address configured in previous step. You will be presented with the setup screen login with the password you used in the first-time setup script

Give the cluster a name and add in the NTP and DNS servers

On each node add the CIMC address and user/password and click verify. It will populate the Serial number from CIMC. Give the node a name, OOB management IP and a IP address on the inband(Data Network). This must be reachable via the inband management of the APICs. Also if you are using a trunk you need to put the VLAN information in

Here is a screenshot of the 3 nodes configurations, click next.

The final screen shows the details verify the OOB, Inband, VLAN if needed are all correct. Then click configure.

The initial ND node will bootstrap, then log into the other nodes and configure the OOB and Inband management addresses.

if you look at the console of one of the nodes you will see the node being configured by the initial node

Here we see the nodes clustering

Here we see images being loaded into registry

here we can see the details of the node being configured setting up kubernetes and configuring

Once the services register and start and cluster forms, the NEXUS Dashboard login will come up. Login using the admin passowrd created in the initial setup

And we have a freshly setup Nexus Dashboard!

Finally from one of the nodes command line, loging as rescue-user and use the ‘acs health” command to verify that the cluster is good and apps can be installed.

Final week of voting 2020 IT Blog Awards, hosted by Cisco

Shameless plug, we are in the final week. Some of your favorite Champs are finalists.  Please check out my blog, and if you like it and are inspired, please vote so I can be even more inspiring next year! https://packetized.wordpress.com/

Go check it out & vote: https://www.ciscofeedback.vovici.com/se/705E3ECD2A8D7180 You’ll have to do a little math to submit; my Blog is highlighted below in the Most Inspirational category, “The Journey of Binary Bits.” Again thanks to everyone who voted and who supports me for writing this stuff at 1 am. If you like muscle cars, astrophotography, particle physics, check out the links to my other sites and the Blogroll, there is some cool stuff in there.

#ITBlogAwards

Installing NEXUS Dashboard for NAE and NIR

We obtained a EFT NEXUS Dashboard back before the holidays so next step was to connect to the APIC fabric in one of the multisites. Because we want to test NIR and NAE, we have to configure the APIC for inband management.

The next few steps will document creating the inband connectivity for the APIC.

In this setup, Since it is a DEV lab we had the original older service engines cabled directly to the fabric. We have 3 NEXUS Dashboard cluster, and all are configured for OOB and CIMC management. The next piece required for the service engine is to configure inband networking for the APIC and service engines. This is required for NIR so it can pull flow data from the switches.

We can configure the NEXUS dashboard by connecting the data interfaces to a L3out connected to the inband network.

We can configure the NEXUS dashboard connecting the inband networking by configuring a EPG to connect to the inband networking as in the following diagram we have the cluster data interfaces connected to a EPG/BD. This is the way we will examine the configuration steps for the EPG/BD method. Down the road we will also experiment with the L3 out concept as well and will write another article for that.

First step is to create a Leaf Profile for the inband connectivity. Navigate to Fabric/Access Policies/Switches/Leaf Switches/Profiles. The right click to start the wizard. Name the Profile some descriptive like the leafs and purpose, then click the plus sign for Leaf Selectors. Again give it a descriptive name and choose block 101 and 102 then click update. Then click Next

We will go back and create the associations so just click Finish to finish the wizard

Next we shall configure The Leaf Interface profile. Navigate to Interfaces/Leaf Interfaces and right click Profiles to start wizard. Give it a Descriptive name

Next Click on Interface selectors + sign to add the APIC and service engine interface connections. In our case we are using 1/9 for the APIC, and 1/10-12 for the Service engine connections. Next we shall need to create the Leaf Access policy group to Apply configurations like LLDP, 10GB etc

Under the Interface Policy Group drop down choose Create Leaf Access Policy

Use a descriptive name , then choose 10GB, LLDP_Enabled in the drop downs. If they are not there create the 10GB and LLDP enabled policies. Next on the Attached Entity Profile choose the drop down to Create a AEP to attach the physical domain to the interface policy and Interface selectors.

Give it a descriptive name and under Domains, click the + sign to ad a physical Domain

Choose Create Physical Domain, give it a descriptive name and create a new VLAN pool.

Give the VLAN pool a descriptive name and click the + sign to add the VLANs. We will use VLAN 1000 for this.

Click OK to close the VLAN and Submit to close VLAN pool

Click submit on the Physical domain

Your physical domain should look like this, click submit

Click submit on the Leaf policy Group

Click Ok to save the Access port profile

Click Submit to finish Creating the Leaf profile

Next Navigate back to the Switch selector to associate the interface selector we just created. Click the + under the Interface Selector Profiles

This finishes the access policies for the fabric now we need to configure management. by default the fabric comes with a bridge domain called inb and VRF called inb and oob under the management tenant.

I am going to configure the in band management GW under the subnet in the L3 config. Click the + sign under subnets to add the subent(Default GW)

I will be using the 10.254.86.1/24 for the inband GW. I need to advertise Externally and share VRF because my L3out is in common tenant.

I also need to advertise to the L3 out in common tenant. Click the + sign under

I choose my L3Out then hit update

Next we need to create the Inband management EPG. Use VLAN 1001 which we created earlier and bridge domain inb

We also need to add contracts to connect the inband EPG to the common L3 Out EPG

I also like to make sure that oob is prefered so if something isnt right i dont black hole myself. By default its set to inband, however since only OOB is configured it uses the OOB. Once you configure the next steps it will change the routing table and if there is something weird upstream you wont be blackholed especially since mine is only a single APIC.

if you notice the default route is OOB, however once I configure inband that will change and i dont want to blackhole if something goes wrong in my config

Click on the Static management node address to create the inband parameters. Here i am using only node 1 since i have 1 APIC. I used the Inband EPG i created earlier and starting address will be 10.254.86.2. If i had 3 my range would be 1-3 and it would use .3and .4 for apic 2 and apic 3

You will see a warning, but since we verified our prefered is OOB we are good.

We see the new Inband and bond interface

This configuration will allow us to gather flow statistics for the NIR application to monitor and troubleshoot traffic flows in real time from the ACI fabric. Look for future articles on how to install NAE, NIR and APPd applications onto the service engine cluster.

The new NEXUS Dashboard and day 2 operations suites.

I had been fortunate to work on the EFT of one of the best integrated day two operations suites providing unparalleled visibility into a datacenter network. Look for this type of visibility across the multiple domains of WAN, LAN, DC, Cloud as Cisco rolls this fabric analytic and flow engines to manage and report on misconfigurations, bugs, traffic drops from a single platform the NEXUS Dashboard.

The NEXUS Dashboard itself is a high-performance platform utilizing high speed compute, fast HDD, and tons of memory to provide a stable platform to run Cisco’s Day2 apps consisting of Network Assurance Engine(NAE), Network Insights(NIR and NIA combined), MSO and the in the future 3rd party apps. There are also linked services such as Splunk, Service Now, Ansible, and Terraform. Features and integrations will go to the ACI fabric, then to DCNM and conventional NX-OS based networks. So the NEXUS Dashboard is a holistic platform for ACI, DCNm, and NX-OS to host both Cisco day 2 apps and third-party telemetry tools.

Nexus Dashboard applications and integrations.

The NEXUS Dashboard Platform is the hardware-based cluster form factor consisting of a minimum of 3 nodes expandable to 7 nodes. There is also a software-based NEXUS Dashboard that can run as a .ova on ESXi hosts. In a future version, a cloud-based NEXUS dashboard is available.

Both form factors are deployed by connecting the inband management network to the nodes. The first mode is by using the EPG/BD mode and directly connect the NEXUS Dashboard to the fabric to gather telemetry from the APICs and flows from the switches.

This image has an empty alt attribute; its file name is image-1.png

There is also a L3out connectivity by connecting the NEXUS Dashboard’s data interfaces via a L3 out connected to inband management.

This image has an empty alt attribute; its file name is image-2.png

Once the NEXUS Dashboard is configured, day 2 applications Such as MSO, NAE, and NI are added to the NEXUS dashboard. We shall take a brief look at the applications that can run on the NEXUS Dashboard(ND)

First is the Multi-Site Orchestrator or MSO. The MSO is used to create VXLAN connectivity policies between ACI on-prem and cloud sites as well as tenant templates that can extend L2 and L3 connectivity seemlessly with a single policy between sites.

Next in Network Assurance Engine or NAE. To understand the role of NAE in ACI day 2 operation, we must look at what intent-based networking is. Let’s look at the standard intent based networking model. We combine business intent (I need users moving to new finance building 5 to have access to their resources) with IT intent (users in building 5 can only access the internet and resources in the Finance Tenant). A policy is created to fulfill this intent and applied via automation to the infrastructure. What has been missing is an assurance engine to verify the policy applied will give you the correct intent. This is where NAE comes in.

NAE works by collecting data from the APIC then comparing it to the intent, policy and state of the fabric. It uses mathematical modeling and validates code to configurations to validate the policy is correct.

NAE can provide endpoint connectivity analysis using policy explorer and natural language search

NAE has a Epoch timeline anaysis to show when a error occured and what change was made

NAE can alos be used for pre-change validation of ACI policy, security compliance

Finally, Network Insights 5.0(NI 5.0) offers many data sources of telemetry such as Syslog, RIB, and FIB tables, streaming telemetry. It then ingests these datasets and extracts the metadata, and correlates against a database updated from Cisco. This telemetry and coorelation of the metadata NI will derive insights and suggest remediation actions for finding root cause analysis and predictive failure.

Some of the use cases for NI are MTTR, OPEX Savings, Availability and uptime and preventative measures, bug notices of the software and hardware versions, and PSIRT notices. These are all derived from a daily updated database from Cisco. For high security or air-gapped networks, proxies and other methods are available to provide an updated database.

In the future, ND 3.1 will offer a third form factor providing the NEXUS Dashboard in the public cloud. Also, in Version 6.0, all of the day 2 operations(NEA and NI) are combined into one application allowing a single pane of glass view never before available in any OEMs fabric.

For further information look at https://www.wwt.com/topic/data-center-networking for upcoming day 2 labs and demos of the NEXUS Dashboard and day 2 operations in action.

Journey Before Destination part deux

Today was my first day back to work after surgery on December 4th. It’s been a roller coaster, and I have not had much time to think about what’s next. I finally saw my surgeon and oncologist last Friday, and they were pleased with the progression, and 6 months of chemo with a port in my chest is the next step next week. So now I have a clear destination, but I don’t want to take that journey. I know many people that have dealt with severe health issues, lost family members, the Covid-19 pandemic; it just freezes you in place to think you and your loved ones are not immortal. This weekend I was pretty lost; it’s been hard for me to move forward. Then I came across this on one of the forums from my favorite series of books, “The Stormlight Archives” by Brandon Sanderson.

You can’t stop as that’s the end of your journey. No matter how many times you fail or are in pain, you have to step forward. And as Sanderson so eloquently said the “The most important step a person can take is always the next one. “ This made my day yesterday figured I would share this as it’s a perfect message to those who feel like they don’t know what to do next.

One of the things that have kept me going through all of this is I am so looking forward to coming back to work at WWT. Family and friends have asked if I was going to retire after all of this? Hell no, I have way too much left in my journey to stop. Plus, I work with the best people on the most cutting edge technology, and there is no end in sight to my journey. I hope this helps someone having a crossroad moment to take the next step.

Christmas Star 2020 (Jupiter-Saturn Conjunction)

The Jupiter and Saturn conjunction picture below was a video taken using SharpCap, then processed in AutoStakkert and PSS to cull the bad frames and keep the most clearest pictures during atmospheric changes(why stars twinkle). In doing any high resolution planetary or moon imaging you must use this technique called lucky imaging to pull only the best frames from the thousands of video frames then stack hundreds of the best into one stacked image. Once the raw photo was produced I spent a ton of time in Photoshop and PixInsight to tweak the image to what you see below.

This has actually inspired me to start doing really detailed planetary images in the future, I’ll need to get a dedicated planetary camera and remove my reducer and get a barlow for more power. Then I’ll be able to take some detailed pics of Saturn and Jupiter once they return (but never this close in the Field of View(FOV)).

Enjoy Merry Christmas to all!

I haven’t done astrophotography in quite a while as it was hard to get my gear in and out of the house the last 18 months due to my PE’s. I wanted to get this once in a lifetime event, and i had my daughter friend help build a wheeled stand for my scope and then lift it on. made out of 2 3/4″ plywood griangle, 5: wheels and a handle to move it it rolls like a dream. I also clamped the legs and mdille of the base with turnbuckles so it is rock solid. 6″ bolts in each corner provide for leveling the entire scope and lock it in place.

The scope is a Celestron 8″ Edge-HD, top mounted guider as well as a off-axis guider in the optical path. A Celestron .7x reducer, Moonlite automated fine focus control, filer wheel and ZWO 1600MM cooled camera make up the imaging path. The scope is a hypertuned and beltdrive converted NEQ6 mount it guides very well after these mods.

The scope was really far out of alignment with the guidescope so had to get that zeroed in on the moon, the used that as a sync point and slewed to Jupiter and Saturn.

On the laptop during the capture you can see Jupiter’s moons and Saturn’s rings. Long way to get from that to final image.

Also a pic of pixinsight during processing