Welcome readers to the CodeNSX blog and to my first first article here. Hopefully you’ll find lots of information you’ll find useful in your day job, or at least add to your repertoire of tools and knowledge. I thought for my first entry, I’d kick things off by focusing on using NSX to automate deployment of data center infrastructure outside the scope of a cloud management platform (CMP). So, this might beg the question, “what exactly is a CMP”?, what does it do?, and what platforms “qualify” as a CMP? I’m already thinking ahead to my next article, and intend to do a deep dive on those topics then, with a focus on automating *with* a CMP. But for now, those topics are outside the scope of this article, so I’ll simplify things by stating, when I talk about a Cloud Management Platform, I’m referring to the likes of vRealize Automation, vCloud Director, or Openstack. More on those later.
So, on to the topic at hand — using NSX (and vSphere) to automate deployment of data center infrastructure without a CMP.
Say I manage an IT organization, and like any IT shop my core responsibility involves providing services to end users (who, are likely other depts within my company, but are ultimately my customers) while maintaining existing infrastructure. Those services can range from provisioning new VMs, patching servers, plumbing VLANs through the network, etc. Like most IT organizations, the manual processes involved in accomplishing these tasks are too slow to keep pace with the demands of my end users and their business requirements. For these reasons, automation is of interest to me. I very much want to realize all the benefits automation promises: speed, error-free provisioning, improved efficiency, and much more. So, how do go I go about doing that?
In my experience at VMware, the vast majority of customers are interested in discussing and actually investing in automation. I’ll be the first to admit, at VMware we’re sometimes guilty of focusing those discussions around vRealize Automation (or another CMP) to the detriment of other options. But the fact is, when it comes to NSX and other virtualization technologies, there is a huge ecosystem of tools available, outside of vRA that will allow you to achieve your automation goals. Don’t get me wrong, vRA is a great platform that can do lots of cool things, but there are organizations that maybe aren’t at the right maturity level to adopt a platform like vRA, or don’t have a need for end user self-service (which is a core function of clouds and thus CMPs), or simply have a different use case for automation that doesn’t require a CMP (of which I can think of many).
It is for these reasons I wanted my first article to focus on automating infrastructure deployment with NSX and vSphere using some of these aforementioned tools, not including vRA. Before discussing tools, let touch on the pre-requisites for automation. First and foremost it’s virtualization, that is, having an abstraction layer over your physical data center infrastructure. vSphere provides this for your compute — the benefits of server virtualization are well known and the debate regarding it’s merits is long over NSX is a much newer technology, and therefore less well known, but like vSphere it provides an abstraction layer for infrastructure, but for your network. Like server virtualization, the benefits of network virtualization are the same. With NSX virtual machines aren’t chained to physical VLANs plumbed through the network. Instead I provision virtual networks with NSX (routers, switches, firewalls, load balancers, etc.) without touching underlying network infrastructure. The virtual networks are built in the software abstraction layer, and while they rely on the underlay for transport between physical machines, they are not tied to the network. This de-coupling of compute and network services from the hardware is the first step in the journey to automation nirvana.
So with the abstraction layer our automation tools don’t need to be tied to a specific vendor hardware platform – we can use any vendor for the physical network or for my compute infrastructure. How we interface with these virtualization platforms is the second pre-requisite for automation. Just like vSphere has a management plane — vCenter — NSX has the same, with a slightly less creative name: NSX Manager. As the management interface to your virtual compute and network infrastructure, vCenter and NSX Manager provide a nice GUI where you can provision all your resources with a couple of button clicks, but, more importantly they have a RESTful API. If you’re reading this blog — a site about using code to automate NSX — you probably have a pretty good idea what an API is. But for those readers that are less sure, I’ll provide my non-technical explanation of it. An API is simply a software interface to which I can direct requests. In the case of our RESTful API, those requests are sent to an IP address and port, in the form of an HTTPS packet. I like to think of the GUI as nothing more than another interface to the API. In the case of NSX this is absolutely true -- the GUI itself is not embedded in the NSX Manager, it’s actually a plugin in vCenter. When I login to vCenter to access that GUI to create an NSX virtual switch for example, all that GUI does is send an API request to NSX Manager to create that switch on my behalf. The key point to understand is that the API provides a way of programmatically interacting with my platform, and because my platform provides a way to provision virtual compute and network infrastructure on-demand, then I now have a programmatic way to automate deployment of infrastructure. But simply having an API is not enough. It only gets us half-way there. Let’s get to the other half.
There’s a plethora of programming languages and an extensive array of software tools out there that were either created exclusively for, or play an integral role in, automation. In the realm of languages, I’m thinking primarily of PowerShell and Python. In regards to software tools, there are many, including Ansible, Chef, Puppet, and of course VMware’s own platform: vRealize Orchestrator. The critical second half I’m involves using these tools or languages. But, that requires having a plugin, or some type of integration between these software tools/programming languages and the NSX API. And guess what, they exist! So, putting them all together: 1) I have my virtualization platform that allows me to create infrastructure on-demand, without touching the physical layer, 2) these platforms have an API that allow for programmatic interaction with them, and 3) I have existing “automation focused” scripting languages and software tools that integrate with the vSphere and NSX API. So, what’s holding us back? Nothing really, it’s a simple matter of understanding the tools that are available and how they can be used, which is where this blog comes in. You simply need to know a thing or two about NSX plus one of these tools (pick one), and understand the integration between the two, to begin using them.
Here’s a quick summary of of some of the automation tools or scripting languages along with a description of how they integrate with NSX.
Scripting language: PowerShell
Integration with NSX via: PowerNSX
Description: Developed by Anthony Burke and Nick Bradford at VMware. PowerNSX abstracts away the NSX API to set of easily used PowerShell functions. I’m not a PowerShell expert, but you don’t have to be to use PowerNSX. If you have a Windows Server with PowerShell and Internet, you simply enter a single line in PowerShell to install PowerNSX and you’ll have 200+ command lines you can run in PowerShell (like “create logical switch” or “delete logical router”) that will call a function which translates that command to an API call. And it’s all nicely packaged together in a scripting language that is tailor made to automate IT functions. Plus Anthony and Nick have been kind enough to post some super useful sample scripts on GitHub that any IT professional can take advantage of, regardless of whether they are PowerShell experts. More details here: https://github.com/vmware/powernsx
Scripting language: Python
Integration with NSX via: PyNSX
Description: PyNSX is pretty similar to PowerNSX, but for Python. It provides a library you import into Python to allow for CLI like interactions with NSX, so again CLI based commands are converted via the library to API calls. More details here: https://github.com/vmware/pynsxv
Automation Platform: Ansible
Integration with NSX via: plugin
Description: Ansible is a great tool and is designed to automate tasks through a playbook. Again, VMware has an NSX module for Ansible available on Github. If you use NSX in your data center, and you use Ansible, you can easily automate the deployment of network topology with this module by including the NSX tasks in your Ansible playbook.
The intention here is to lay the groundwork to give you a better understanding of how virtualization technologies, combined with existing software tools (and leveraging the existing integrations with these tools) can help you automate infrastructure deployment in your data center.
A more specific example of automated infrastructure deployment with NSX and one of these tools is in order here. I’ll use the example of onboarding a new customer to my data center, which in my experience, can be a time-intensive process involving lots of manual provisioning to set-up the customers’ environment. In a previous role I’d worked in, a design engineer would engage the customer to understand and capture their requirements, usually in spreadsheet form which included specific design details like IP addressing, subnet names, domain names, and more. Once this stage was complete, the provisioning process began, which could take well over a week. Now this was several years ago before virtualization technologies like vSphere and NSX were ubiquitous. But, in a modern data center, I would want both these platforms deployed, and I would leverage them to automate that provisioning process, which even without any scripting experience would not be a complex task. Take a look at the NSXBuildFromScratch.ps1 PowerNSX script here, created by Anthony Burke:
This script does two things: 1) automatically deploy NSX into a vSphere environment and 2) automatically deploy a sample 3-tier application. You an execute the script such that only the second half runs if you already have NSX deployed. The only thing you need to do to make this script work is customize the variables in it for your environment. Without extensive scripting experience you could develop a very similar PowerShell script which leverages PowerNSX (and PowerCLI – which is the vSphere equivalent) or write an Ansible playbook to automate the provisioning stage of the customer onboarding process. Perhaps a standard configuration for a new customer environment requires the following: 1) provisioning of three customer VLANS, 2) domain controller for AD & DNS, 2) dedicated router for uplink, NAT, and external connectivity, 3) firewall for security, and 4) reserved IP subnets from IPAM. So, you could execute a script which would pull configuration details from the design spreadsheet and then provision the network and compute resources for the customer’s environment. Using PowerNSX, the script could easily provision the customer router with the NAT configuration, stand up the VLANs (actually in NSX it’s VXLANs, but same concept), and configure the NSX firewall rules all via the API. This can all be done in a matter of minutes rather than waiting days or weeks for the change requests to be completed via the normal manual processes.
It was my intention here with my first post to cover the possibilities around automation and explain the significance of the API, and the integration of NSX with existing automation tools like PowerShell. In subsequent posts we’ll drill into finer detail with more specific examples of real world use cases and accompanying scripts.