Archive

Posts Tagged ‘powershell’

Extracting information from a Cisco router config with Powershell

August 27, 2011 Leave a comment

Why this script?

Information about systems in a local network is often distributed over several devices/sources. These sources are not always all up to date. After something (or a lot of things) changed in your network, you might find yourself facing the task of bringing all these devices/sources to a consistent configuration state.

From the outside, your gateway router or firewall is the ‘entry point’ to your local network. So if if something changed in your network, it is a good point to start checking network configuration consistency by looking at the gateway router. This script was written to help with this task.

What it does

The script will read a Cisco router config file and extract some interesting bits of it by applying regular expression pattern matching to each line. This kind of ‘lazy parsing’ used here is far from complete. My main goal was to get information about hosts (represented in the router config by IP addresses) having ports opened individually for them, e.g. smtp, www, imap, etc. For that reason I don’t handle config statements which open ports for a address ranges of (internal) destination addresses. Also, in my environment we use ‘static NAT’ for some hosts (we’re moving away from using it), so the script extracts information about the mapping between private (internal) and official (external) IP addresses as well. Some general information about the router itself is also processed (router hostname, name servers in use, interfaces and assigned IP addresses, …).

The information extracted from the config file is transformed into an internal XML representation. After processing the script simply writes the XML representation to a file. Although extracting this information might already be helpful in itself, it would be overkill to use Powershell and XML only for this basic task. A few simple grep commands might have been enough for that as well. Representing the extracted information using XML makes two things easier. First, you can relate bits of information from different locations in the router config file (‘on what interface was ACL 110 used again?’). This isn’t so easy if you just grep against the config file. Second, storing the extracted info in an XML file allows easy further processing done by additional scripts. These will be part of a blog post yet to come. Just to give you an idea about what will follow later:

  • The information will be augmented by doing reverse DNS lookups (to get the host names for the ‘naked’ IP addresses).
  • Also, pinging the IP addresses will (usually) show us whether the systems are still alive.
  • In case of IP addresses representing Windows systems, using WMI might get us even more information about a host which was originally represented in the router config only by a meager IP address.

You might get your XML to start from cheaper…

If you have a Cisco router running with a newer version of IOS (XR), you might be able to directly save its configuration to an XML file. Then you wouldn’t need this script at all. But you might still be interested in the upcoming posts about augmenting or analyzing router information.

Design rationale of the XML representation used

General structure

If you look at the XML file generated by this script, its structure might at first look overly complicated. On the top level there is a twofold distinction. One branch contains information about the router itself (hostname, name servers, interface configuration, …), all under the top level node <my_config>. The other branch – starting with <systems> – is listing items from the config which are about (internal) systems known to the router. This information is again quite deeply structured. Why not use a simple representation of open ports per internal system, which would be along the lines of ‘port X is open for destination IP Y’ or – in XML – <open_port src_ip=”…” dst_ip=”…” port=”…”/>? Why use a far more complex representation which first postulates the existence of a <system>, having an <interface>, which is assigned an <ip>, for which finally the router has a something to say about open ports or static NAT entries? The reasons for this complex representation are extensibility and reusability. The information about systems extracted from a router configuration is very rudimentary and can be quite useless if not augmented by additional information from other sources. One example: Cisco router configurations are all about IP addresses. Some network administrators might know every system accessible from the outside just by looking at its IP address, but then again not all will do so. For that reason it makes sense to augment the extracted information by doing a reverse DNS lookup later on. And if we later on have to augment the information we got from a router anyway, why not start with a representation of internal systems which is right from the start designed for being easily augmented?

Avoiding the abundant use of attributes on XML nodes

An early unpublished version of this script was encoding a lot of information into XML node attributes – like shown in the XML fragment given in the previous section. The reason for using attributes in the first place was that this results in a very compact encoding of information. If every bit of information for an XML node is encoded into attributes, you won’t even need an explicit closing tag – ‘/>’ will do. But just using attributes has several drawbacks. First, you can’t assign multiple values to an attribute – at least not without giving attribute values some internal structure like <host ip=”a.b.c.d;e.f.g.h”/>. And doing something like that would only mess things up completely. Second, imagine you have to join information about systems originating from two different sources A and B. From every source you have generated a separate XML file which contains the information the source has about internal systems. Now you would like to merge the XML files into one more complete representation. Merging would be straight forward if the systems in question would be disjunct between the two sources. Unfortunately this won’t happen, so you have some information about a system X from source A and other bits of information from source B. Joining these bits automatically is possible if both sources include a common item for a system, e.g. an IP address or a host name. The actual joining can be done easily with tools available for XML if you just have to copy the child nodes from an XML node for a system in source A to the equivalent system node in source B. But if you make heavy use of attributes, the same task suddenly gets very difficult since there is no easy way to copy all attributes from one XML node to another. If you do know one, please tell me.

The Script

# scan-cisco-config.ps1
# Scan a configuration file of a cisco router and extract some general information
# about which ports are open for which IP addresses.
# Extracts some general router config info as well.
# The extracted information is saved as XML to enable further analysis and reuse.
# (c) Marcus Schommler, 2011

# default value for host name (used until one is read from the config file):
$hostname = "cisco-router"

$ip_pat = "([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)"
$proto_pat ="(tcp|udp|ip|icmp|gre|pim|esp)"

# create a stub xml node set from a string:
$xcfg = [xml] "<root><my_config><system/></my_config><systems/></root>"

# read a saved cisco config file to process:
$cisco_cfg = @()
$cisco_cfg = Get-Content .\cisco-bo-config.txt

# iterate over the lines read:
foreach ($cl in $cisco_cfg) {
	if ($cl.length -lt 3) {
		# line too short to be of interest
		continue
	}
	if ($cl -match "ip nat inside source static $ip_pat $ip_pat") {
		# processing static nat entries
		$curr_itf = $null
		
		# add a new host element to the xml document we're bulding:
		$h = $xcfg.CreateElement("system")
		$sxml = "<interface><ip>" + $Matches[1] + "</ip>"
		$sxml += "<nat_ip src='$hostname'>" + $Matches[2] + "</nat_ip></interface>" 
		$h.InnerXml = $sxml
		
		# Since the straight forward '$xcfg.root.hosts.AppendChild($h)' doesn't work (please ask MS why),
		# we're using an alternate syntax.
		# thanks to: http://www.terminal23.net/2007/09/powershell_nuance_with_appendc.html
		$xcfg.root["systems"].AppendChild($h)

	} elseif ($cl -match "access-list (\d+) permit $proto_pat (.*)") {
		# found acl permit entry, continue processing with the matching parts of the line:
		$curr_itf = $null
		$acl = $Matches[1]
		$proto = $Matches[2]
		$permit = $Matches[3]
		$add = $true
		$do_continue = $false
		$src_ip = $null
		$src_mask = $null
		if ($permit -match "any host(.*)") {
			$permit = $Matches[1]
			$do_continue = $true
		} elseif ($permit -match "host(.*)host(.*)") {
			$src_ip = $Matches[1]
			$permit = $Matches[2]
			$do_continue = $true
		} elseif ($permit -match "$ip_pat\s+$ip_pat\s+host(.*)") {
			$src_ip = $Matches[1]
			$src_mask = $Matches[2]
			$permit = $Matches[3]
			$do_continue = $true
		}
		if ($do_continue) {
			if ($permit -match "$ip_pat (eq|gt) (\w+)") {
				# single port or 'greater than'
				$dst_ip = $Matches[1]
				$oper = $Matches[2]
				$port = $Matches[3]
			} elseif ($permit -match "$ip_pat range (\w+)\s+(\w+)") {
				# a range of ports is open
				$dst_ip = $Matches[1]
				$oper="range"
				$port = $Matches[2] + "-" + $Matches[3]
			} elseif ($permit -match "$ip_pat`$") {
				$dst_ip = $Matches[1]
				$port = "all"
			} else {
				$add = $false
			}
		}
		if ($add) {
			# look up the ip in our xml host list:
			$h = $xcfg.SelectSingleNode("/root/systems/system/interface[nat_ip='$dst_ip']")
			$h2 = $xcfg.SelectSingleNode("/root/systems/system/interface[ip='$dst_ip']")
			if (($h -eq $null) -and ($h2 -eq $null)) {
				# a system entry was not added while parsing static nat entries
				# -> add one now:
				$h = $xcfg.CreateElement("system")
				$sxml = "<interface><ip>" + $dst_ip + "</ip></interface>"
				$h.InnerXml = $sxml
				$xcfg.root["systems"].AppendChild($h)
				$h = $h.SelectSingleNode("interface")
			} 
			$p = $xcfg.CreateElement("open_port")
			$p.SetAttribute("src", $hostname)
			$p.SetAttribute("acl", $acl)
			$p.SetAttribute("proto", $proto)
			if ($src_ip -ne $null) {
				$p.SetAttribute("src_ip", $src_ip.Trim())
			}
			if ($src_mask -ne $null) {
				$p.SetAttribute("src_mask", $src_mask.Trim())
			}
			$p.SetAttribute("op", $oper)
			$p.SetAttribute("port", $port)
			if ($h -ne $null) {
				$h.AppendChild($p)
			} else {
				$h2.AppendChild($p)
			}
		}
		
	} elseif ($cl -match "interface (.*)") {
		# located the beginning of an interface definition
		$curr_itf = $xcfg.CreateElement("interface")
		$curr_itf.SetAttribute("name", $matches[1])
		$xcfg.root.my_config["system"].AppendChild($curr_itf)
		
	} elseif ($cl -match "ip name-server\s+(.*)") {
		# located name server entry for the router
		$dns = $xcfg.CreateElement("name_server")
		$dns.InnerXml = "<ip>" + $matches[1] + "</ip>"
		$xcfg.root.my_config["system"].AppendChild($dns)
		
	} elseif ($cl -match "\s+description\s+(.*)") {
		if ($curr_itf -ne $null) { 
			# found description for an interface
			$curr_itf.SetAttribute("desc", $matches[1])			
		}
		
	} elseif ($cl -match "\s+ip access-group\s+(.*)") {
		if ($curr_itf -ne $null) { 
			$curr_itf.SetAttribute("acl", $matches[1])					
		}
		
	} elseif ($cl -match "\s+ip address\s+(\S+)\s+(\S+)") {
		# IP address for an interface
		if ($curr_itf -ne $null) { 
			$ip = $xcfg.CreateElement("ip")
			$ip.InnerText = $matches[1]
			$ip.SetAttribute("netmask", $matches[2])
			$curr_itf.AppendChild($ip)					
		}
		
	} elseif ($cl -match "\s*hostname\s+(.*)") {
		# extract configured host name for this router:
		$hostname = $matches[1]					
		$h = $xcfg.CreateElement("name")
		$h.InnerText = $hostname
		$xcfg.root.my_config["system"].AppendChild($h)
		
	} elseif ($cl -match "ip route") {
		# currently we're doing nothing with routing information,
		# just reset the current interface def:
		$curr_itf = $null
	}
}

# save the complete generated xml to a file:
$xcfg.Save(".\cisco-cfg.xml")
Advertisements

After a long time my mind finally made this connection…

August 7, 2011 Leave a comment

After using Powershell for a few months now, only just today I stumbled upon this analogy: In New Zealand you’ll find molluscs called Paua. Sitting right here in my bathroom are the remains of one of these – a Paua shell.
Besides the homophony, there is another striking similarity between a Paua shell and Powershell: In their natural state they’re both quite ugly. That’s how a Paua looks like when taken from the ocean:

And like with Powershell, only if you invest some work and do some polishing you’ll get results with a shine:
Paua Shell

But the Paua still has a big advantage over Powershell: It doesn’t take you a long time to get a decent meal out of the former! 🙂

Categories: powershell Tags: ,

Adding some beef: Multi server multi host DNS resolution testing

August 6, 2011 Leave a comment

When you’re hosting multiple web presences and you’re also being responsible for the DNS name resolution zones, you might find yourself in a situation where touching anything DNS-related is giving you a bad feeling even before you start. It is getting even worse if for some reason you also have to deal with a ‘split brain’ DNS setup (serving different IP addresses for the same host name depending on who is asking, either an external or internal client).

For helping me in that situation I started to build a script that allows me to run automated DNS resolution tests against groups of DNS servers. All configuration and test data is kept external to the script in an XML file. It’s astonishing how easy it is to work with XML in Powershell. Some introductory other examples show you just how to select nodes and manipulate XML to generate XML output again. This script uses the data structures built by reading an XML file to directly iterate over XML node subsets with the purpose to generate the DNS test command lines, for looking up the DNS servers to execute the test against and to check whether the results meet the expected IP addresses specified beforehand.

The actual DNS lookup is done wrapping a call to nslookup. This is because the alternative .NET class System.Net.Dns used in my last post doesn’t allow you to specify the DNS server to be used, it will always use the one from the local NIC configuration.

# Test-IPResoluton.ps1
# (c) Marcus Schommler, 2011
# Testing DNS resolution for multiple DNS server and multiple host names.
# The configuration and test data is being read from an XML file.

# ==========================================
# Open issues & possible enhancements:
# - It might be of interest whether the answer of a DNS server is non authoritative.
# - Also it might be helpful to specify for test cases whether an authoritative answer is expected.
# - XML could be used for output as well...

# assertEquals()
# inspired by: http://www.leeholmes.com/blog/2005/09/05/unit-testing-in-powershell-%E2%80%93-a-link-parser/
function assertEquals (
    $expected = "Please specify the expected object",
    $actual = "Please specify the actual object"
    )
{
    if(-not ($expected -eq $actual)) {
        $res = "FAILED. Expected: $expected.  Actual: $actual."
    } else{
        $res = "OK. $actual";
    }
    $res
}

# forward_lookup(): Do DNS forward lookups by using nslookup
# inspired by: http://powershellmasters.blogspot.com/2009/04/nslookup-and-powershell.html
Function forward_lookup ($hostname, $dns_server) {
    # Build command line including stderr redirection to null device:
    $cmd = "nslookup " + $hostname + " " + $dns_server + " 2>null"
    $Error.Clear()
    $global:nonauthAnswer = $false
    $global:controlladns = $false
    $result = Invoke-Expression ($cmd)
    trap {
        $global:controlladns = $true
        $solved_ip = "0.0.0.0"
        continue
    }
    if ($Error.Count -gt 0) {
        # nslookup does output to stderr if a name resolution result is not authoritative.
        # Simplified assumption here: This is the only reason for generating error output.
        # echo "answer not authoritative"
        $global:nonauthAnswer = $true
    }

    # Line 4 of the nslookup-Output contains the resolved IP address
    # -> check and extract by (simplified) pattern matching:
    if ($result.SyncRoot[4] -match "([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)") {
        $solved_ip = $matches[1]
    }
    $solved_ip
}

## main code section ##

# read all settings and test cases from an XML file:
$test_settings = [xml]( Get-Content .\dns-tests-2.xml )

# get start node for the dns server groups to use:
$server_groups = $test_settings.SelectNodes("ruth/dns_server_group")
# get start node for the test cases to check:
$test_sets = $test_settings.SelectNodes("ruth/dns_test_set")

# init array for storing information about failed tests:
$fails = @()

# iterate over test sets found:
foreach ($set in $test_sets) {
    # lookup up which group of DNS servers should be used for this test set:
    $sg_name = $set.getAttribute("dns_server_group")

    # get the individual dns server nodes for this dns server group:
    $sg = $test_settings.SelectNodes("ruth/dns_server_group[@id='$sg_name']/dns_server")

    Write-Host "Test-Set: " $set.GetAttribute("id")
    # get the actual tests to perform
    $tests = $set.SelectNodes("dns_test")
    # iterate over the servers in the sever group:
    foreach ($server in $sg) {
        # iterate over the test cases:
        foreach ($test in $tests) {
            # extract the needed information for test execution from various xml nodes:
            $hostname = $test.getAttribute("host")
            $expect_ip = $test.getAttribute("ip")
            $dns = $server.getAttribute("ip")
            # execute the test:
            $ip = forward_lookup $hostname $dns

            # check the result of the dns resolution against the expected IP address:
            $res = assertEquals $expect_ip $ip
            # generate output:
            $s1 = "DNS: $dns, Host: $hostname : "
            if ($res.contains("FAIL")) {
                $fails += $s1 + $res
            }
            Write-Host $($s1 + $res)
        }
    }
    Write-Host "===== End Test-Set: " $set.GetAttribute("id")
}

# Output of a Summary: To have only the failed tests all in one coherent block of text,
# we repeat the output just for them:
Write-Host "============"
Write-Host "All Failures"
foreach ($f in $fails) {
    Write-Host $f
}

And that’s how an XML file to be used with this script could look like:

<?xml version="1.0" standalone="yes"?>
<ruth>
	<dns_server_group id="Gesis external lookup">
		<dns_server note="dns2.gesis.org" ip="194.95.75.2" />
		<dns_server note="unix1" ip="134.95.45.3" />
	</dns_server_group>
	<dns_server_group id="Google public DNS servers">
		<dns_server note="goopub1" ip="8.8.8.8"/>
		<dns_server note="goopub2" ip="8.8.4.4"/>
	</dns_server_group>
	<dns_test_set id="gesis.org external" dns_server_group="Gesis external lookup">
		<dns_test host="www.gesis.org" ip="172.16.4.252" />
		<dns_test host="ftp.bonn.gesis.org" ip="193.175.238.3" />
		<dns_test host="jws.bonn.gesis.org" ip="194.95.75.5" />
		<dns_test host="listserv.bonn.gesis.org" ip="193.175.238.78" />
		<dns_test host="webmail.koeln.gesis.org" ip="134.95.45.6" />
		<dns_test host="download.za.gesis.org" ip="134.95.45.13" />
	</dns_test_set>
	<dns_test_set id="PartnerHosting external" dns_server_group="Gesis external lookup">
		<dns_test host="www.asi-ev.org" ip="193.175.238.210"/>
		<dns_test host="www.dgo-online.org" ip="193.175.238.92"/>
		<dns_test host="www.iconnecteu.org" ip="193.175.238.140"/>
		<dns_test host="www.soziologie.de" ip="194.95.75.36"/>
	</dns_test_set>
	<dns_test_set id="PartnerHosting external - google dns" dns_server_group="Google public DNS servers">
		<dns_test host="www.asi-ev.org" ip="193.175.238.210"/>
		<dns_test host="www.dgo-online.org" ip="193.175.238.92"/>
	</dns_test_set>
</ruth>

Categories: powershell Tags: , , , , ,

Humble beginnings: DNS Resolving for a list of host names

August 6, 2011 2 comments

Here’s my first small Powershell script to share. Tackling DNS cleanup and reorganization over multiple locations and DNS servers, one of my first tasks was to get a reference list of host names and IP Adresses for about twenty domains hosted by us. While ping or nslookup seemed still feasible to use, I anyhow decided to let a script do the tedious work. It takes just a few lines…

 # resolve-all.ps1
# (c) Marcus Schommler, 2011
# Reads in all lines of the specified file and tries DNS name resolution on every line.
# (Apparently the file should contain host names).
# The output is done with Write-Host in a 'XML-snippets' format for reuse
# in another script/project (testing DNS resolutions for multiple DNS servers and multiple host names).

$entries = Get-Content www-all-zones.txt
foreach ($e in $entries) {
   $ips = [System.Net.Dns]::GetHostAddresses($e)
   # A host can have multiple IP adresses, so we iterate over the result of GetHostAddresses():
   foreach ($ip in $ips) {
      # XML-snipped style output for reuse in another project (mass testing of DNS resolutions):
      Write-Host '<dns_test host="'$e'" ip="'$ip'"/>'
   }
}
Categories: powershell Tags: , , ,

Powershell: the powerful beast

August 6, 2011 Leave a comment

With about 30 years of programming experience (I started at age 15 on a programmable calculator, the TI-58C) I’ve tried a lot of programming languages in my time. The latest one I began to use was Powershell. Originating from Microsoft and being pushed by this company as the main supported base for some important business/server products (e.g. Exchange), this was something to look into for my day-to-day IT-administrative needs in the office.  With that decision began what you could call a love/hate relationship – if taking a more professional or detached approach wouldn’t forbid such a strong choice of words.

While this programming language is powerful and flexible in its use, its basic syntax and some of its idiosyncratic concepts are so strange that I started to wonder if the design team tried to outperform Perl in unreadibility and C++ in obscurity of ‘wording’. And yes, I do have coding experience with both of the latter languages.

But even with these points of personal dislike in the background I recently find myself coming back to Powershell more often. Currently it is my premier choice when trying to automate tasks in the area of Microsoft-based IT systems management and consolidation. Powerful, flexible and free is hard to beat, even if you have to accept some ugly warts in return.