Here is an article by Andy McBride which explains how to track down
the cause of sluggish application response as seen by the users.
The Time Honoured Argument:
Application Geek: "My code is fine, the network is slow
- that's the problem."
Network Supremo: "The network has plenty of
bandwidth, something must be wrong with your application."
Stop the Madness
As you can see, the above dialogue could be classified as less than
productive. Both the network engineer and the application developer had
some tools to diagnose what was happening in their area of expertise,
but they each had little to no visibility into how the network and
applications were working together. The two departments were
experiencing the effects of technology silos. Something had to change
in how we defined and measured applications. In the mid to late
1990's that change happened, and management vendors on both sides of
the fence began to offer solutions that included application and
network monitoring features. These included:
Hardware probes that sniff traffic and analyze what is
on-the-wire.
Synthetic transaction software that inserts and measures
artificial transactions to model actual user transactions
Distributed software agents to measure user transactions.
Flow analyzers to discover what applications were consuming
bandwidth, application vendor modules focused on managing their own
application.
Network management system (NMS)-based application monitoring,
which combined application and management packages.
Server Processes/Service and Daemon Monitoring
Windows servers - By far the most common method of monitoring
processes and services is Windows Management Instrumentation (WMI). WMI is Microsoft's implementation of a couple of standards: CIM and
WBEM. WMI offers monitoring of counters on a Windows system to track
machine-level items like memory, processors, etc., and OS information
like server work queues, processes, services and connection status.
Functionally, WMI is a lot like SNMP in its use of counters, polling and
information bases. The best way to get to know WMI is to give it a spin. Download and try the free SolarWinds WMI Monitor tool. Make sure to
spend some time with the WMI browser included in the free tool. It's
a good way to see firsthand the types of objects you can manage with
WMI and how WMI operates.
Linux or Unix servers - These servers have been around for decades
and have mature implementations of SNMP. Like WMI, SNMP on these
servers monitors machine-level and OS components. Unlike WMI, SNMP is
a completely open standard. You can use a MIB browser and walk through
a server's MIB much the way you do in a WMI browser. As an example,
if we want to see that an FTP server is able to operate correctly we
should monitor the FTP process on that server. This can be
accomplished by setting up a WMI monitor and monitoring the FTP
process WMI counter on the FTP server. We could also do this using an
SNMP monitor for the FTP process.
Protocol Port Monitoring Protocols can be thought of as two-way
radios: they allow bidirectional information flow between two or more
points. The protocol ports in this analogy are the radio frequencies. They offer multiple channels of communication within dedicated
spectra. So TCP port 20 is like TCP channel 20. If you were able to
tune in and listen to port 20 you'd be hear computers delivering
files, as TCP port 20 is reserved for File Transfer Protocol (FTP). If you tuned to port 21 you'd hear computers asking for file
transfers that will be sent on port 20. These two ports work together
to supply FTP control (port 21) and an FTP file carrier (port 20). So if
you just want to see if files are able to be transferred, you only
need to setup a monitor for port 20 on the FTP server. On the other
hand, if you want to make sure the FTP network service is able to
function, you would monitor both ports. There's actually a trick we
can do to monitor port 21 directly and 20 indirectly. We'll see that
below.
"10.1.1.1 calling 10.1.1.2, please ship me file1.txt on port 20"
"Hi 10.1.1.1, this is 10.1.1.2, I'll meet you on port 20 with
file1.txt"
This example is a bit more elaborate than actual protocol port
monitoring. In protocol port monitoring what we do is look to make
sure the server is able to use the specified port rather than
listening to the conversations on that port. Examining the
conversations is accomplished by flow analyzers and sniffers. That
level of monitoring is overkill for the type of application
monitoring we are discussing here.
Now we can monitor ports and determine FTP is working port-wise, but
that is only two components of the FTP network service. There is also
an FTP service running on the server OS that receives the requests
and serves the files. As we learned in the previous section we
could monitor the FTP process using WMI or SNMP.
Now that we have combined
the FTP service monitoring with FTP port monitoring we are close to
claiming we are monitoring the FTP application. But what if the
service and ports are operational and someone has deleted the target
file to be served? While the components of FTP are operational, the FTP
network service cannot complete user requests for the absent file. We
should test the service by requesting the file.
Thus utility makes it easy to check the health of a router or firewall.
Check the real-time performance, and availability statistics, for any
device on your network. Get started with an extensive collection
of "out-of-the-box" monitors for popular network devices. Give Network
Monitor a whirl - it's free.
Download your free Network Device Monitor
This component of application monitoring involves making an automated
request of an application to model a real user request. These are called
synthetic transactions because they mimic a user transaction but don't
originate from a user. Typically they are initiated from the NMS.
Let's add a user experience monitor for FTP to the process and protocol
monitors above.
The FTP user monitor in this case will make an FTP
request to the FTP server from the NMS. The request can be for a file
normally requested as part of a business transaction or a test file. When the NMS makes the request for the file it can record the
following:
Time to complete transfer of the file.
File integrity
via checksum.
Here's the trick I mentioned: If we monitor TCP port 21,
we examine FTP requests. Now by placing a user experience monitor on
FTP, we are also testing TCP port 20 indirectly and monitoring the
actual ability to serve a file. After all, if TCP port 20 is not
functioning the user experience transfer will not happen. Let's
review how we've created a holistic application monitor using each of
the component monitors above:
What application do we want to monitor?
FTP of file1.txt from FTP server001.
What components define this
application instance?
Service monitor - The FTP service on server001
Protocol port monitor - TCP ports 20 and 21 on server001
User
experience monitor - File1.txt delivery
How will we measure each one
of these?
Create an application monitor for FTP which includes:
Status of the FTP service on server001 (WMI or SNMP service/ process
monitor)
Status of TCP ports 20 and 21 on server001 (protocol
monitor)
Delivery of file1.txt (user experience monitor)
Often
these component monitors are grouped to make an application template. This can help to save a great deal of time when there are multiple
instances of an application to monitor. Instead of having to create a
new application monitor for FTP on server002, wouldn't it be nice to
apply the template created for server00 instead?
»
Simple and Complex
Applications
For our purposes, I'll define these as:
Simple application = an
application that operates as a process, independently from other
applications or processes.
Complex application = an application that
requires multiple processes and services and interacts with other
applications to complete its functions.
This does not imply anything
about size or ease of use of the applications, it just refers to
application and process/service interaction. In our example we
created an application monitor for a simple application, FTP, which
happens to be a network service. Let's take SolarWinds Orion Network
Performance Monitor with the Orion NetFlow Traffic Analyzer module as
an example of a complex application. Here are some of the components
that we could monitor to make a complete application monitor:
HTTP monitor of Orion home page. This monitor will log in to the
Orion home page automatically and test both authentication and
delivery of the http home page.
SQL monitor. This monitor will
automatically query the SQL server and return the results.
That
makes 10 individual monitors to monitor the application, and you could
probably add a few more. These monitors cover all three of the component
areas we defined in Section 3 - service monitors, port monitors and user
experience monitors. Here is where application templates are crucial. With 10 or more components being used to monitor an application, there
is plenty of opportunity for mistakes if you had to create the
application monitor from scratch each time. If you apply a verified,
saved application monitor template, the risk of applying monitors
with mistakes is eliminated. Another option would be to create an
application monitor template for the SQL server and one for the DNS
server that NetFlow uses to resolve endpoint IP addresses to domains.
We have seen the following points in the above sections:
1) An
application is not just a service or process running on a server.
2) Applications depend on multiple resources to operate correctly,
including:
Services/processes - both the actual application services
and associated services which support the application Server
resources (RAM, CPU, hard drives, etc.)
Protocol ports
Secondary
applications such as SQL in section 4
3) Applications can be effectively
monitored by:
Examining the application components and applying the
proper monitors, including: WMI and SNMP component monitors for
processes/services
Port component monitors for protocol ports
User
experience monitors to demonstrate the application functionality. I
hope this has been a helpful overview.
If you have any questions or
comments, participate in our online community Thwack, made of over
58,000 users of SolarWinds products
Additional Free and Trial SolarWinds Network Software
Here are nifty gadgets which I have evaluating on my network.
Some are completely free, while other downloads are trial versions of the
full product. I think SolarWinds have a great strategy, namely,
supplying a free utility, yet providing a big-brother suite of programs for
larger organizations.
Guy Recommends:
SolarWinds' NPM - Network Performance Monitor
SolarWinds' performance monitor is designed for detecting network outages,
making it easy to see what's working, and what needs your attention.
This utility guides you through creating network maps; it also helps
identifying whether the
root cause is faulty equipment, or resource overload. Give NPM a try.
