Introduction to Memory Bottlenecks
If ever your Windows 2003 server is running slowly, then the first place to look for a bottleneck is memory. Another way of looking at server performance, is that machines with plenty of RAM rarely give problems. A bonus of plenty of memory is, that to a degree, abundant RAM compensates for strain on other resources.
On old servers, lack of memory would give you the full sensory input, you could hear the disk paging, see the light flashing, and ‘Mad’ Mick swears you could smell the disk thrashing. Even with these sensory clues, it is still worth while monitoring memory with Performance Logs. Please also remember the big picture. So once you have had a quick look at memory, remember to check the processor and disk counters.
The servers most likely to suffer from memory shortage are pure database servers for example, Oracle or SQL. Email servers also require plenty of RAM. Pure domain controllers are less likely to experience memory problems.
Memory Topics
- Available MBytes
- Three Paging Counters
- Other Memory Counters
- Identify the root cause of a memory bottleneck
- Advice for application servers e.g. Exchange, SQL or Web
- Solutions to Memory Problems
♦
Key Memory Counter: Available MBytes
The more available memory the faster the server can respond. When I check a server’s memory with performance monitor, the first counter that I add to the log is Memory\Available bytes. As long as the trace indicates more than 10MB of free memory, then I conclude that the server has sufficient RAM.
Diagram 1 shows a white descending line, and the legend confirms that Available bytes are down to 3MB. Clearly this machine needs more memory.
Suppose a spreadsheet wants to start a new thread or a database needs to sort data, what each needs is memory. The operating system provides this memory at least 100 times faster using RAM, than it could using a disk based pagefile. This is why a large pool of free memory is so important to an application server.
Three paging counters in performance monitor
1) Memory: Pages / sec
Take care to distinguish between these two paging counters:
1) Pages /sec (Hard page faults)
2) Page Faults /sec counter is likely to be at least twice the value of the above.
Two problems with monitoring in general, firstly no counter should be taken in isolation, secondly spikes should be ignored, or at least played down.
The less paging the better your server’s performance. Most authorities agree that Memory: Pages / sec is a key memory counter. This counter measures ‘hard’ page faults, in other words the page in nowhere in memory, so the VMM (Virtual Memory Manager) has to fetch the data from the pagefile on the disk; in computing terms that takes an age.
I am reluctant to disagree with other authorities, but from my experience, I would put the threshold as high as 20 pages /sec, before blaming paging as the bottleneck. Moreover, I would not trust pages /sec as an indicator of a bottleneck without confirmation from low Available bytes. (see above)
In truth, if you put 5 experts in the same room, they could all spot a memory bottleneck, but when they wrote up their notes, they would use different time slices and different thresholds, consequently, it would seem that there was a conflict where none actually existed.
2) Memory: Page Faults / Sec
Page faults / sec is the sum of hard and soft page faults. Soft page faults are where the data is found elsewhere in RAM. For example, Word has opened the spellchecker, and now Outlook wishes to use it, there is no need for another call to the disk as the spellchecker is already in memory.
Hard page faults are generated whenever the VMM has to fetch data from the pagefile on the disk. Incidentally, you can use PowerShell to collect information with Get-Counter.
3) Page File: % Usage
While the pagefile is less likely to be a bottleneck it is easy to check, and satisfying to fix. You could also confirm that it is on the most suitable disk, and if possible split the pagefile over two disks and thus improve access times. (Note the object here is Page File not Memory)
Because the changes are so gradual, you are better off using this Page File % Usage counter as an alert, rather than a log. I suggest setting an alert on a limit of over 70.
Guy Recommends: A Free Trial of the Network Performance Monitor (NPM)
v12
SolarWinds’ Network Performance Monitor will help you discover what’s happening on your network. This utility will also guide you through troubleshooting; the dashboard will indicate whether the root cause is a broken link, faulty equipment or resource overload.
Perhaps the NPM’s best feature is the way it suggests solutions to network problems. Its second best feature is the ability to monitor the health of individual VMware virtual machines. If you are interested in troubleshooting, and creating network maps, then I recommend that you give this Network Performance Monitor a try.
Download your free trial of SolarWinds Network Performance Monitor.
Creating a Memory Bottleneck
If you really want to see a memory bottleneck that you can measure with performance monitor, then add the MAXMEM switch to your server’s boot.ini. For example: multi(0)disk(0)rdisk(0)partition(1)\Windows="Windows Server 2003" /MAXMEM=256
Other memory counters
For leaky Applications -Memory: Pool Non-paged Bytes
Thanks to better programming, the problem of applications leaking memory is slowly disappearing. Programs need memory – fair enough, but when they close, if they fail to release the memory back to the pool – not fair.
Monitoring the slow build-up in the non-paged pool is a classic job for an alert. You are unlikely notice much creep in a day’s logging, but over a month you could have lost a significant amount of memory. If you are in a hurry or just plain ruthless, a reboot solves memory leakage into the non-paged pool.
Memory: Cache Bytes
With Memory: Cache Bytes, you have the choice of monitoring the total or breaking down the cache into four categories.
Note Memory: Cache Bytes = Total of these four individual counters
- Memory: System Cache Resident Bytes
- Memory: System Driver Resident Bytes
- Memory: System Code Resident Bytes
- Memory: Pool Paged Resident Bytes
Cache: Data Map Hits %
Data Map Hits is the percentage of data maps in the cache that could be resolved without having to call the disk. It is a measure of how many data pages are already in physical memory. (Note the object is Cache not Memory like the others.)
Committed Bytes
If the value for committed bytes is greater than physical memory, then more RAM would help.
Guy Recommends : SolarWinds’ Free VM Monitor
The best feature of this new this new version of SolarWinds VM Monitor is that it checks Windows Hyper-V. Naturally, it still works with virtual machines on VMware ESX Servers. VM Monitor is a clever desktop tool that not only tests that your server is online, but also displays the CPU and memory utilization for each node.
It’s easy to install and to configure this virtual machine monitor, all you need the host server’s IP address or hostname and the logon info. Give this virtual machine monitor a try – it’s free.
Download your free copy of SolarWinds VM Monitor.
Identify the root cause of a memory bottleneck
Your goal is to discover which application (process) is using most of the memory. Either quickly check the Processes tab of Task Manger or better, create a log of the processes with performance monitor. Developers could consider investigating other tools such as PViewer from the Resource Kit.
Advice for Application Servers e.g. Exchange, SQL or Web
Many applications have their own settings for configuring memory usage. It is not that they can magically produce more memory but they can make the most of the memory the server has. Programs like SQL or Exchange have settings to control their caches. The key question is: should the memory be tied up in a file cache or would it better used by the application’s own cache? I favour the latter. So adjust the applications cache or memory settings, and then see what effect your changes have by logging with System Monitor.
See more about Exchange Memory here
Guy Recommends: SolarWinds Engineer’s Toolset v10
This Engineer’s Toolset v10 provides a comprehensive console of 50 utilities for troubleshooting computer problems. Guy says it helps me monitor what’s occurring on the network, and each tool teaches me more about how the underlying system operates.
There are so many good gadgets; it’s like having free rein of a sweetshop. Thankfully the utilities are displayed logically: monitoring, network discovery, diagnostic, and Cisco tools. Try the SolarWinds Engineer’s Toolset now!
Download your fully functional trial copy of the Engineer’s Toolset v10
Solutions to Memory Problems 
Not enough available memory? The easiest cure for memory problems is to open up the server and add another stick of RAM. Salesmen always seem particularly helpful and knowledgeable on the best type of RAM for my server.
I have a suggestion for future purchases – Always buy machines with more RAM than you need now.
Virtual Memory Settings – Pagefile
Perhaps I should have started by suggesting that you check the performance of the pagefile, particularly as it may offer a cost-nothing solution to memory problems.
The ‘Old Trick’ is to move the paging file away from the operating system’s partition, and if possible, create a paging file on a separate disk. Opinion’s vary about the size of the paging file relative to the amount of RAM, but x2 is a good rule of thumb.
To check or set Virtual Memory follow this path: Control Panel, System Properties, Advanced, Performance and Advanced.
Beware: While you are at the menu, see screenshot, beware of the ‘No paging file’ radio button, other than test I cannot think of any reason why you need to set the radio button next to this setting.
Another Pagefile Trap: If you don’t have any pagefile on the system partition then you lose the ability to debug Stop errors. That said, debugging is a tricky business, and one that I have had little success. If the main Stop error code that you see when the screen goes blue does not help to solve the problem, then in my experience the detail in the memory dump is just too hard for ordinary techies to interpret.
Performance Monitoring with PowerShell
PowerShell has a cmdlet called Get-Counter, which provides an alternative technique to employing Perfmon for investigating computer memory usage. See more about Get-Counter.
If you like this page then please share it with your friends
More Help for Detecting Computer Bottlenecks
Learn the secrets of which counters to monitor. Master performance monitor logging, develop your skills with structured exercises and examples. Print out a copy to read, while you design logs and alerts to detect network bottlenecks. |
