UNIX Monitor - LoadRunner

I often encounter an age-old hassle over monitoring servers and infrastructure under test. Being a performance engineer, I need to see system resources for the application in real-time as a test is running. Sometimes the company does not know about all the infrastructure that needs monitoring. Other times they just don’t want to give me the required access. SaaS is a whole different issue. For this article, let’s assume on-premise servers with Windows and Unix/Linux machines. After tracing down the machines that need to be monitored, permission is technically easy. That is, once you win the political battle. My article on how to monitor Oracle with a baseball bat is still making the circles, but I digress…

With LoadRunner version 12.56, a new feature was quietly introduced that extremely beneficial. There isn’t a lot of information written about it online, except in the LoadRunner help documentation. It’s important to know why I think this is a big deal, so let’s review how we got here to provide context.

History

When Mercury Interactive purchased a little company in Colorado called Freshwater Software, management decided to integrate SiteScope into LoadRunner. At a business level, this seemed reasonable, but it turned into a problem for many LoadRunner users. LoadRunner already had many native monitors built in. Instead of continuing to add future monitors (or embed the ones from SiteScope), it would require that SiteScope be installed and configured as well.

In addition double the work for installation, LoadRunner would require additional settings to display SiteScope graphs. This was clunky and meant more machines required for the test lab. The SiteScope graph in LoadRunner is ONE, BIG, FREAKING graph with every metric from the SiteScope server. Analysis graphs were manually parsed out to separate them. Anyone else remember that?

In companies that only used a Microsoft OS for their servers, life was great! With the appropriate permissions to access Microsoft Performance Monitor (aka Perfmon) DLL libraries, monitoring was easy. For awhile, UNIX was not a big deal, either. All you needed was the rstatd process running. There was no need to add SiteScope unless there was something LoadRunner did not have. This is still the case for Windows only shops. Occasionally you might use SiteScope for something, like a VMWare monitor.

Hello, Sarbanes-Oxley

After Enron and Sarbanes-Oxley became a thing, Big 5 Auditors had lots of work and racked up the big bucks. Part of the auditing process focused on securing servers. Suddenly, rstatd wasn’t just a daemon, it was a DEMON, and must be exorcised! For performance engineers, it meant no rstatd, and no monitoring of UNIX machines directly. A UNIX admin would monitor it with a command line tool. They would export it out into a text file.

UNIX admins are grumpy and they really don’t like doing this. They have long, gray beards like ZZ Top, with crumbs in them. It’s scary because this applies to both male and females like the drwarf race from Lord of the Rings. Gimli would have been a good UNIX admin. But we had to deal with that, unless we had SiteScope. SiteScope became very useful at that point. LoadRunner could product real-time UNIX system resource metrics without rstatd. SiteScope could use SSH to remote in, scrape the values, and send them back as data points and LoadRunner could make the graphs.

It was still a hassle of getting an SSH account with the right permissions (with another baseball bat), but it was a lesser evil. Now we wouldn’t need to bug the UNIX admins for help, interrupting them from playing Dungeons and Dragons in the back of the server room. We were still having to parse the that ONE, BIG, FREAKING catch-all SiteScope graph into smaller segments! Did I mention it was one big freaking graph? But it was a small price to pay not to have to look at those beards.

Make It Stop

For too many years, many LoadRunner users have requested an enhancement to ditch SiteScope. It is not that I hate SiteScope. I like it. It isn’t efficient to use it with LoadRunner. All I really want is one tool to monitor them all. SiteScope is great for stand alone use. It is useful when rolling up monitoring data to a production APM application. However, the days of strategically integrating SiteScope into LoadRunner are over (IMHO). I want native monitoring performed through LoadRunner only. I don’t know why they can’t take the core code and embed it. It is possible there is a good reason why.

When Docker became the hip new thing for every human being to spend time on, someone released SiteScope as a container:

https://hub.docker.com/_/sitescope

This saves a download and installation. As a result, you can literally have SiteScope up and running 5 minutes. Unfortunately, it hasn’t been updated in two years. There is another issue where newer versions of Chrome and Firefox won’t run the JAVA-based dashboard. I have read that the Unified Dashboard is HTML5. “More and more” of the code is being moved over, but I think the train has already left the station on that one and Micro Focus needs to get ahead of this or else get left behind. There are already articles showing how to port SiteScope alerts to ElasticSearch (ELK):

https://medium.com/@kottapar/sitescope-alerts-to-elasticsearch-4bb19a011daf

Finally, A Solution

During the release of LoadRunner 12.56, in the “What’s New” section of Controller and Analysis, I read “LoadRunner now supports monitoring for UNIX machines using SSH. Check out How to Set up the UNIX Monitoring Environment.”

I ran naked through a corn field at night with a semi-automatic sling shot I was so excited! This saves me several hours of work for each project where I need the Unix monitor with no rstatd. This is quite common today. I immediately downloaded and installed it. To test it out, I fired up one of my trusty Odroid XU4’s. IF you don’t know about them, you should check them out:

https://wiki.odroid.com/odroid-xu4/odroid-xu4

They are more powerful than a Raspberry PI, and run Ubuntu 18.04. I know, I know – UNIX versus Linux. Who cares! Let’s walk through setting things up together, shall we? To proceed, you will need:

  • LoadRunner installed
  • Unix/Linux machine
  • Appropriate SSH access.

Setup UNIX Resources

Launch the LoadRunner Controller. Go to the Run Tab to view the real-time monitors. Just like adding Windows Resources, drag the UNIX Resource graph into a free space, and right-click to add a monitor:

Add a Server:

I put in the machine name of my Linux server and the SSH credentials in this displayed pop up:

It immediately comes up with the machine and default counters I can apply:

If you want to know what is available, here is a list of UNIX Resource Metrics from the current online help from Micro Focus:

https://admhelp.microfocus.com/lr/en/12.60-12.62/help/WebHelp/Content/Controller/r_UNIX_counters.htm

If you need help determining what to monitor within a UNIX OS, here is a reference for that (start with chapter 4, page 60):

https://www.microfocus.com/media/documentation/loadrunner_and_performance_center_document.pdf

One you have selected the metrics, click OK. Wait a few seconds to ensure the monitor is working by showing the live data points:

Boom, daddio! How easy was that? Ten years late? Maybe. But, I’m happy I can finally get UNIX graphs natively. Why is nobody else raving about this? Is it because LoadRunner’s monitoring capabilities still aren’t being explored due to a lack of training? Is it management resistance and internal politics? They have baseball bats for that…

If you missed this new feature, it is understandable. It wasn’t a focus of the release announcement. Knowing is half the battle. Enjoy your new capability! Why not send a positive note to the LoadRunner product managers at Micro Focus? Let them know you appreciate them. I did.

About the Author

Scott Moore

Scott Moore specializes in application performance engineering and testing. This includes education, hands-on implementation, and application performance monitoring.