Pentaho BA Server: Analysis Report Performance Test – Part 1

Problem

Few days ago we were told by the pilot users of our BA Server that for few minutes their performance when creating reports drastically dropped. Our initial guess was that probably it was somehow related to one of the ETL processes running in the background. We started monitoring the database, but we did not see nothing interesting. Hence we decided to get deeper insight into the performance of the BA server itself. My colleague decided to use JMeter but me, I wanted to see whether Selenium engine can get me somewhere.

What is Selenium?

Firstly, to create a POC I installed a Selenium IDE, a Firefox plugin to create Selenium tests through GUI. Basically what you can achieve using Selenium is to simulate user’s activity on the website like: open this link, fire up this button, fill-in this form and then verify whether page responds in the expected manner: whether you have been redirected to the proper page, received expected error, a success information was displayed, etc. etc..

Executing the report

I wanted to see how difficult it would be to define following scenario.

  1. Open BA Server page
  2. Execute one of Analyzer reports
  3. Check the amount of time that took BA server to produce the report

I started with below test case.

  1. Open home page and authenticate to the BA Server
  2. Expand drop-down menu under Home header
  3. Choose ‘File Browser’ from the menu
  4. Select one of the folders
  5. Open previously created report

Selenium

Here is how the test looks like.

Selenium

As you can see I authenticate with server through Spring Security endpoint under j_spring_security_check sending credentials in the url, which is far from perfect but I have figured out for the POC it is close enough.

Later I recalled that you can actually create a deep-link to the report (see picture below), so that you can open it using a specific URL instead of looking for it using the File Browser.

Create deep link

Nevertheless, I very much enjoyed programming the Selenium steps. At the beginning the test did not success consistently. Sometimes it worked, sometimes it did not. I discovered that putting a pause command between steps solved the problem. Thanks to that I avoided situations when a step tried to open a hyperlink that was not yet rendered and ultimately caused the test case to fail. However, it seemed very ineffective and I felt like these pause commands cost me a lot of time. Fortunately I found a waitForElementPresent command that allowed me to wait untill an element was rendered before reaching it.

Retrieving the execution time

We have reached the point when we are able to open a particular report in the BA Server. But how to check the execution time? Luckily there is a possibility to do it right from the Analyzer interface by opening Log menu.

Selenium

This will open a new window with a table presenting individual steps done to produce the report.

Selenium

We will be most interested in the last row that contains total execution time.

Problem arises though, we cannot configure Selenium to open the log window right after it opens a report document as at the beginning it will not include the execution time. We have to wait untill the report is fully generated. To monitor this I use the Refreshing indicator from the top right corner. When it is gone I open the Log window.

Selenium

First I change focus to the newly opened window where logs are stored (using selecPopUp command. Then I save the last row of the Log table to a variable (using storeText commnad) and then print it to the Selenium console (using echo commnad).

Selenium

Taking all the above into considerations, this is how these steps look in Selenium.

Selenium

Loops

I wanted to see whether it would be possible to loop this execution using Selenium IDE. I followed instructions on this site to install a Selenium IDE plugin for while loop and added store (for loop counter), while and endwhile commands. As for every run we open a new window (the one with Logs table), at the end of the each iteration there is a close command to remove the redundant window. Even though the window was closed, Selenium would try to use it in the next iteration of the loop and that would lead to an error. To fix this problem I use a selectWindow command which takes back the focus to the first window and allows program to continue its work.

Selenium

What next?

The evaluation of Selenium IDE was done to verify whether it will integrate with the BA Server seamlessly. Most certainly it does. Nevertheless, it is only a plugin to Firefox and as such it has few flaws. Firstly, despite getting the execution time it is hard to save the result anywhere outside of the plugin, I have seen some workarounds for this, but I was afraid to loose time on something I will not use and ultimately I decided not to give it a go. Secondly, you can open only one Selenium window at a time and hence cannot run more than one test at once.

Therefore, I am planning to export this test case and rewrite it using Selenium API for Java. This should give me more control over the test case itself, solve all the aforementioned obstacles and as well let me conduct stress tests. Please check soon for the Part 2 of this article.

Please join discussion