Mouse in the House

Posted by admin on April 6th, 2011

Along with the big list of protocols you can monitor using the WatchMouse service and its global infrastructure, you can also execute “transaction application” tests or as they are more commonly referred to, “functional” tests.

Before going through the steps on how to create and upload such a script to your WatchMouse account, lets briefly see what transaction application testing actually is.

Transaction Application Testing

On top of testing the availability and the performance of a website or web application (non-functional testing), you can also test the individual components of it such as, a login procedure, the results of a search in a form, an article submission and so on.

Transaction application testing differs from non-functional system testing in that, with transaction application testing you have to specify and test the functions that the web service is expected to perform.

Creating a transaction testing script

WatchMouse uses the JMeter scripting engine to run transaction application tests.  A JMeter script is like a browser which executes steps that test the functionality of a web application. Note however, that JMeter does not support all the actions supported by browsers, for example it doesn’t execute JavaScript functions.

To create a valid JMeter script we strongly suggest to use the Badboy windows application, which can be downloaded here, with Badboy you can easily:

• Record the actions you want your script to perform, in a browser environment
• Replay the actions you recorded to validate the script functionality
• Export the script to .jmx format, so you can open it with the JMeter application or
• Upload the script directly from Badboy to your WatchMouse account

Exporting your Badboy script to JMeter correctly might require some customization, due to a few differences between Badboy and JMeter execution:

• JMeter doesn’t execute JavaScript, so in order to simulate JavaScript functionality you might need to pass values (for example a session ID) from one call to the other, manually. You can do this by saving a specific value, after an HTTP request, in a variable and use this variable in subsequent HTTP requests.
• Badboy executes its actions in a linear fashion while JMeter needs to define a scope for every action (element). So for example, if you add an assertion element, to match a text which appears after a login procedure (i.e. the text “log out”), in JMeter you should add that element as a child of the login HTTP request rather than putting it after the request in the list of calls.
• Unlike Badboy, JMeter doesn’t download the embedded elements and assets of a web page (images, css and JavaScript included files etc.). It only tests the functionality of it. You can enable downloading of embedded elements by choosing the corresponding setting in the JMeter application.

Uploading your scripts to WatchMouse

To upload the script to your WatchMouse account you have to:

• Create a new monitor
• Choose “script” in the “type” dropbox
• Upload the script, using the upload form
• Save your monitor

The WatchMouse engine will check the validity of your script and then create the new monitor.

NOTE: Due to the number of calls a script monitor performs, WatchMouse has a default timeout of 20 seconds for these type of monitors. You can adjust the timeout, according to your script, in the monitor “expert mode” settings.

Getting Help from us

You can find a set of example scripts we have created for reference, which test different kinds of applications (SOAP, OAuth, HTTP authentication) here: WatchMouse JMeter repository

We are also happy to help to construct the scripts. Just send the script to helpdesk AT watchmouse.com along with a small description of the difficulties you are facing and we will fix the script for you.

We hope this post will help you understand, as we do in WatchMouse, the importance of transaction monitor testing and also the fun of creating such tests for you websites and web applications.

Post by Nikos Prodromidis: I am a QA Tester and Junior Developer at WatchMouse. I joined the team in June 2009. I find the process of making and understanding functional tests for web applications (i.e. scripting) very interesting and innovative, also I like learning and implementing new technologies.

‘WatchMouse Weekly’ tweets and corresponding blog posts aims to be an introduction with tips and tricks for getting the most out of your WatchMouse monitoring. For all ‘WatchMouse Weekly’ blog posts go here.

Posted by mark on April 1st, 2011

URL shorteners. Lots of people use them every day – including the team at WatchMouse. URL shorteners like bit.ly, Google’s goo.gl, Twitter’s t.co, and Facebook’s fb.me are widely used nowadays, but how reliable and how fast are they really?

We took a look at the pros and cons of URL shorteners in March 2010, and thought it only fair to see how URL shorteners are performing one year later.

And, what a difference a year makes! Last year, Facebook’s fb.me was at the bottom of the list in terms of speed, but this year fb.me tops our list joining Google’s goo.gl and BudURL with 100% uptime and a much improved loading speed.

Why (not) using URL Shorteners

There are some obvious pros and cons of URL shorteners.

On the plus side:

• URL shorteners obviously provide useful features like making a long URL shorter (e.g. so it fits easily in a Twitter message)
• They enable you to track and analyze clicks on a specific short URL
• Some URL shorteners like t.co and mcaf.ee also provide some browsing safety by analyzing the target URL for harmful website code or phishing attempts

But on the down side, URL shorteners also introduce:

• An additional single point of failure: when a URL shortener service is down (or corrupt) the link won’t work
• Additional load time for a page to fully load

URL Shorteners Uptime

WatchMouse monitored the most popular URL shorteners from February 24 – March 28, 2011 to find out how they are doing in terms of availability and speed. During that time we monitored 25 URL shorteners and collected the uptime and performance statistics. Uptime is an issue for URL shorteners because it has a direct impact on the uptime availability of the website that the URL shortener actually directs to. The uptime results are shown in the chart below:

Uptime is still clearly an issue for some of the URL shorteners, but what a difference a year makes! Last year Facebook’s fb.me landed at the lower regions of our list. Things have changed dramatically this year and now only fb.me, goo.gl, and BudURL scored a perfect 100%. And to be fair, Twitter’s t.co would also score a perfect 100% if they weren’t blocked from China, which is obviously out of their control.

According to our data, twurl.cc, tr.im and to. appear to be dead in the water and inactive with over 31 days of downtime. Digg.com racked up over 19 days of downtime, while snurl.com had over 14 hours of downtime, making them our worst performers and by far the slowest among the active URL shorteners.

URL Shorteners Speed

The performance results can be seen in the chart below:

Note that we left out the resolve time in this chart, please see the full report for a version with the resolve time included and what it means.

• lnkd.in is the slowest and adds over 700 milliseconds on average to the page load time after the click on a link (excluding the resolve time), which is really way too much for just a URL redirection. This substantially affects the user experience.
• goo.gl is super speedy and does a redirection in just about 100 milliseconds, which is really impressive we think.

Live URL Shortener Status Report

We continuously monitor URL shorteners and share the results publicly through our website portal. The real-time status of each of the sites and a seven-day history can be found at http://url-shorteners.public-website-status.com/. You can also receive Twitter alerts so you know immediately when URL shorteners go down by following http://twitter.com/url_shorteners.

URL Shortener Popularity

It’s not obvious to measure the popularity of URL shorteners, but traffic metric for the domain does give an indication:

This information comes from Alexa and was requested for the five most “famous” URL Shorteners.

Seeing that bit.ly is seeing way more traffic than the others we can conclude they are doing a very good job in terms of availability and speed.

[disclaimer: bit.ly and Twitter are WatchMouse customers]

Methodology and full report

The URL shorteners were checked every five minutes from one of the 58 WatchMouse global website monitoring stations. For each short URL, only the redirection was measured, not the actual loading of the target page. The redirection was expected to complete within four seconds without any errors (like when a server error occurred or if the expected target URL location was not found in the http header). If that time was exceeded, WatchMouse verified the results using another of its monitoring stations and the result was counted as unavailable.

The full report can be found here: Performance and Uptime of URL Shorteners.

What do you think? Have URL shorteners improved dramatically over the past year or is there still room for improvement? We welcome your feedback and comments!

Posted by mark on March 29th, 2011

Posted by mark on March 23rd, 2011

More and more web applications have published APIs that enable developers to easily integrate their data and functionality to make existing data more dynamic through the creation of useful mashups. The result is that thousands of mashed up websites and applications now depend on these APIs to work right all the time, and when an API goes down it creates a downtime domino effect in real time. Sites everywhere fail, but companies with open APIs have an obligation to ensure their APIs are up at least 99.9% of the time.

Today, we released a new API report revealing the uptime of 50 of the most popular “mashed up” APIs as ranked by ProgrammableWeb, including Google Maps, Google Search, Flickr, Twitter, YouTube, Amazon, eBay, Facebook, Microsoft Virtual Earth, and Wikipedia. The good folks over at ProgrammableWeb currently list over 3,000 open APIs and nearly 5,700 available mash-ups on their site, and there are even more on the market.

The Results

Ten APIs performed perfectly with 100% uptime during the reporting period including Basecamp, Delicious, eBay, goo.gl, Google Buzz, Google Charts, Google Maps, Google Search, Quora, and SimpleGeo. Two of the APIs, GeoNames (97.4%) and Eventful (97.2%) performed poorly with nearly a full day of downtime each. Ailing MySpace was at the bottom of the list with 94.3% uptime, the equivalent of nearly two full days of downtime.

The WatchMouse website for monitoring popular APIs, API-status.com, reports on monitoring checks for a valid result on each of the APIs, and if the result is wrong or is received after four seconds, it is noted as an error and unavailable. The percentage of availability or uptime is based on the number of errors reported. Additional details on API-status.com include a seven-day history along with a 24-hour snapshot and performance indication across multiple countries.

30-Day Report Card and Methodology

We monitored the availability of 50 API/cloud web services from February 16, 2011 to March 17, 2011. In accordance with industry standards, availability of greater than or equal to 99.9% is regarded as “good”, while anything below 99% is regarded as “poor” site uptime. The methodology for testing the sites includes one simple API call and check for a valid result. This typically means an authentication action for most APIs, including a login, followed by a search or listing action, plus a check of the expected result action. The expected result can immediately return as an error or if the expected result action is reported after four seconds, it is also logged as an error. These errors are used to create the percentage of availability or uptime for each of the sites. Each site is checked in real time using the WatchMouse API Monitoring technology which can be used to measure and report the availability of any public API or web service. The resulting data is made public with the WatchMouse Public Status Pages. Companies like Twitter and Wikipedia use the tool, which is hosted on the Amazon cloud platform to inform customers and report publicly on the status of their services.

Click here to read the full report on the uptime and performance of the 50 most popular API website services or visit www.API-status.com for real time status and statistical data on each website.

Posted by stan on March 22nd, 2011

We have a complete API at WatchMouse, which is used by our partners, clients, and ourselves too. It can be used to:

• integrate monitoring functionality in other services, like a DNS failover that one of our clients implemented
• base web widgets on, like the Apdex widget,
• create new web services with, like Loads.in, and DownOrNot,
• build mobile apps like the iPhone cloudstatus app.

The unabridged version of the manual is on apidoc.watchmouse.com. To get you started with the API, however, I thought to do a short walk-thru and show you some tips and tricks along the way.

First tip is the API endpoint, http://api.watchmouse.com/1.6/ (1.6 being the current version). You use this in your code obviously, but the nice thing here is that you can also use it in a web browser to experiment with it. Just click it and you see all calls that are currently supported. Now click on cp_list for example, and you will see a short description, the price (free in this case), and the parameter list together with the type of each parameter, if it’s optional or not, and the default values.

To retrieve a list of all monitoring stations as an XLS file, just enter xls in the ‘format’ field and hit ‘exec’. Or to get one random station that supports FTP checks, fill in 1 for the ‘limit’ parameter and ftp for the ‘protocol’ parameter, hit ‘exec’ again and a XML document will be returned with the requested information.

Tip 2: By default, all call return an XML document. The structure of that XML document is documented just below the ‘exec’ button. However, by entering some string in the ‘callback’ field, a JSONP value is returned, which is very convenient when creating HTML+JavaScript widgets. Do give it a try with this cp_list call now!

Now the cp_list call is fairly simple and does not need any authentication, but when you want to access information in your account, e.g. to get the current status of one of your monitors, you obviously do need to authenticate first. For this the API supports both sessions and permanent authentication tokens. The sessions are managed with acct_login and acct_logout, while the permanent access token is obtained by a call to acct_token. Both acct_login and acct_token return an authentication string, named the ‘nkey’. For convenience, these calls also set a cookie so you don’t actually have to provide the ‘nkey’ when playing around with the API web interface.

So try it now and obtain a token using the acct_token call (provide your username and password, and never hand out that token to anyone else!) and then visit rule_get to get the last 50 checks that were performed in your account (don’t forget to set the ‘reverse’ parameter to ‘y’ or you will get the oldest 50).

Okay, that’s it for now. Do play around and let us know what you think. And if you need help or have a request, just open a ticket in the helpdesk and we’ll follow up asap.

Post by Stan van de Burgt. Stan is CEO and co-founder of WatchMouse. Stan writes regular expressions for fun, and Python is his favorite language du jour.

Posted by mark on March 16th, 2011

We just released a minor update to our Public Status Pages. This release includes new features to make it easier to communicate outages or performance issues to your visitors.

We got feedback from a valued customer that the “announcements” you can push to your Public Status Pages are:

• sometimes hard to find,
• disappear too fast, and
• cannot be linked to directly.

We addressed those issues with this release by introducting the following enhancements:

• Announcements/messages for a specific monitor will stay visible on both the overview page and the detail page for a specific monitor. How long it will stay there depends on the user settings for the PSP.
• All messages now have a permalink to a single page with just the message on it. We will extend this in a future release by providing a full message history app, much like a simple blogging engine.
• In the public status page management console this permalink is available next to each message.
• On the Public Status Pages itself the “date and time” field links to the message page.
• A “tweet this” icon is added to the public status page management console to easily share the message through Twitter.

I hope you find these small improvements useful, and we are looking forward to your feedback!

Posted by mark on March 15th, 2011

When setting up a new monitor, we monitor by default from all stations that support the selected protocol or monitor type. In most cases that means your site or server is monitored from all our, currently, 56 stations.

Depending on your situation and requirements, this default might be desired, but maybe it is not. Deciding how to pick your monitoring locations is pretty straightforward:

• If you have a global audience we recommend to use the default setting. In that case a random monitoring station is selected for each individual check.
• If you have a global audience and would like to monitor evenly from all locations you will have to change the scheduling algorithm to “sequential”, see below how to do that.
• If you have an audience in multiple countries, but not all, simply make a custom selection of the stations in the countries you are interested in.
• If your visitors come from a single county, simply pick a station from that country, and change the scheduling algorithm to “master”.

So that was no rocket science right? Next: how to actually set that up.

First of all the default setting. Here you specify the setting for all new monitors you create. Simply go to the Account preferences and find the “checkpoint selection”. Select all the stations you want to be used in your monitoring pool. Note that a minimum of three stations is required. The reason is that for some monitoring errors (like time-out’s) we perform a second opinion check from another location than the one that reported an error to prevent false alerts.

Existing monitors will not be affected by the changes you made in the account preferences.

To change individual monitors, go to the Monitor settings and click on a monitor (or create a new one). The “Checkpoint order algorithm” and the “Checkpoint selection” settings can be found in the “Expert mode”.

The “Checkpoint order algorithm” determines how the monitoring scheduler operates. The following settings are possible:

• Random: a random checkpoint is chosen each time this monitor is checked. This is the default.
• Master: the first check is always done from the checkpoint specified by you. If an error occurs, the second-opinion check is performed from a random (other) checkpoint.
• Sequential: all checkpoints are used in a fixed order (round robin)
• Sticky: same as random, but when a checkpoint detects an error, the monitor will be checked only from that location until the error disappears.

Selecting a specific set of stations is done at the “Checkpoint selection”. Simply check the check-box and you’ll find the same view as in the account preferences, enabling you to select the stations you want to participate for this specific monitor.

Please leave a comment if you have questions about this or open a ticket at the helpdesk.

Post by Mark Pors. Mark is CTO and co-founder of WatchMouse. His favorite editor is emacs, but he hardly gets to use it nowadays.

‘WatchMouse Weekly’ tweets and corresponding blog posts aims to be an introduction with tips and tricks for getting the most out of your WatchMouse monitoring. For all ‘WatchMouse Weekly’ blog posts go here.

Posted by simone on March 8th, 2011

I am sure that many of you reading this post already know what loads.in is. If you have used loads.in before then skip to the paragraph “Getting the most out of Loads.in”.

Quick Introduction to Loads.in

Loads.in is a service that gives you the ability to see how fast your (or any) website loads in a real browser from over 50 locations worldwide. You can read more and test our service by visiting loads.in You can also visit the post How Fast Does Your Website Load – Here and Abroad?

Getting the most out of Loads.in

So now you know how how many seconds it takes to load your site from locations all over the world. Loads.in also provides snapshots of the webpage as it loads. But, is that enough? Of course not, so we provide you with a waterfall chart for each result based on the browser profile. I believe that many of you may not really know how to read these waterfall charts so, stay turned to what follows. I advise you to load your site using loads.in, click on the waterfall chart icon and continue reading.

How to read a Waterfall Chart

Each row in a waterfall chart represents a different object such as text, image, CSS, JavaScript files. As you will see, there are some objects that load simultaneously, the number of simultaneous downloads depends on the browser’s settings. Remember that each browser renders a site differently. Using Loads.in you can verify your site’s load time using different browser profiles.

Each object requires time to be loaded which can be analyzed in the waterfall chart.

• The green bar represents the connect time, which is the time that the server needs to set up a TCP connection
• The bright pink bar represents the blocking time, which is the time taken while the object waits for another files to be completely downloaded
• The purple bar represents the waiting time, which is the time to first byte: the time until the browser receives the first byte of the object from server
• The bright purple bar represents the receiving time, which is the time the browser needs to receive the whole file

Additionally, there are two vertical lines:

• The blue vertical line shows “DOM is loaded”: when the unformatted text and HTML markup have being loaded
• The red vertical line shows “Page loaded”: when all assets of the page including images, CSS, JavaScript etc. have loaded but before the user’s JavaScript has been being executed

For a fast webpage you want:

1. As few rows as possible
2. The “DOM is loaded” and “Page loaded” vertical lines to occur as early as possible and be as close together as possible.

You can read more about understanding waterfall charts in these four articles:

• How to understand your site’s performance via waterfall chart
• Waterfalls 401: Advanced analysis for non-beginners
• How to read the AOL PageTest waterfall chart
• Firebug Net Panel Timings

Post by Ziogas Chris. I am the youngest (and most fun) web developer at WatchMouse. I started coding seriously when I was 15 years old and from then on, I’ve live & dream in this world. I always want to add new technologies to my back-pack and use it on new projects. I am pleased that WatchMouse helps me to search & learn new technologies.

Posted by simone on March 1st, 2011

Setting up a WatchMouse Public Status Page is a simple task performed from the WatchMouse website.  There are also a few nice articles that walk through the whole procedure and can be found at http://www.watchmouse.com/en/feature/public-status-page.html or download the User Manual here: http://www.watchmouse.com/assets/docs/WatchMouse_PSP_Guide.pdf.

What might not be obvious is the logic behind the Public Status Page that indicates performance issues or a service disruption. In this post, I will reveal this little secret and show you how to tweak the algorithm.

Two parameters are predominantly taken into account when measuring the performance of a monitor: “first limit” and “second limit”. Both those parameters can be configured in the monitor setup pages under the “monitoring” dashboard, after switching to the “expert mode”.

If the total time of a public monitor stays below the first limit, the server is performing well. If it totals to a value between the first and second limit, the server is considered to perform poorly. Above the second limit, the performance is considered bad.

A WatchMouse Public Status Page uses both these parameters to identify performance issues and service disruptions.
For the history, it compares the average total time of each day with those parameters. The current performance measurement is based on exponential weighted average of most recent check results.

Setting up these parameters correctly is very important for your Public Status Page. Having them too low will result in a Public Status Page that continuously indicates performance issues whereas having them them too high will hide performance issues from your visitors which, they will eventually find out anyway.
If you haven’t already tuned these parameters, I’d strongly recommend that you do so after considering the following tips:

• Get to know your monitors; check the performance charts under the “reports” dashboard.
• Set the first limit slightly higher than the average total time of your monitors.
• Set the second limit close to the total time it takes to load during a high traffic period.

For example: if you see that the average page load for a specific monitor is 4 seconds, set the first performance limit to 5000ms and the second limit to 8000ms. You can always check your Public Status Page to ensure the performance icons reflect what you had in mind. If not, you now know how to fix it!

For any questions or assistance just leave a comment or contact us through the help desk.

Post by Dimitris Balaouras. I’m the Lead Programmer at WatchMouse. I joined this great team of nerds back in 2006 and I have remained a true fan of WatchMouse ever since. Passionate about software engineering, I enjoy programming more than anything. I’m based in Greece and recently moved from the crowded Athens to Larisa, a small town in Northern Greece where I can code in peace

Posted by simone on February 22nd, 2011

Our new ‘WatchMouse Weekly’ tweets and corresponding blog posts aims to be an introduction with tips and tricks for getting the most out of your WatchMouse monitoring. Written predominantly by our architects and coders, ‘WatchMouse Weekly’ will provide suggestions about functions you might not otherwise find.

We’ll let you know who’s produced each post so that you can start to know the expertise and personalities that make up the WatchMouse team. So, to the first tip…

An alert escalation hierarchy can be set up in the “contacts” tab, under your “monitoring” dashboard, after creating a contacts group. Corresponding to the number of detected errors, your escalation hierarchy can issue alerts to a range of contacts. For example, initial alerts could be sent to your webmaster. Following the continuous identification of the same error, further alerts could be issued to IT managers and eventually after many consecutive errors, an alert could be sent to your CTO. Your monitors can also be set to only alert your daytime staff during their working hours. If errors are detected outside standard working hours, alerts can be sent to your on-call staff.

In addition, the “expert mode” link (available in the “monitors” tab, under your “settings” menu) allows you to set up maintenance schedules, limit monitoring to certain times/days and customize your monitor to alert different contacts.  For full instructions refer to: http://www.watchmouse.com/howto/Getting-Started-with-WatchMouse-Performance-Monitoring.html

Post by Simone Maier. I’m the Product Marketing Manager at WatchMouse. I found WatchMouse while working at a domain registrar; attracted by the clever and flexible approach of Mark (CTO) and Stan (CEO), I joined in 2007. I help write many of WatchMouse’s guides and marketing communications, support our Resellers and our sales activities. I’m based out of the TechHub in London.