Using Flot Charts with Ajax

How to Call Flot With Real-Time Data

We had a half-day of pain trying to get real-time updates to work in a Javascript time-series chart. We were knobbled by Javascript’s poor debugging options.

On this page we describe the problems we encountered and share our solution.

Flot is a free charting library written in Javascript. We’ve chosen it to assist with development of our logfile analysis project, but our long-term goal is to generate the charts on the back-end and write them to a JPanel.

For now, Javascript will do. Although this is our first experience with flot, we quickly came to like it and would recommend it to front-end developers. We installed the library for the first time on Saturday, and had our charts fully operational on Monday.

As back-end engineers we loathe all that mucking about in the dark under the hood of the browser and, in spite of the short implementation time, it was inevitably a little painful for us. Like sitting on a cactus kind of pain, if you get our meaning.

First thing we needed to do was develop the server-side classes and SQL to generate the data and prepare an HTML framework to hold the charts. That was the easy part.

Charting Requirements

Static charts are a breeze with flot. You simply write the chart data into your HTML as a string representation of a JSON object, and issue a call to flot. Implemented in Java as:

        out.println("<div id=\"placeholder\"" 
                + "style=\"width:600px;height:300px\"></div>");
        out.println("<script type=\"text/javascript\""
                + "src=\"js/jquery.flot.js\">"
                + "</script><script<$.plot($(\"#placeholder\"), " 
		+ gd.TheBackEndMethod()
	        + ", {  })>/script<");

Our data arrives as a string from the call to the gd method, where gd is our class for extracting time-series data from the DB.

Real-Time Updates

It was when we tried to convert that to real-time updates that we hit problems. The first one to be aware of is that real-time updates in time-series are not suitable for asychronous programming.

As it’s a time-series we do not want updates to arrive out-of-order, and we’d rather disgard any that are ‘missed’. Subsequent updates will include anything that’s missing, and lost results will, in the worst case, lead to stuttered scrolling.

A common mistake with flot real-time updates is to put something like this into your javascript:

$.post('TheBackEnd', {pageName:pageName, pageNum:pageNum}, function(responseText) { 
  $.plot($("#placeholder"), responseText, { }:
});

Then spend an hour or two frantically searching google and other ad-supported sites to find out why it doesn’t work.

The reason it does not work is because $.post is an asynchronous call and no matter how fast your back-end is responseText will be empty at the time $.plot is called.

You will get your graph drawn, but there will be no data. There are tricks you can use if you insist on an asynchronous call, but it’s better here to drop down to ajax and make it sycnronous.

	$.ajax({
		method: "POST",
		url: "TheBackEnd",
		dataType: "text",
		async: false,
		data: {pageName:pageName, pageNum:pageNum},
		success: function(response) {
			$.plot($("#placeholder"), response, { });
		}
	});

Once we’d developed this, sure enough, it still didn’t work.

Typing

We still got an empty graph. So we inserted an alert(response) just before the call to $.plot and could see all our data there, correctly formatted. The same data that worked just fine when embedded statically in HTML.

It took us a few hours but we finally hit upon the unintuitive and characteristically front-end solution of inserting a bizzare-looking eval statement.

	$.ajax({
		method: "POST",
		url: "TheBackEnd",
		dataType: "text",
		async: false,
		data: {pageName:pageName, pageNum:pageNum},
		success: function(response) {
			response = eval(response);
			$.plot($("#placeholder"), response, { });
		}
	});

Now it works like a charm.

What is happening is that when the data is written into HTML, Javascript converts it into an array automatically before sending it to $.plot. But when we are getting the data back from our Ajax call to the back-end we must manually convert the string into an array before sending it to $.plot.

This is what eval(response) does, although you’d never have guessed it. Without knowing this you might find yourself re-writing a perfectly good backend to deliver strings in some circumstances, JSON objects in others. Or adding further Javascript to parse JSON objects into HTML.

But there’s a simple one-line solution that will permit the use of strings throughout and minimize front-end code – once you know how. Why strings? Because most of our charts will be static and by embedding the data in the HTML as a string we get better performance and avoid an unnecessary call on page-load that would be required to fetch the data if we used JSON.

To be on the safe side, our class on the backend has been written such that if we decided to use JSON in the future the refactoring will be simplified. If we decided to use flot’s more advance formatting options, JSON would be the better type.

We saw a lot of people having this issue, and hopefully this page will help others avoid what we went through.

Framework

There’s a little bit more required to get real-time updates, and for completeness here’s our full code. This assumes that the HTML framework, including the placeholder div, is already rendered. We have a menu choice on screen that when clicked will fetch and render the HTML, then invoke timeSeriesScroller() which provides the data updates.

var interval = null;
var updateMillis = 1000;
var updatingPage = false;

function timeSeriesScroller(){
	getPlotData('TheBackEndMethod', 2);
	interval = setInterval(update, updateMillis );
	updatingPage = true;
};

function update() {
	getPlotData('TheBackEndMethod', 2);
};

function getPlotData(pageName, pageNum){
	$.ajax({
		method: "POST",
		url: "TheBackEnd",
		dataType: "text",
		async: false,
		data: {pageName:pageName, pageNum:pageNum},
		success: function(response) {
			response = eval(response);
			//alert(response); 
			$.plot($("#placeholder"), response, { });
		}
	});
};

We cancel the interval as soon as any other page is requested by testing for updatingPage. For added robustness and production quality we’d also add timers to abort any backend requests that do not arrive in a specified time and do more thorough handling of the ajax response state.

We would also handle any latency stutter by retaining the last known dataset and ‘flatlining’ it over the missing time-series, perhaps cancelling the updates after a certain number of missed updates.

It took the best part of a half-day’s effort to come up with that – we’re back-end server developers after all – but it was time well spent.

Hope you find this helpful.