Creating Comet applications with ASP.NET
Sometimes it’s useful to hijack a particular technology to do something it was never intended to do. Comet does exactly that, and it’s not for the faint of heart. In a nutshell, “Comet-style” applications use features of the HTTP request-response communication model to morph it into a streaming context whereby the server can essentially push data to the client (instead of the client request data from the server). Wikipedia puts it well:
Comet is a programming technique that enables web servers to send data to the client without having any need for the client to request it. It allows creation of event-driven web applications which are hosted in the browser.
When faced with the need for a real-time web application, there will always be trade-offs between performance and scalability. One possibility is to implement a pull architecture that periodically polls the sever. The benefits include higher scalability per server with the trade-off of increased latency and lower performance (in terms of information availability). On the other side of the coin is a persistent connection which allows the server to push streaming data to the client. Benefits include great performance (low latency), but low scalability since every client connection is held open for long periods of time which today’s web servers are not optimized for. Lightstreamer.com has a good white paper describing the pros and cons of the various approaches (including some hybrid methods), and, of course, why theirs is the best and you should buy it. Truth be told, I think they have a very compelling product offering. (I haven’t tried it, though, just read about it.)
So how can a Comet ASP.NET application scale? Very poorly. Since each client connection holds open a dedicated thread from the ASP.NET thread pool, there are really only two ways to scale:
- Increase the number of threads in the thread pool to the maximum. This is done in the <processModel> element located under <system.web>. Set the maxWorkerThreads attribute to the max of 100. This is a per-CPU value, so assuming you ran this on a quad-processor server, your application has the potential to scale to 400 concurrent users per server. Assuming you grab a barebones Dell PowerEdge 4-proc server for $10,000 USD, you’ll be paying $25 per concurrent user. Fantastic numbers! Not.
- Add more servers – in real-time applications it’s typically not feasible for each client connection to end up on different servers. Once the initial connection has been made, the client is “pinned”. In cases like these, additional servers can be added but they have to live behind some “sticky load balancing” routers.
The other option is to write a custom web server that is optimized for concurrent connections. I wonder if WCF could handle the challenge, and while it seems like it could do better than IIS+ASP.NET, it appears you wouldn’t necessarily get a huge bang for your buck. Some real-world testing would be much more definitive however. Here’s a few links talking about streaming data back over an HTTP connection with WCF:
Just for fun I’ve created a very brief ASP.NET sample application that demonstrates a basic Comet implementation. It streams the current server time back to the web browser, which just displays the value. Most of the work is in javascript on the client page. The key ASP.NET components are simply the disabling of output buffering, and flushing the response stream as appropriate. I didn’t use any “fancied up” javascript libraries so as not to exclude anybody who isn’t familiar with any particular library.
And now, after having bashed my own approach, here it is. Create a new Web Site in Visual Studio 2005. Add a page called simply “Service.aspx” and put this code in the code-behind:
public static string Delimiter = "|"; protected void Page_Load(object sender, EventArgs e) { Response.Buffer = false; while (true) { Response.Write(Delimiter + DateTime.Now.ToString("HH:mm:ss.FFF")); Response.Flush(); // Suspend the thread for 1/2 a second System.Threading.Thread.Sleep(500); } // Yes I know we'll never get here, // it's just hard not to include it! Response.End(); }
Add a new HTML page to the project (I called mine “ajax.html”) and overwrite the contents with this: (Explanation to follow.)
(NOTE: The formatting got screwed up during a blog import to Wordpress – download the zip file linked below for “readable” code. 
)
<html xmlns="http://www.w3.org/1999/xhtml" ><head> <title>Comet AJAX Sample</title> <script language="javascript"> function getData() { loadXMLDoc("Service.aspx"); }var req = false; function createRequest() { // branch for native XMLHttpRequest object if(window.XMLHttpRequest && !(window.ActiveXObject)) { try { req = new XMLHttpRequest(); } catch(e) { req = false; } // branch for IE/Windows ActiveX version } else if(window.ActiveXObject) { try { req = new ActiveXObject("Msxml2.XMLHTTP"); } catch(e) { try { req = new ActiveXObject("Microsoft.XMLHTTP"); } catch(e) { req = false; } } } } function loadXMLDoc(url) { try { if (req) { req.abort(); req = false; } createRequest(); if (req) { req.onreadystatechange = processReqChange; req.open("GET", url, true); req.send(""); } else { alert('unable to create request'); } } catch (e) { alert(e.message); } } function processReqChange() { if (req.readyState == 3) { try { ProcessInput(req.responseText); // At some (artibrary) length // recycle the connection if (req.responseText.length > 3000) { lastDelimiterPosition = -1; getData(); } } catch (e) { alert(e.message); } } } var lastDelimiterPosition = -1; function ProcessInput(input) { // Make a copy of the input var text = input; // Search for the last instance of the delimiter var nextDelimiter = text.indexOf('|', lastDelimiterPosition+1); if (nextDelimiter != -1) { // Pull out the latest message var timeStamp = text.substring(nextDelimiter+1); if (timeStamp.length > 0) { lastDelimiterPosition = nextDelimiter; ProcessTime(timeStamp); } } } function ProcessTime(time) { var out = document.getElementById('outputZone'); out.innerHTML = time; } </script></head><body onload="getData()"> <b>Server Time:</b> <span id="outputZone"></span></body></html>
It works by making a connection to the server and processing the response as it streams back. “getData()” will initiate an AJAX request for the service page, which will continue to stream data back. When new data is available, the client parses out the new data beginning from the last location in the response that was processed. If the response text gets too large, the connection is severed and a new one made (semi-polling — one of the hybrid approaches) as processing an ever increasing response body can ultimately slow down the web browser unnecessarily.
There is a lot more to a successful Comet implementation, obviously, and I encourage you to read the information available at Wikipedia including the reference links.
Download the 3 files (ajax.html, Service.aspx, Service.cs) here: ASP.NET Comet Example.zip
August 22nd, 2007 at 11:29 pm
@Mario – it turns out that has to do with IE, and not IIS vs. Cassini. I’m not sure what the resolution is, I haven’t spent the time to figure it out. If you do find out, post back here!
Thanks
August 22nd, 2007 at 11:29 pm
Aaron,
It does work with Firefox. You are right. However, the most disconcerting thing is that Microsoft, having implemented a native XMLHTTP object in IE7, has not made provisions to allow partial responses. See Native XMLHttp readyState for more on the subject. Following what they described in the above documents, I tried using other properties such as responseStream and responseXML to no avail.
August 22nd, 2007 at 11:29 pm
Aaron,
Nice idea. I cannot get your code to work. When a response is received and the state==3, reading the responseText throws and exception:
“The data necessary to complete this operation is not yet available”
Could you please explain what version of IIS are you using and how is it configured?
August 22nd, 2007 at 11:29 pm
Aaron,
Nice idea. I cannot get your code to work. When a response is received and the state==3, reading the responseText throws and exception:
“The data necessary to complete this operation is not yet available”
Could you please explain what version of IIS are you using and how is it configured?
August 22nd, 2007 at 11:29 pm
I think that might be viable, but it really would depend on just what the async operation was doing — would it be making a call to a web service? A database? If so, sure, that could work. If it’s just “waiting” for data (depending on how the data arrives), then somewhere somehow within the same process space a thread will be tied up, and your performance problem is back again.
An alternative could be to implement a service of some sort (windows service, not web service) that is built specifically to handle large numbers of incoming connections, and have ASP.NET async pages hand off control to the service, invoking the callback when new data arrives (as you said).
August 22nd, 2007 at 11:29 pm
I may have missed something, but wouldn’t using the asynchronous capability of ASP.NET allow you to greatly reduce the number of threads (from the thread pool) that are actively being used/blocked? E.g. say a new comet request comes in. We could create begin an async operation (using ASP.NET’s async page feature), so that the thread for that request is returned to the pool. The callback could be triggered when either there is data to be sent, or the timeout kicks in. With a few more smarts, you could throttle the maximum number of callbacks that can be fired concurrently, to avoid “choking” the whole thread pool when data needs to be sent. I’ve not tried this in practice, but am curious whether anybody else has, or whether anyone can see a reason why this would not work or be an sensible approach.
August 22nd, 2007 at 11:29 pm
Huh – thanks for the info! Without investigating the discussion about it (bug vs. feature) I’d initially classify it as a bug, since the spec is pretty clear that referencing that property should not throw an exception.
August 22nd, 2007 at 11:29 pm
Unfortunately it is not related to the host running the application; it’s just the way XmlHttpRequest implemented in IE (there is a lot of discussion on whether it’s a bug or feature).
Luckily – there is a trick letting to implement http push in IE – it’s streaming of either script sections or data into the hidden page IFRAME. I going to publish example of such solution on my blog soon.
August 22nd, 2007 at 11:29 pm
You’re right – but I believe (untested) that the error you mention will only occur when you run the application on localhost. I didn’t have a readily available ASP.NET server I could tinker with to try it, though.
I was surprised to get that error, because according to the spec it shouldn’t throw an exception like that.
August 22nd, 2007 at 11:29 pm
Your example would work only in Mozilla, not in IE.
.
In IE – you’ll get exception trying to access req.responseText in processReqChange(). This property is unavailable until connection is closed
August 22nd, 2007 at 11:29 pm
Right, the client would need to be smart enough to know if and when to re-open the connection. If it was any amount of time, maybe the server could tell the client “oh I think it’ll take about x seconds” and then the client would close the connection and then reopen the connection then. And if the server got done early, then he could queue the response until the client reconnected. This would definitely help in scalability since x would definitely get bigger as the load on the server increased.
August 22nd, 2007 at 11:29 pm
The caveat with that approach is that the whole idea behind Comet is for the server to *push* data to the client. Since web connections are initiated one-way only, how would a client know when to re-open the connection if it was closed due to a lack of use?
And if you don’t absolutely need Comet-style push streaming, then by no means should you be using this approach.
August 22nd, 2007 at 11:29 pm
About your two ways to scale Comet… Wouldn’t there be a third option here: Utilize persistent server connections responsibly. If you only kept server connections open when you knew the server connection would be used and if you timed out connections appropriately, you would cut down on a lot of unnecessary traffic and open connections. Of course that would be difficult if the response time from the server really is that varied or if you don’t know how many responses the server may send back or if the server simply does send a lot of data pretty frequently, but if that’s the case then maybe you have some other design issues to iron out.
August 22nd, 2007 at 11:29 pm
Wait, I’m confused… Just how does this relate to powershell?
September 11th, 2007 at 7:38 pm
I think that the asynchronous capability of ASP.NET allow you to greatly reduce the number of threads (like some one here asked). I even tried to access the server with a lot of clients, and measured the number of threads (asp.net threads) – I can say that only 2 or 3 were used, althought there were a lot of clients opened.
March 11th, 2008 at 5:31 am
It don’t work with ie browser,throw the exception “The data necessary to complete this operation is not yet available”.
I try for follow example:
req.responseBody,req.responseStream…
,but failed , why?
April 7th, 2008 at 10:37 am
[...] httpHandlersPresented by Chris Love I have an unrealistic burning desire to personally make Comet efficient in IIS, and I have an inkling that a custom httpModule will do the trick with maybe a Windows service [...]
May 27th, 2008 at 7:18 am
Has anyone tried this with IHttpAsyncHandler? I wondering how well it would scale but I haven’t seen much documentation.
June 11th, 2008 at 1:56 am
Hi Aaron,
It don’t work with ie browser,throw the exception “The data necessary to complete this operation is not yet available”.
Any suggestions, thanks!
August 24th, 2008 at 12:50 pm
pretty clear example for beginners. good job thanks
November 29th, 2008 at 1:11 pm
I have a better solution than Comet applications. Use Flash, it contains the XMLSocket object which can create an open socket connection to a server-side application, which means your application would use an IP address and a port number like any FTP or chat application (but within your browser). Each connection may need to be on it’s own thread, but you aren’t held back by ASP.net threads so you can have 1000+ concurrent connections. Flash will send javascript commands to your client-side web application when a response comes in from the server, and you don’t need find a solution for sending commands back to flash from javascript to make a request to the server (which will not be supported in some browsers). Instead, you can send a javascript xml request (AJAX) to the server and ASP.net can relay the request to the server-side application.
December 2nd, 2008 at 10:27 pm
What’s the point of adding a plug-in to do the trick, mark?
March 3rd, 2009 at 10:25 pm
The example above is somewhat naive. Any Comet server that uses techniques such as thread.sleep is doomed. With more than about 20 users, the .net threadpool will be saturated and spin up new threads. It does this slowly, because it is designed with the idea that the tasks it excutes are short-lived and don’t block, and is attempting to find an optimal number of threads to service the queue. This will lead to high latency connection times.
Using Flash in the manner described above is also not so good because the socket API for flash does not use browser settings for proxy information.
There is a way to do this correctly in IIS. A couple of people mention it above. Deal with the request (quickly) then go async. Thread is returned to threadpool, threadpool remains responsive. When there’s something to reply, end the async op, quickly borrow a thread from the pool and send the response. Most of the time nothing happens. Blocking many threads to do nothing is inefficient, and ultimately the server will spend more time swapping thread contexts than serving data.
The neverending download style is also perilous because several proxy server implementations will not send to the client until the response is complete, and IE’s XMLHTTP also doesn’t give you a look in until the response is complete
So, all in all, the techiques above should be considered harmful.
March 3rd, 2009 at 10:32 pm
Thanks spender. Of course the example above is naive – it’s an example! Keep in mind I was attempting to illustrate the concepts behind COMET, not necessarily promoting best practices. If I had production experience implementing this (and not just doing it for fun) I would have a lot more to share. I look forward to you posting a comment here with a link to your blog post that will contain an exhaustive treatise of the correct way to do things, given that what you described (which is fairly accurate) basically means that true “push” COMET is not possible outside a “long poll” approach.
March 4th, 2009 at 7:53 am
sure thing… just ironing out the deadlocks at the mo.
March 26th, 2009 at 11:01 am
spender, did you get any further with your suggested async approach?
April 22nd, 2009 at 12:42 am
The async approach is used by this control which looks like it has source code available: http://www.codeproject.com/KB/aspnet/CometGrid.aspx
the article specifically says that threads are not blocked… The technique should be able to handle thousands of concurrent COMET connections.