Eagle was a senior developer. Not a formal leader, just guy keeping development going, knowing all dark nooks of our app, build scripts and interfaces; all stuff. A go-to fellow. He was proud owner of SUN Ultra 10, with whole gig of RAM, and he always built the final release of the system. Because his workstation was, pretty much, the only one capable enough to build whole system within less than half a day. He was a gate keeper and an expert.

Eagle didn’t talk much. He rather showed in his cubicle early morning, typed with the speed of machine gun and bah! Next feature ready. We, all the greenhorns, pulled his new stuff from CVS immediately and tried understand all that brilliantly esoteric bits. He was the only guy who actually actively used all cast types in C++: dynamic, reinterpret, static and const casts. And he understood the difference. He was probably the only person who understood this on planet Earth (OK, maybe Bjarne Stroustrup knew this stuff, too, however I’ve never talked to him, so I’m not sure:). Huh! That was mastery. Truly. And this was how Eagle had communicated with the rest of the world. Through the stuff he was working on; the code. He checked bits into repository, we pulled it and tried to understand. A communication loop. Or stream, strictly speaking. Because there was no feedback back-channel closing the loop. Eagle didn’t care much about our code, granted we followed his ideas and style. And we followed.

One day our manager popped in and communicated we have to start working on a brand new module. And we started. We formed small team: Eagle, Paw – a competent fellow developer, and me – the novice guy. Next day Eagle showed up, and he didn’t start coding immediately, as he usually had used to. He stared at me for a while, came to my cubicle and put white, a little bit fatigued paper brick on my desk. A book with friendly, blue letters: “Design Patterns: elements of reusable object-oriented software”. And he went back to his Ultra 10, and started typing. After few seconds I got e-mail: “We gonna use Design Patterns in this new module. Read this book. Regards. Eagle”. Holly! Eagle emailed me. Can You believe? And we gonna use Design Patterns, ain’t this cool?! ‘Design Patterns’…Wait! ‘Design’ what? What the hell are ‘design patterns’? And remember this was in late nineties, the prehistory. No google search, no wikipedia and all this usual stuff we are so used to nowadays. My altavista search didn’t cut it; neither my experience from university days. Nothing. “This must be even more esoteric stuff than all that C++ casts we used”. I thought.

This afternoon I brought “Design Patterns” book with me and started reading. Indeed, this was more esoteric than all this ‘C++ casts’ stuff. Lots of high-level ideas, with names, in the form of ready-made recipes. And all that taxonomy (and Yes, I looked up paper dictionary to discover what ‘taxonomy’ actually meant. Remember, no google search back then). Lots of examples in Smalltalk. Wow! Smalltalk! I’d used Smalltalk during one of my university courses. Wrote few trivial, hello-world size projects. Didn’t even know somebody used this stuff in real-life. Huh! Those GoF guys must be smart, they use Smalltalk! And Eagle asked me digest this book. Cool! Two days later I was speaking patterns. Ha! Design patterns. I’ve seen patterns everywhere and was ready to put this stuff into my code. I was wondering how the hell I survived without patterns actually.

Next day we gathered: Eagle, Paw – a competent fellow developer, and me – now design patterns guy. Design patterns and… still novice. Anyhow. We brainstormed our high level design, full of patterns. The only guy who had a problem with all those patterns was… Paw. Meh! Eagle and me voted for patterns, Paw was against. But eventually all that Paw’s ‘Why?’ annoying questions were refused. Didn’t he know GoF guys are smart, so we were, using their patterns, huh?! Boring. Eagle took final decision. Use design patterns. Period. Paw went away with his opinion in his mind, anyhow.

We started coding brand new module. Full of patterns. We built beautiful architecture with all possible levels of abstraction. We created every single object with Builders. We glue stuff with Bridges and Delegates. We implemented business logic with Chain of Responsibility. We built SQL queries with Visitors. We implemented security with Proxies. Everything with grain of Singletons and MVC. Just to name few patterns. You know… all this smart stuff. Let me rephrase: all this smart stuff. Literally. We actually worked two months more than expected and produced half the code whole system had so far. But who cared? We did great stuff, right? At least I was proud of this.

Few months later Eagle left company, and me and Paw eventually ended up with this design patters legacy. Alone.

Fast forward. Quite few brand new modules later… Now, I walk Eagle’s shoes.  I’m the expert. I’m go-to guy. I even understand all C++ cast types. Huh! And I drive all development, including our shiny design patterns sandbox. Still have to support it. But guess what? It’s the worst module in whole system. Seriously. How come? Somebody did something wrong in the meantime? Somebody has broken beautiful architecture? Sneaked some wrong ideas?… Antipatterns? Nope. Nobody even wanted to touch it because every single change took literally weeks. Almost nothing changed since its inception. So what happened, then? Nothing. It’s actually the same, original module we wrote few years ago. Untouched. Oh…wait… there were quite a few workarounds carefully bypassing this stuff. It stayed in its own vacuum.

It was quite few Years, and few gigs ago, in the middle of all that Patterns mumbo-jumbo. Some Patterns became antipatterns since then. And we eventually drank our cool aid. But guess what? I like this story. It taught me more than all Patterns books I have read. I learned that patterns, without enough care, might become harmful. Astonishingly harmful. Even more astonishingly harmful without proper communication and reasoning. It’s what I call a Patterns Antipattern. What this story all about: blindly followed ideas, just because someone smart wrote a book about them. And someone had wild idea to check if all this stuff work together. And won’t implode. And someone else followed. Do not try this at work. Avoid Patterns Antipattern.

So… be the Paw, not the Eagle! Challenge ideas. Don’t drink cool aid without asking questions. Paw, thanks for the lesson, pal!

Secondly I’ve become Dzone MVB (dzone.com Most Valuable Blogger). Huh, being a MVB, together with such great bloggers like Ted Neward, Neal Ford, Adam Bien (just to name few) is quite challenging too. Expect more, and high quality content . My first Dzone entry, with my rant about state of JEE 6 SDK sparkled quite heated discussion on javalobby with some tough reactions, and massive, hot discussion with JEE proponents (which I’m too, actually;), including JCP member and Caucho Resin lead, Rahman Reza. Overall, despite some initial rude reactions, I’m happy that my rant sparkled such a interesting discussion .

The problem with SDK

It’s not SDK actually; it’s just Glassfish bundled with some documentation, examples and random APIs. That’s it. Here’s what You get with JEE bundle:

.org.opensolaris,pkg
bin
glassfish
install
javadb
mq
pkg
uninstall.exe
uninstall.sh
updatetool
var


Ascetic, isn’t it? Expected some standard stuff like lib or, better, dist directories? Some jars or APIs? Or some form of readme, or docs? It’s SDK, the Software Development Kit, right? It should have all that usual stuff somewhere. Well, sort of. If You’re stubborn enough and lookup Glassfish directory, even though don’t expect too much.  This is “enterprise” stuff,  it can’t be just as simple as picking some jars and start working with it . So where are all expected JEE APIs?  I’m not quite sure, except some mere suspicion. After quick bashing it turned out that $SDK_HOME/glassfish/modules contains most interesting bits. Most… except, believe me or not, JMS and JTA APIs. Surprised? Most probably. SDK comes without core stuff like: JMS and JTA APIs. Or again, maybe it’s just me who failed? Would love someone point out I’m wrong. I know, I know: I can find all this stuff here. But isn’t it SDK supposed to come with?

Even though $SDK_HOME/glassfish/modules contains something… it’s Glassfish runtime, and I have no confidence what all those jars actually are. Are they official APIs, or maybe OSGI bundles, or is this Glassfish specific stuff, RIs, or something else? I’m not sure, neither my google is. Anyhow I don’t think server runtime is the best place to throw APIs in. Thanks God SpringSource guys didn’t mix Spring Framework with Tomcat distribution.

How it should look like, then?

Enough ranting. How to fix it? Take some time and take a look how does competition work. Just create simple, lightweight SDK (“lightweight” is the main theme behind JEE 6 after all, isn’t it?). Just bundle APIs, documentation and samples (and keep separate Glassfish bundle, of course). That’s it. JEE is supposed to be portable and universal; not everyone is interested in full Glassfish distribution. Make it simple for beginners and give a chance the JEE SDK passes 5 minute usability test. That’s all.

Hope “Do more with less work” would work better next time… or it’s only me? Should I RTFM? Anyone care to comment?

To make long story short: I do believe into real Software Architects. Unfortunately there is no single, agreed upon definition what Software Architect actually is. And that’s the problem Sergey spotted. The Software Architect role is not only often unclear, sometimes blurry, but worse, has depreciated over the years. The part of the problem is lack of formal definition, but other part is that often this title means nothing, except somebody, somewhere thought it would be nice to put such a ‘Software Architect’ tag on somebody’s visit card. It’s not uncommon to become Software Architect overnight, just because one finished few projects successfully in a row or did something extraordinary like that. That’s the problem. I see this recurring pattern especially in consulting companies. You can earn Architect title for nothing.

What makes somebody the Software Architect? It’s not a title on a visit card definitely. Fortunately I have met brilliant software architects during my life and actually only few of them hold ‘Software Architect’ moniker. If I had to point out one, single quality those sharp guys had in common – it would be: Big Picture. They had seen Big Picture; the whole thing. Be it module (Software Architect), single system (System Architect), whole solution (Solution Architect), some problem domain (Domain Architect) or even whole corporation (Enterprise Architect). They had this rare ability of seeing that thing, they were working on, was not just simple sum of parts, it was something more – the whole thing. In every possible aspect: functional, non functional, technical and, yes, business. And they were steering others into the right direction. With their help, things went smoothly and everybody felt much more comfortable. That was precious and encouraging. They were Lead Developers, Senior Developers, some of them where Analysts, and few of them were actually ‘Architects’. So I should have actually said: I believe into Software Architecture, not necessary into Architects. Show me one, and I can tell if he or she is worth the title she or he holds. Actually this is the question of seeing, not believing.

This time things seems to be different, though. Real innovation comes from the very core layer of JSR-315 with notion of asynchronous servlets. Unfortunately as few times before, JCP put great marketing effort to make this piece as much esoteric as they could. At least from my perspective. Documentation is sparse at best, examples are hard to understand and We, the community, are not far better in this very matter. Admittedly there are pretty good examples and introductions around the web, but none of them draw big picture.

So this is my first instalment of trilogy. I want to cast some light on this topic, will try to put all puzzles together to easily grasp whole asynchronous servlet concept: what’s it, what’s not, what’s good, bad and the ugly. In the second part I’ll show some real life examples of asynchronous servlets (and clients), introduce higher level frameworks and techniques: cometd and Bayeux. In the last part I will show, what I’m particularly most interested in, how this whole idea fits into web services world. But…

First things first

Before we grasp the idea of asynchronous servlets, we have to understand basics: how HTTP servers actually work. I won’t dive into nuts and bolts here – I’ll briefly describe some nuances of HTTP protocol handling, and how modern HTTP servers work. Just to understand what’s so innovative in asynchronous HTTP processing. If You are interested in more in-depth analysis i highly recommend great article by Xinyu Liu.

Thread-per-connection

Nowadays most HTTP engines use some variation of so called thread-per-connection pattern. Basically, when HTTP connection arrives, HTTP server picks single thread, of out a pool, to handle it till it’s committed (meaning all response data are written and response output stream is closed), then thread is freed-up back to the pool.  This approach scales pretty well, but it has some drawbacks, though. One of significant aspects of HTTP 1.1 protocol is keep-alive option. Client and server may (and they usually do) negotiate to keep connection open and span multiple HTTP request and responses. In traditional approach this means that thread can potentially be allocated for significant amount of time, even if it’s just idle. Fortunately this problem might be mitigated with…

Asynchronous IO and Thread-per-request

With Web 2.0 and AJAX, web application comes with far richer user experience and, usually, needs hundreds of request to render and refresh its content. This raises performance bar significantly. Fortunately, most HTTP server vendors have undertaken tremendous amount of research, and they reach stunning numbers in the of concurrent connections they can handle simultaneously. With help of asynchronous IO it is no longer needed to maintain thread up until connection is closed. As long as request processing is finished, processing thread can be reused, while IO stuff can be handled asynchronously. This effectively means that even Client and server keep connection alive for significant amount of time, we don’t need any thread associated with it. Fortunately, this is done completely transparently from Servlet API perspective.

The need of asynchronous servlets

So why we actually need asynchronous servlets? Well, Web 2.0 redefined how web applications work, but what’s even more important, Web 2.0 redefined Web architecture as well in the terms of rich user experience, high interactivity and low latency messaging. Asynchronous servlets address low latency, asynchronous messaging aspect.  This aspect has quite broad use-cases, from real-time messaging services (like google wave) to enterprise level services, like trading, stocks etc.

Quote Service

To make this whole story more concrete I will illustrate above idea with simple Stock Quote Service and Stock Quote Widget. Stock Quote AJAX Widget will present real-time share price provided by Stock Quote Service. We will use Observer, or more broadly publish/subscribe pattern – so every Client would subscribe given channel and watch stock prices. Of course, we cannot guarantee how frequently we get information from our backing stock exchange system. It’s asynchronous. Since we don’t have real connection to real-time sock quote service, we will mock it with simple MockStockQuoteService:

package org.restfusion.jsr315.stockquote;

import java.util.Observable;
import java.util.Random;

public class MockStockQuoteService extends Observable implements StockQuote, Runnable {

public void run() {
setChanged();
notifyObservers();
}

public String getSymbol() {
return "GOOG";
}

public Double getValue() {
return Math.abs(new Random().nextDouble());
}
}

For sake of brevity, Observable holds simple stock quote data behind this simple interface:

package org.restfusion.jsr315.stockquote;

interface StockQuote {
public String getSymbol();
public Double getValue();
}

And for testing purposes it will just return some random share price for “GOOG”. Nothing unusual so far.

Our first, naive approach would be like this: write simple synchronous servlet and poll it every second from within Our widget using some flavour of AJAX toolkit. Our servlet would observe MockStockQuoteService and return information if something has changed, or not. This is actually pretty simple and common approach. Here’s how it’s gonna work:

It’s clear that only portion of responses would carry interesting information, and the rest would effectively just say ‘nothing changed’. As You can see only second pair (in red) of request/response carries some significant value. So far so good. Now lets imagine that one day someone find Our widget useful and put it on igoogle. We have to prepare to face with something of google scale, right? Let’s be modest, say 10000 clients will subscribe Our service. Now we have to handle (potentially) 10000 requests per second, ergo we have to scale Our server to handle 10000 threads. Pretty impressive number, isn’t it? What’s worse, most of those threads will do virtually nothing except saying: ‘nothing changed’. It’s where asynchronous servlets shine.

Long polling

The trouble with Our initial approach comes from the fact that we use synchronous approach to solve asynchronous problem. Stock quote service is asynchronous by its nature. We don’t ask Stock Exchange for stock price, they will usually give us that information. It would be nice to follow this approach on Servlets API – listen for events, rather than poll. Now our second approach would be Long Polling. Here is how it works:

Compare this with previous one. Can You spot the difference? Right! Now, every response carries some valuable information. Note that we actually establish HTTP connection and wait for response up until we have some interesting information available (in this case new stock price). We do poll still, but at much lower ratio, and are waiting for some valuable information. When information arrives we consume it, and poll once more. Potentially ad infinitum. Here is sample Stock Quote Widget:

<html>
<head>
<script type="text/javascript" src="jquery-1.3.2.min.js"> </script>
<script type="text/javascript">
$(document).ready(longPoll);

function longPoll() {
$.get("/jsr315/longPollingStock", {}, function ( data, status ) {
$("#quoteWidget").html(data);
longPoll();
} );
}
</script>
</head>
<body>
<div id="quoteWidget"></div>
</body>
</html>

jQuery crash course: longPoll function simply sends GET requests to Our servlet and waits for completion (new stock quote). Once it arrives, it updates our widget (quoteWidget div) and initiates next poll.

Now, finally, lets look how Our server side stuff support this approach.

@WebServlet(urlPatterns = "/longPollingStock", asyncSupported = true)
public class LongPollingStockServlet extends HttpServlet {

static MockStockQuoteService stockQuoteService = new MockStockQuoteService();
static ScheduledThreadPoolExecutor worker = (ScheduledThreadPoolExecutor) Executors.newScheduledThreadPool(1);

static {
worker.scheduleAtFixedRate(stockQuoteService, 0, 5, TimeUnit.SECONDS);
}

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
AsyncContext ctx = req.startAsync(req, resp);
stockQuoteService.addObserver(new QuoteObserver(ctx));
}
}

Here are two things to note:

asyncSupported = true – this flag instructs HTTP container that this servlet might support asynchronous processing.

AsyncContext ctx = req.startAsync(req, resp) – this is where we actually initiate asynchronous processing by calling HttpServletRequest#startAsync. As servlet documentation says:

Calling this method will cause committal of the associated response to be delayed until AsyncContext#complete is called on the returned AsyncContext, or the asynchronous operation has timed out.

And that’s the whole thing. We delay request  committal up until we process this request in the future, asynchronously. If we wouldn’t do above, Our HTTP request would be effectively committed right after doGet method is finished. This time we postpone request processing until Our  QuoteObserver gets notified by MockStockQuoteService asynchronously. At the same time we finish doGet and give processing thread back to the HTTP container. We register QuteObserver on MockStockQuoteEngine which would notify all its observer every five seconds, as You might deduced looking at  ScheduledThreadPoolExecutor. More on this later, now lest assume that’s just black box, it would notify us asynchronously. Just don’t look into that box right now .

Now, let’s look where actually something useful happens:

public class QuoteObserver implements Observer {

private AsyncContext ctx;

public QuoteObserver(AsyncContext ctx) {
this.ctx = ctx;
}

public void update(Observable observable, Object arg) {
StockQuote quote = (StockQuote) observable;
try {
ctx.getResponse().setContentType("text/html");
Writer writer = ctx.getResponse().getWriter();
writer.write("<span>Symbol: " + quote.getSymbol() + ", Price " + quote.getValue() +  "</span>");
observable.deleteObserver(this);
ctx.complete();
} catch (IOException e) { /* ignore */ }
}
}

Here we are doing pretty useful stuff. Once we get notified, we inform Our subscribers that we just got new stock prices. And effectively causing response committal calling ctx.complete(); (and we are removing this Observer from Observers pool, because we don’t know if client would choose to poll anymore). Note that we do this processing in completely different context.  doGet unwound probably few seconds before, and Our previous processing thread is serving next request, hopefully. Now…

Putting it all together

Finally, when You deploy this application and reach Your Stock Quote Widget You would see something like this (and the price part will be refreshed every five seconds).

Don’t look into the box

Lest step back a little and take a look at ScheduledThreadPoolExecutor. You may probably wonder: why actually use another thread just to handle request asynchronously while we said, that all that stuff is about limiting thread usage, right? Now, isn’t it counter-intuitive to spawn other Thread (or even Executor) just to handle request asynchronously? Well, that’s pretty valid question. It’s just for sake of brevity. In the next instalment of this series I will show how we can integrate this technique and available asynchronous options, like JMS and asynchronous methods (new in spring 3.0 and EJB 3.1). For now… just don’t look into the box.

To sum this part up: this approach seems to be more reasonable. It reduces significantly polling frequency, and request carry only important data. Though, it still has some important holes. First of all, subscribers are not durable. We can miss some information between subsequent polls. This is probably not the problem in Our, simple case, but it might be in plethora of other scenarios. Fortunately here comes…

HTTP streaming

Idea behind HTTP streaming is actually pretty simple. Client initiates connection and keeps it open possibly ad infinitum. Subsequent messages from servlet to client are sent using previously allocated communication channel. Conceptually we newer commit response. Here is how it works

This seems to be better approach that previous one, we eliminate polling, and once subscribed, we won’t miss communicates. So how Our servlet would look like? Exactly like before:

@WebServlet(urlPatterns = "/streamingQuoteService", asyncSupported=true)
public class StreamingStockServlet extends HttpServlet {

static MockStockQuoteService stockQuoteService = new MockStockQuoteService();
static ScheduledThreadPoolExecutor worker = (ScheduledThreadPoolExecutor) Executors.newScheduledThreadPool(1);

static {
worker.scheduleAtFixedRate(stockQuoteService, 0, 5, TimeUnit.SECONDS);
}

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
AsyncContext ctx = req.startAsync(req, resp);
stockQuoteService.addObserver(new StreamingQuoteObserver(ctx));
}
}

And here comes StreamingQuoteObserver

public class StreamingQuoteObserver implements Observer {

private AsyncContext ctx;

public StreamingQuoteObserver(AsyncContext ctx) {
this.ctx = ctx;
}

public void update(Observable observable, Object arg) {
StockQuote quote = (StockQuote) observable;
try {
ctx.getResponse().setContentType("text/html");
Writer writer = ctx.getResponse().getWriter();
writer.write("<span>Symbol: " + quote.getSymbol() + ", Price " + quote.getValue() +  "</span>");
writer.flush();
} catch (IOException e) { /* ignore */ }
}
}

Note that we keep connection open all the time (here is no ctx.complete()).  On client side we will get stream of events:

<span>Symbol: GOOG, Price 0.8843558760137272</span>
<span>Symbol: GOOG, Price 0.30434186388689377</span>
<span>Symbol: GOOG, Price 0.673062794397491</span>
...

Now it’s client’s responsibility to parse incoming stream of events and update Our Stock Quote Widget. Unfortunately there is no simple, reliable, standard way to do such a thing in modern browsers. Basically most browser limit partial response retrieval. Note that, in previous example, this function

$.get("/jsr315/longPollingStock", {}, function ( data, status ) {
$("#quoteWidget").html(data);
longPoll();
} )

gets called when response is committed. That’s why this technique is somewhat limited right now. Of course we can still use this technique with other clients, including java ones. Here is how this looks like with curl:

$ curl http://localhos:8080/jsr315/streamingQuoteService
<span>Symbol: GOOG, Price 0.8843558760137272</span>
<span>Symbol: GOOG, Price 0.30434186388689377</span>
<span>Symbol: GOOG, Price 0.673062794397491</span>
...

The Good, The Bad and the Ugly

Let’s try to sum all above this. Definitely asynchronous servlets spec opens whole new era and possibilities. Historically few HTTP Servlet container vendors built their own, incompatible implementations. Now it’s standardized, and that’s probably makes JEE far ahead of competition in this matter. That’s good.

The bad thing is this whole new servlet processing model comes at a cost

• Some basic assumptions, which were true before are any more what would lead to some important implications
• Also some simple things are not so simple anymore

Here also comes the ugly bit. But it’s not necessarily associated with Servlet 3.0 spec, but rather with the architecture part.

You have probably noted before that HTTP streaming is rather hard to achieve

But also it’s somehow hard to proxy Your asynchronous application through intermediaries, which is really problematic since most companies I know front their application server through bunch of intermediaries (proxies, load balancers, application level firewalls etc.)

Here are also some esoteric limitations of HTTP spec, followed by some clients, which limit this approach also

But problems aside. I believe asynchronous servlets open new, broad horizons, but most likely average developer would not touch asynchronous API directly. In my opinion it is solid foundation for higher level frameworks, which will be subject of next instalment of this saga. Enjoy and stay tuned!

Further reading

If You are interested in this topic, I strongly recommend follow Greg Wilkins (of jetty fame) blog and cometd.org. This is valuable source of information, which this entry bases upon heavily.