I was at TheServerSide.com’s Java Symposium last week and got a fascinating perspective on where high-end Java apps are headed, and the infrastructure that will be needed to support them.

Hedge funds and more traditional financial service firms all are deep into creating what they call grids (they’re not talking about time sharing across occasionally idle computers) for doing performance intensive stuff like programmed trading. Imagine hundreds of motherboards ripped out of servers and velcroed into racks, all running stripped down Linux cores and highly tuned Java Virtual Machines (JVMs) on top.

Kirk Pepperdine, veteran Java performance tuning consultant, discussed the growing reliance on non-volatile memory over disk (you’ll have to register to download the talk, but it’s pretty easy) to reduce latency in these applications. NVRAM can mean high-memory footprint motherboards, as we are now in the hundreds of gigabytes for some systems. But it can also mean solid state disk, which maybe undergoing one of its periodic surges. Not only are people trying to put entire programs in memory, but as much data as they can, too.

Typical databases of, say, 1990, would easily fit into today’s NVRAM. But not necessarily the databases of today, which have grown into millions and millions of rows. But objects and services give developers an option they are starting to explore: stuffing the data into the objects themselves and stuffing the objects into solid state caches, whether that’s onboard or outboard.

One thing that is making this possible is using the grid. Some shops are using what is effectively distributed memory, as Iona Technology’s technical director, John Davies, pointed out. Products like GiagSpaces, Oracle’s Coherence and GemFire create and manage a memory space across many machines.

Another take comes from Azul Compute Appliance, your classic black box.The company has attacked a specific and troublesome problem with Java apps – garbage collection. The JVM can pause for as long as 30 seconds every few minutes to do its thing. Even if garbage collection isn’t that extreme, brief pauses are not acceptable for high-performance trading apps. So Azul designed its own chips that use a proprietary instruction set to make garbage collection non-disruptive. They stuff up to 768 of them in a box, with up to 768GB RAM. Software on the hosts redirects calls to the JVM to the Azul box, where it runs as if it was on the host.

Azul’s boxes run in the tens and hundreds of thousands of dollars, but they can run many JVMs at once. According to benchmark’s run by Pepperdine, they definitely turn up the heat on Java apps.

Server managers in the big Wall Street firms are already dealing with these new concepts, and you can expect them to migrate outward in coming years. Just as storage and networking have been disaggregated from the computer, some amount of memory and processing, at least for specialized purposes, may also migrate on to the network (probably 10GigE or InfiniBand). One of the big reasons pointed out by Pepperdine: with multicore processors, clock speeds are not increasing. Therefore, app developers must seek other ways to increase performance.

Given that parallelism is still only minimally doable for all but the rocket scientists, techniques like greater use of caching are bound to gain popularity, so it’s probably worth your while to start investigating this whole technology area. At the architecture level, there’s plenty to understand: should you virtualize and cluster at the app level, or go the route of Virtual Iron or ScaleMP, which allow you to concatenate multiple physical machines into a single large VM?

Never a dull moment.