Nice Updates To The Druid Site

The gang that works on Druid have done some nice work on a new website http://druid.io. The focus has been around clarifying what Druid is and isn't and simplifying adoption. Lots more work to be done but we are pleased with how it's going.

Also, Russell Jurney wrote a useful blog covering understanding and getting started with Druid via a Twitter feed.

Meet the Druid and Find Out Why We Set Him Free

Whipped up a blog on Druid. Check it out. Meet the Druid and Find Out Why We Set Him Free

What Would You Do Differently With Reliable In-Memory Big Data?

In The Beginning

Two years ago, Terracotta introduced a transformative new technology called BigMemory, delivering revolutionary advances in both scale and predictability for in-memory data management. Since then, our customers have leveraged BigMemory to eliminate tuning and achieve 1000x per-node jumps in scale—from about 2 GB per node to 2TB per node—all while achieving lower, more predictable latencies.

Moving Forward

Since that time we've layered in additional powerful technologies like on-heap bytes based tuning (ARC) and Search.

Once you start acting on that much data in-memory it becomes important that your able to keep it there. After all, a crash that requires you to rebuild gigs and gigs of data from other sources when the node restarts makes the application impractical. So we recently added a fast, fault tolerant, restartable store (FRS). 

With that kind of data one needs to see what's going on or you'll feel blind and helpless. So we followed up with a secure monitoring and management console to go along with it called Terracotta Management Console (TMC).

The Problem

These pieces when put together create an extremely powerful in-memory data management solution. There was just  one more problem. If people weren't used to storing large amounts of data in-memory/in-process, then how would they ever start to explore what's possible?

Resolution

We decided that the best way to change behavior was to make it free. So you can now download BigMemory Go—and use it in production up to 32 gigs—on as many machines as you desire. And you won't pay us a dime. 

So....

What Would You Do Differently With Reliable In-Memory BigData?

I look forward to finding out. Download BigMemory Go free at http://terracotta.org.

How Do You Say "Ehcache?"

What's the most pressing issues at Terracotta? A challenge that has been dogging us for years? Why of course it's teaching people the proper pronunciation of "Ehcache." Have no fear. Here is a video that will help you to get it right!

Click here and be educated!

What's Apache licensed, improves performance, reduces tuning and is now GA?

I'm glad you asked... Ehcache 2.5 with ARC went GA today. At a couple of megs Ehcache with ARC is lightweight and helps with real tuning and performance issues in Java applications.

Like Ehcache itself I'll keep this blog lightweight.

Here is why you care:

• How Much/No OOME's - Specify how much data you want to cache in bytes for on heap unserialized objects. Ehcache keeps the cache under that size to avoid OOME's
• Runtime, Tune as needed - Specify total heap to use for cache and then, when you feel like it, tweak individual caches for optimal performance. 
• CacheManager - Max Bytes on heap (Percentage of heap or fixed number)
• Cache - max bytes on heap (Percentage of CacheManager or fixed number)
• Lots of caches made easy - Use hundreds of caches without sizing each one individually
• Hibernate people, this means YOU.
• Did I mention NO OOME's
• Fence your heap - Like Virtualized Machines give your app's internals a percentage of the total heap and your users a percentage. Prevent the two from stomping each other.
• People who ship software to others TAKE NOTE.
• Pinning - Pin caches or entries to heap that need to always be available and extremely fast.

Who Should Care?

• Hibernate users
• Lots of caches, impossible to tune each individually
• Spring Annotations users
• Pick the amount of heap you want to dedicate to caching and then just mark the methods that need caching
• Software Shippers (Frameworks, Servers or Apps)
• Don't force users to size caches
• Reserve part of the heap for your internal state and assign a safe size for users
• Application Admins
• Monitor an application in production and do sizing/pinning where needed on the fly

Conclusions

Ehache and ARC are free, lightweight and likely already in your application (embedded in Spring, Hibernate, Cold Fusion and countless other servers and frameworks.) This is the most requested feature set in the history of Ehcache. 

Give it a try!

More Info:

http://www.terracotta.org/products/enterprise-ehcache/automatic-resource-control

http://ehcache.org/documentation/configuration/cache-size

http://scaleaholic.blogspot.com/2011/09/spring-performance-with-annotations-and.html

http://scaleaholic.blogspot.com/2011/07/easy-java-performance-tuning-manage.html

http://www.dzone.com/links/r/new_efficient_ehcache_automatic_resource_control.html

Spring Performance With Annotations and Ehcache ARC

The goal of this blog is to highlight Ehcache's new ARC feature using Spring Annotations.

Terracotta is releasing Ehcache 2.5. It's biggest new feature is ARC (Automatic Resource Control). ARC brings runtime performance tuning and caching to the Systems Administrator while simplifying it for the Developer. I've written a couple of blogs describing what it is and the value it brings:

• Local Caching++ 
• Easy Java Performance Tuning: Manage Objects By Size On The Java Runtime Heap

These two blogs and the Ehcache docs are excellent sources of information on these highly requested/useful new capabilities. 

Getting Started

I wanted to give a brief demonstration of how someone would leverage Ehcache ARC in practice so I pinged the guys who do Spring Annotations and they OK'd me to leverage one of their samples. Spring Annotations enable one to mark methods that do performance sensitive operations to be cached to improve the application's speed. 

In order to follow along with me you'll want to read up on and check out the Spring Annotations sample here: 

http://code.google.com/p/ehcache-spring-annotations/wiki/WeatherServiceExample

Download it and play with it a bit to get familiar with how it works.

This sample is fairly self contained. It show's how to use the various annotations to improve performance in a simple and concise way. What it doesn't talk about is the configuration of the underlying cache in use. This ends up being very challenging in practice. Without help you'll run into at least a few of these 5 problems:

• Crash - Out of Memory Errors can crash your Application
• Pause - Long GC's will pause your application leading to an unpredictable user experience
• Wasted Space - In order to avoid the above two problems you over provision memory wasting ton's of space
• Tuning Hell - Either you or your users spend ton's of time making arbitrary decisions trying to balance out the memory usage of your caches to avoid the above 3 issues
• Poor Performance - Caches don't really help because they are incorrectly sized

Let's look at the ehcache.xml that ships with this sample:

The important parts of this config are from lines 30-31. It sets maxElementsInMemory to 100 and timeToLiveInSeconds to 300.

Why do these need to be set? Let's look at them individually:

maxElementsInMemory=100

When you started your Tomcat instance some amount of memory was reserved in the command line (Xms Xmx). If you didn't set a maxElementsInMemory your cache would grow infinitely until long GC's and then an OOME occur. So maxElementsInMemory is a resource management choice. But why 100? Is it because you know how big your entries are and you know 100 is exactly some percentage of the heap? More likely it's a guess. Probably a low ball guess so that you don't OOME wasting a whole bunch of heap. Now say you have 10's or 100's of caches that hold objects of varying size. How much harder is it to set the maxElementsInMemory now?

timeToLiveInSeconds = 300

This is a data freshness control. If your using an unclustered cache and multiple nodes then objects are likely being updated in a DB and the data in your cache can get out of date. How out of date? Depends what you set your timeToLiveInSeconds to.
NOTE: Keeping all nodes up to date is one of the reasons people use clustered caches.

Configuring a cache using the above controls makes an application sensitive to use case changes, changes in JVM heap settings, and number of caches. It also leaves the user of the application with a choice of leaving a lot of headroom to handle the worst case, risking OOME's and forcing the admin or user to tune at every deployment.

Ehcache ARC Gets Performance Without All The Tuning

These are the problems that Ehcache ARC, the most requested feature in Ehcache history, are designed to eliminate. Ehcache ARC makes your cache or caches controllable using the same metric the JVM's are started with. The same resource you are trying to manage. Bytes! In addition, instead of having to tune each individual cache you can instead make a high level choice at the CacheManager level and let ARC balance the resources for you.

In this change to the sample we have removed the cache entry count and instead allowed the cache to use 50% of the JVM heap (look at line 4 where it says maxBytesOnHeap). This setting tells the cache manager to allow for up to 50 percent of the heap for caching (Can also be configured using a fixed number of bytes instead of percentage). It will balance this heap across what ever caches are defined with this manager. This allow's an application to manage the available resources without waste and without GC pauses/OOMEs.

I updated and rebuilt my spring-annotations to use 2.5.0 and then rebuilt the sample using my newly created version of Ehcache Spring Annotations. You can grab the updated Jar here though I suspect the Spring Annotations guys will update the official build soon.

By using this approach as you add more and more @Cachable annotations to your application you don't have to change anything about the caches you create in ehcache.xml. They make use of the available cache resources as provided.

Wrapping Up


It has always been pretty easy to integrate a cache into an application. Things like Spring Annotations move the bar on that simplicity even further. The challenge was to make those caches effective. How does one get the most performance out of the resources available. Ehcache ARC was created to solve that problem for System Administrators, OEM's Developers and the entire OSS community. We are looking forward to people getting started with it and hope to receive lots of feedback.

What Is Terracotta?

One of the biggest challenges in software is telling people what your software is in a way that helps them make decisions about it. This challenge can often be as difficult as designing and building the software itself. The Terracotta team recently hooked up with the gang from Epipheo studios to create a 2 minute video to do just that. We spent a lot of time thinking about and refining things in order to get a clear succinct message. I think it came out quite well.

Check it out:

Easy Java Performance Tuning: Manage Objects By Size On The Java Runtime Heap

The Problem?

For those of you who use caching with frameworks like Hibernate, Spring or anything else, you know what a pain performance tuning can be. The tools you have available for tuning, resource management and avoiding OOME's boil down to counts and age controls. These are actually not resource management controls at all. They are data freshness controls and should be treated as such.

How do you even begin to figure out how many entries to allow in each Hibernate cache when you have a hundred of them? What if things change and objects get bigger, or smaller. What if you change your heap size and/or usage patterns?

You've Gotta Try This...

With Ehcache 2.5 you can just specify a percentage of heap or a heap size in bytes that your graphs of Java objects are allowed to use. This is accomplished by passing a simple size description into a Cache or CacheManager. All objects held onto by the cache will borrow their space from the cache's or manager's specified pool. Entries will get evicted from the cache as space runs low without any intervention from the developer. This can be done at the Cache Tier level whether it's on heap, disk or BigMemory. It's another way to reduce tuning, improve performance while avoiding OOME's. Learn more here.

If you do things in config or in code it's just a one line change:

A few More Details...

It works on any 1.5 or 1.6 JVM (Tested on Oracle, JRocket and IBM). Doesn't require any object serialization for the on heap management.

Learn More:

http://ehcache.org/documentation/configuration.html#Memory_Based_Cache_Sizing_Ehcache_2.5_and_higher

http://www.terracotta.org/

http://dsoguy.blogspot.com/2011/04/biggest-problems-in-local-caching.html

Try it out and give us lots of feedback on the Terracotta Forums:

http://forums.terracotta.org

Exciting Times... Terracotta and Software A.G.

Exciting times over here at Terracotta.

Read all about it:

http://www.softwareag.com/corporate/company/terracotta.asp

http://terracotta.org/terracotta-joins-software-ag

http://blog.terracottatech.com/2011/05/terracotta_joins_software_ag.html

This is great news for Ehcache, Quartz, and Terracotta customers and communities!

Local Caching++

Ok, so you've built your application in Java. You've used all the usual tools. Tomcat, Spring, Ehcache, Quartz, etc. Or maybe you went the JRuby, Grails or Scala route. You test your new application or hand it off to run in production and it's too slow. This is just a single node application at this point. It services 20-100 users. It's churning and burning the database, creating and recreating the same Web Pages, Users and other relevant data. You want to start caching locally to solve your latency and throughput problems. Then, upon a more detailed look at your application, you get scared.

You find your application has:

• 40 DB tables in Hibernate that can be cached
• A web cache
• A user session cache

Then it hits you. Caching is easy but cache tuning is hard.

What Makes Cache Tuning Hard?

In conversations with 100's of cache users there are actually a small handful of difficult to work through challenges applying caching to an application:

• Hibernate/Lots of caches - When using Hibernate you often end up with as many as 100 tables in your DB. How do you balance a fixed amount of resources(Heap/BigMemory) across 100 caches?
• Indirect knobs/Bytes vs Count/TTL - In local Java caching the control points are almost always measured in number of entries and time to live. But wait a minute! When I start the JVM I don't say how many objects the heap can hold and for how long. I say how many bytes of memory the heap can use?
• Who Tunes and When? - At some companies the desire is to have the "Application Administrator" do the tuning. At others it's the "Developer." They have different understandings of the application. The developer can tune by knowledge of the application. The app admin can only tune based on what's happening when the application is running.

These are the challenges we are working to solve in the next version of Ehcache. While it's early days on the dev side we would love feedback on our approaches. You can get a sense of how it's going to work from the doc on ehcache.org.

What We Are Building

Greg, me and the dev team spent a bunch of time pondering the above problems over the last few years. We felt that with two key improvements to how people tune we could address most of the above (and a few more items hit the rest):

• Tune from the top - Define max resource usage for the whole cache manager and then optionally define it for the individual caches underneath it as needed. So if you have a hundred caches you can start with, "Give these 100 caches N amounts of Heap/OffHeap." Then monitor and see if any specific caches need special attention.
• Tune the constrained resource, Bytes - TTL is a cache freshness concern not a resource management concern. Max entry count does not directly map to available heap resources. So we are adding "bytes" based tuning. This eliminates the mistake prone process of trying to control resources by TTL/TTI/count and hope you get it right. Instead you say, "I want to allow caching to use 30 percent of heap." We take it from there.

Wrapping Up

With those two key improvements a developer or admin is now directly turning the knob (Size of cache in bytes) that maps to the resources available in the JVM and doing it at a global level or a local level as needed to avoid hard to tune individual cache constraints.

This will work with all of our JVM level cache tiers (onHeap, BigMemory, Disk).

When you put those features together with other items coming in the next major release like entry and cache pinning and a snapshotting bootstrapper for warming we feel like this will be a very powerful release.

Please check out the new docs and give us feedback by commenting on this blog or posting the the Ehcache forums.

Help us make Ehcache as easy to use and powerful as we possibly can.

http://ehcache.org/documentation/configuration.html#Memory_Based_Cache_Sizing_Ehcache_2.5_and_higher

"What" "When" and "Where" ... Quartz Scheduler 2.0 Goes GA

What is Quartz?

Quartz is The Lightweight, Open Source, Enterprise-class Job Scheduler for Java. For years Quartz has put the "When" in Java applications via it's full featured scheduling capabilities. As a result, from Spring to JBoss, Quartz is embedded in just about every major Java product. Quartz is extremely robust and full featured providing things like HA, Transaction Support and all the precise guarantees one needs to assure reliable Job execution.

Terracotta has done a number of incremental improvements since taking over stewardship of Quartz. We've been evolving Quartz one step at a time while collecting what features the Quartz community were really interested in.

Quartz 2.0 is the realization of those user requested features!

"What" "When" and "Where"

Let's step back a bit and review where Quartz fits in the Terracotta world view. As most people know Terracotta is focused on adding Snap-in performance to JVM based applications through Scale-out (Terracotta Server Array), Scale-up (BigMemory) and Speed-up (Ehcache). We do this while maintaining our goals of simplicity, predictability, and density (can your datastore hold 2 billion entries per node in-memory?) throughout our product line. We view this data layer scaling layer as solving the "What" or the data problem of an application. But when going from 1 to many nodes that's just the first of two major scale points.

Up until now, Quartz Scheduler has been solely focused on the "When" part of code execution. Making sure "code execution" happens exactly "When" it's supposed. It has been pretty much ubiquitous in that space with literally 100s of thousands of users. While it Scaled-out with JDBC and Terracotta, it gave precious little control over where jobs execute.

With Quartz Scheduler 2.0 we've now added that "Where." This is about where code executes. Instead of randomly choosing a node for job execution in a scaled-out architecture you can now create constraints that help make the decision based on "Where" it should be executed. You can do this based on:

• Resource Availability - CPU available, Memory Available, custom constraints
• Ehcache's data locality - Bring the work to where the data is
• Static allocation - Just decide where it goes

This gives Terracotta the ability to Snap-in and handle both the major scale-out points.

What's new in Quartz 2.0

The simple answer is A LOT!

• Easy to use Fluent API - Quartz 2.0 has a new, easy to use fluent interface that hides the complexity of building out the description of your jobs behind a simple description of what you want to happen and when. I wrote a short blog about this when it was in beta.
• Quartz "Where" - Constraint based system for controlling where jobs execute based on things like CPU and Memory usage, OS, and Ehcache data locality
• Quartz Manager - A flash based GUI console for managing and monitoring your scheduler in production.
• Batching - Helps improve a schedulers throughput by allowing one to make trade-offs between perfect time execution and benefiting from batching.
• Ton's of bug fixes and features - Lots of long requested features. Check out the link for the list.

Quartz 2.0 Is Now GA

Quartz 2.0 makes a big leap in usability, power, visibility and management and it's NOW GA. We did this while maintaining Quartz's reliability and predictability. We focused on making sure existing users would have an easy time moving forward so check out the migration guide if you need help. It really is the next generation of Quartz. Give it a try and give us feedback as we are constantly working to make things better!

More Reading

• http://www.codespot.net/blog/2010/12/quartz-where/
• Quartz Scheduler "Where" Screencast
• Quartz Manager Screencast
• What's new in Quartz 2.0
• Quartz Scheduler

Quick 5:41 Intro To Ehcache Search (Now GA)

Ehcache 2.4 went GA today. Still lightweight (under 1MB) and still backward compatible to 1.x so no reason not to give it a try. Below are the highlights and a short video on getting started with Ehcache Search:

• Search - Brand new search API. Allows one to get beyond the key based lookup of objects (Check out this sample)
• Local Transactions - Fast optimistic concurrency without the need for a TransactionManager (Check out this sample)
• Bigger BigMemory (ee) - 2 Billion entries, 1.3 million TPS, Extreme predictability for meeting SLA's
• Bigger Disk Store (ee) - Swap your Ehcache to disk. Grow to hundreds of gigs with no on heap footprint

In the past I've enjoyed short videos that teach me something. This is my first try at doing one to benefit others. I've created this short 5 minute 41 second video to get people started on using Ehcache Search...

Here's the Slides...

Ehcache At 2 Billion...

What's Up With Ehcache 2.4

Ehcache is the de facto caching standard for Java that everyone uses (500,000+ production deployments; the majority of enterprise Java applications). Ehcache 2.4 is coming out soon and includes some capabilities that will make it even easier to use, more powerful, while still maintaining it's light weight.

The highlights include:

• Search - Quickly find entries based on the criteria of your choosing. String matching, dates, ranges, sums, averages etc.
• Fast local transactions - Improved performance of JTA and added a new non-jta transaction api for user level control
• Even more capacity and performance

What I've been Testing

I've written before about BigMemory for Enterprise Ehcache and how it solves the problem of long, unpredictable GC pauses in Java. The first release of BigMemory was… well, big. In Enterprise Ehcache 2.4, BigMemory has gotten even bigger.

Using the Enterprise Ehcache Big Memory Pounder I was able to show that Enterprise Ehcache 2.4 now easily handles:

• Entry Count: > 2 billion entries (I reached 2 billion on the hardware I had; with bigger hardware, I could probably have gone much higher).
• Throughput: 1.3 million operations per second (symmetric read and write; CPU bound)
• SLA/Predictability: No GC pauses and a predictable 38-42 ops/thread/millisecond throughout the test
• Data Size: 1-350 GB in-memory cache (again, I was limited by the hardware I had; with more RAM, I could probably have gone much higher)
• Flexible Efficient Entry Sizes: The cache can now dynamically handle very large (10-100 MB) and very small entries (just a few bytes) together more efficiently with no tuning (This test used small entries in order to fit as many entries as possible into the memory I had. I also ran tests with fewer entries in order to validate wide ranging sizes)
• Tuning: All tests were done with NO TUNING. Right out of the box.

Here's the hardware and software stack I used for my testing:

Cisco UCS C250 Server

Dual Intel x5670 2.93 Ghz CPU

384 GB RAM ( 8 GB x 48)

Redhat 5.4 Enterprise Edition

Sun JDK 1.6_22

For this test, all of the data was in memory.

A Bit About Ehcache BigMemory

BigMemory is 100% pure Java and in process with a Java application. No magic or special JVMs (works on IBM and JRocket as well). The cache data is safely hidden away from Java GC and the pauses that occur with large heaps by instead storing data in a BigMemory off-heap store.

Embedding

BigMemory got it's start as a component in the Terracotta Server Array and as a result it is particularly useful for embedding. It's performance characteristics and no tuning approach improves "The Out Of The Box Experience" and saves money on support by removing tuning required by users and problems caused by GC pauses.

You may be thinking...

"I don't have 2 billion entries in my caches?"

That's ok. Ehcache is a lightweight core library (under 1MB) for caching that's ubiquitous and easy to use. When it's needed, Ehcache lets you scale up and out to billions of entries and terabytes of data. It does so at a manageable server density without changing code/architecture and without a bunch of tuning and learning. This protects not only your knowledge investment but your code investment.

More about BigMemory for Enterprise Ehcache:

http://terracotta.org/bigmemory

BigMemory Whitepaper

BigMemory Documentation

More about the 2010 Ehcache user survey:

Ehcache User Survey Whitepaper

Ehcache To The Rescue (Comic Strip)

Working on a product that has as much adoption as Ehcache is extremely rewarding. Every day millions of people's applications run faster with this tiny little library. It's one of those, "Of course I use it, doesn't everyone " kinds of products and what Greg Luck has accomplished with it is really amazing. One of the challenges we face as we improve and extend it's functionality is getting the story out to those million or so users. Yes, it's still an extremely fast extremely light weight library that pretty much everyone just uses. But it also future proofs your app with snap-in scale-up, scale-out and HA each with just 2 lines of configuration.

In the coming release we are adding Search, Local Transactions and an even better BigMemory (We've been regularly testing with 2 billion entries in memory with linear, predictable, fast, performance but that's another blog) and we want people to know about this stuff, try it, and give feedback so we can constantly make it better.

Anyway, after watching an inspiring talk from Edward Tufte on visual representation I've been experimenting with ideas on how to teach people about Ehcache in quick, fun ways. In the process I wrote my very first comic strip. Check it out...

Ehcache To The Rescue Comic Strip...

Quartz Scheduler 2.0 Beta 1 Welcomes New Fluent API and "Where"

Quartz Scheduler, the most widely used Java scheduler is getting some major improvements for 2.0. I'm going to talk a little about what to expect but you can also read the release notes here.

Goals

We had two major goals when planning for Quartz 2.0 began.

• Simplify/modernize the Quartz API.
• Improve the Quartz experience when leveraging a cluster

Quart 2.0 API

In order to improve the usability and readability of Quartz 2.0 over passed versions we spent a lot of time evaluating the parts and how a user interacts with them. We found that too much knowledge of those parts needed to be understood at construction time. As a result we moved to a "Fluent API/DSL" approach. The best way to get a feel for the kind of improvement this gives is to compare the samples from 1.8.4 to the same ones translated in 2.0 Beta 1.

In the simple of case of example 1 from the Quartz Kit you can see the basic philosophy change:

Improvements include:

• The date/time related methods have been moved off of the Trigger and Job classes into a Date building class called "DateBuilder"
• We've removed the need to know details about which Job and Trigger classes you need and instead infer them through the building methods you call.
• The construction now reads more like a sentence. new job withIdentity "job1", "group1". new trigger withIdentity "trigger1", "group1" start at runTime

This is pretty subtle in a simple case like example 1 but gets more obvious as the cases get more complex. Let's look at example 2.

In this case notice how the constructors are growing with no real indication to what each parameter means.

Now lets look at an example where you have to choose a specific Trigger type.

This example shows how in Quartz 1.8.4 you would have to know to select a different Trigger type but in 2.0 it's just abstracted away in the scheduling.

It's worth going through the samples yourself and making any suggestions you may have. Still plenty of time to get your API suggestions in via the Quartz Forum!

Quartz "Where" And Other Improvements

Clustered Quartz has gotten two major improvements in 2.0. Improved performance as node count increases and Quartz "Where"

I wrote a blog about this a while back but now you can play with it. Alex Snaps has written an excellent blog with code samples on the topic. Check it out to really dig in.

Ehcache 2.4 Beta 1 Welcomes Search, Local Transactions and more...

Ehcache 2.4 Beta has been released. We've been spending the last few years working hard to make Ehcache the best possible solution for users with caching needs. 2.4 beta 1 is another big step forward. The beta includes some significant new features and improvements that can be read about here. This blog is a few notes on getting started with those features and a few comments on why they are important.

Ehcache Search

This is one of the most requested features for Ehcache. I can't count how many times that I have had a map or cache and just wanted to quickly and easily search for an entry. Sure, one can write an iterator and use Java's matchers but that's

• A bit of annoying coding
• Only practical for unclustered caches

Ehcache 2.4 Beta 1 has a new search interface in core Ehcache that works throughout our scale continuum. We used the "Fluent Interface/DSL" style that we've been moving towards over the last couple of years in both Ehcache and Quartz Scheduler. This style of interface makes an API easy to use and read like english.

To learn more about it check out this self contained sample app and the documenation. Give us feedback early and often on the Ehcache Forums as the API is still under development and can still be improved.

Local Transactions

Back in Ehcache 2.0 we added JTA support. This was helpful but we found that for some use-cases people wanted a few things.

• Transactions without a JTA transaction manager
• More speed

We improved the speed across the board of our XA compliant JTA stuff and we also added the concept of local transactions. Local transactions provide optimistic concurrency and atomicity across caches. They are like XA transactions but aren't completely tolerant of external resource crashes outside of Ehcache (i.e. cache + databases don't have all the guarantees of XA in the node failure cases. Does provide the guarantees across Ehcaches), don't require a transaction coordinator (though can be used with them) and tend to be less resource intensive (read faster).

I've whipped up a little self contained sample here.

Improved Enterprise Disk Store

In 2.3 we added an Enterprise version of unclustered Ehcache core. The version added support for very large caches through the seamless BigMemory add-on (1 line of config). We also, somewhat more quietly, added the beginnings of a new Enterprise disk store that no longer stored keys in memory and can handle larger restartable caches.

2.4 beta 1 takes this to another level. The disk store is now way more efficient, better leverages SSD drives and can efficiently grow to very large sizes without any additional heap usage or fragmenting. One can exercise the improvements using the Ehcache Pounder.

Other Improvements

While not everything for the final release was done in time for Beta 1 some significant improvements are in.

• NonStopCache now built in. Rather than have to add a jar and configure a wrapper to get the non-stop characteristics in clustered land this is now built into the product core and be turned on via configuration
• Search now works clustered - The new search API is backed by the Terracotta tier. This is still early and we have a lot of performance and HA work to do here. That said, it is testable and usable so give it a try.
• Explicit locking module is now in the core kit
• Rejoin now works in non-stop (You can disconnect from a cluster and reconnect to that cluster without restarting)

For those interested in scheduling we have also released Quartz Scheduler 2.0 beta 1. More stuff will be coming in a month or so when we do beta 2 for all products.

Direct Buffer Access Is Slow, Really?

Lots of claims in the blog sphere around direct memory buffers being slow. We have been working with them a lot and hadn't seen that slowness but I'm more of a try it and see kinda guy so I did just that.

I cooked up a quick single threaded test to compare offheap direct memory buffers to on heap byte buffers. What I found is that at least on my notebook direct is between 2-5 percent of onheap. Even on my notebook I was writing and reading 2G/second in small chunks to random parts of the byte buffer (Every operation on the test does one write and one read).

The nice thing about them is they occupy no heap so the data stored in them is hidden away from the JVM GC. Of course whether that 2 to 5 percent matters depends on how much data your app is trying to crank through them so as always you need to look at your use-case's, latency, through-put and SLA goals and code accordingly.

While more testing can and should(and has) be done, here is a place for people to start.

Here's the code and my results from my 1.6 ghz notebook. I only spent a few minutes on it so suggestions to improve it are welcome:

Type: ONHEAP Took: 8978 to write and read: 10737418368

Type: DIRECT Took: 9223 to write and read: 10737418368

Type: ONHEAP Took: 8827 to write and read: 10737418368

Type: DIRECT Took: 9283 to write and read: 10737418368

Type: ONHEAP Took: 8813 to write and read: 10737418368

Type: DIRECT Took: 9604 to write and read: 10737418368

https://gist.github.com/700690

A Couple Minutes With Ehcache Search...

Welcome To Ehcache Search

Now that we are releasing our, GC Busting, Ehache BigMemory product we are starting the process of gathering feedback on the new Ehcache Search API. While Ehcache Big Memory solves your Latency, through-put and SLA issues, Ehcache Search gives you the ability to handle a users data retrieval needs in a simple and efficient way.

What is it?

With todays data explosion and hardware memory size explosion, in-memory and caching have become an important part of everyone's software. Now, getting at the data in that cache in a flexible, simple and fast way has really become an important area of focus. Things like finding entries that match criteria, performing aggregations and calculations (eg average, sum, max, min) need to be easy and efficient.

That's where the new Ehcache Search API comes in. It will provide fast efficient searches and aggregations across the Ehcache Snap in, Speed up, Scale out continuum (Single node performance through scaled out architecture).

Please try out the API (It's checked into Ehache core's trunk) by grabbing the this self contained, buildable sample and open source Ehcache nightly jar I've put up on GitHub.

https://github.com/sharrissf/Ehcache-Search-Sample/downloads

It's simple to use and 100 percent open source. We are very excited about feedback and plenty of time still exists for making changes and improvements. Please ask lots of questions and give lots of feedback.

To try it you can just unpack the kit:

tar -xzvf ehcache-search-sample-0.0.1-SNAPSHOT-distribution.tar.gz

and run it:

sh run.sh

The source code is included in the kit and can be edited in this directory:

src/main/java/org/sharrissf/sample/

rebuilt if you have Maven.

mvn clean install

And rerun using the run.sh command.

You can even package it up yourself by using

mvn clean assembly:assembly

Looking forward to hearing from the community. It will be interesting to see how projects like Grails, Hibernate, Liferay and Cold Fusion leverage cache search. Those uses will help set direction for years to come.

A Couple Minutes With Ehcache BigMemory Pounder...

Introduction

UPDATE: This product was released (No longer in beta)

Ehcache with BigMemory is now out in beta. One of the challenges when confronted with a new technology is exercising and understanding it's characteristics. Sure, you can read the docs, Google for some blogs, integrate it with your application and maybe write some samples, but that's a lot of work. Plus those approaches may not give you a clear picture of the various characteristics of the software. In order to make things a bit easier I've released a configurable pounder application for Standalone Ehcache with BigMemory.

Getting Installed

Here are the steps to get started:

• Get the Ehcache with BigMemory Beta and a license key to use it.
• Get the Standalone Ehcache Pounder distribution
• Unpack the Ehcache with BigMemory distribution

tar -xzvf ehcache-core-ee-2.3-distribution.tar.gz

• Copy the Standalone Ehcache Pounder kit into the ehcache kit and unpack it

cd ehcache-core-ee-2.3

cp ../ehcache-pounder-0.0.5-SNAPSHOT-distribution.tar.gz .

tar -xzvf ehcache-pounder-0.0.5-SNAPSHOT-distribution.tar.gz

• Copy your license file and your ehcache core jar into the pounder kit

cd ehcache-pounder-0.0.5-SNAPSHOT

cp ../lib/ehcache-core-ee-2.3.jar .

and copy terracotta-license.key to the ehcache-pounder-0.0.5-SNAPSHOT as well.

Running the Pounder

Now you are ready to go. First take a look at the start script in one of the template directories (i.e. templates/1G-BigMemory):

sh run-pounder.sh

Out of the box it looks like this:

java -verbose:gc -Xms200m -Xmx200m -XX:+UseCompressedOops -XX:MaxDirectMemorySize=64G -cp ".:./jyaml-1.3.jar:./ehcache-pounder-0.0.5-SNAPSHOT.jar:./ehcache-core-ee-2.3.jar:slf4j-api-1.5.11.jar:slf4j-jdk14-1.5.11.jar" org.sharrissf.ehcache.tools.EhcachePounder

This should work fine for most people. It uses a small heap of 200 meg (you may need to grow this for really big caches). It is fine to have a MaxDirectMemorySize that is larger than your memory size on your machine. Just don't have a maxOffHeap size in your config.yml that is greater than your available physical memory on your machine. Also, make sure you leave room for your OS and the JVM. For example. If you have 8G of physical memory you might do a 400m java heap, a 6G offheap and leave the rest for the OS to use.

The script defaults to verbose gc because it's useful to see those stats and compressed oops BECAUSE YOU SHOULD BE USING A 64 BIT JVM and this setting makes it much more efficient (closer to 32 bit object pointers when possible using much less heap).

I tend to run this script like this:

./run-pounder.sh | tee out.txt

That way all output of the test goes to both the screen and to the file out.txt

Configuring the Pounder

You configure the pounder using the config.yml file. You can learn about all the options in the README. Here is the sample config included in the kit:

storeType: OFFHEAP

threadCount: 33

entryCount: 1000000

offHeapSize: "1G"

maxOnHeapCount: 5000

batchCount: 50000

maxValueSize: 800

minValueSize: 200

hotSetPercentage: 99

rounds: 40

updatePercentage: 10

diskStorePath: /export1/dev

NOTE: The most important setting here is the offHeapSize. If you set this to a number greater than the amount of memory you have on your machine you will not be happy.

The offHeapSize + the heap size + the amount of memory your OS needs together must be less than the amount of physical available memory on the machine.

Output

When running the output will look like the following:

size:492120 time: 7434 Max batch time millis: warmup value size:796 READ: 0 WRITE: 15151 Hotset: 99

size = total size of cache

time = total time to execute batch

max batch time = either warm up for the load phase or the longest time it took to execute a batch (note that in a multi-threaded test on a cpu bound machine the batch times are going to be impacted.)

value size = size of the value in this batch

READ = number of reads in the batch

WRITE = number of writes in the batch

hotset = percentage of the time that reads are done from the on heap cache

At the end of each round you'll see something like this:

Took: 10899 final size was 995240 TPS: 91751 MAX GET TIME: 20

Took = total time the round took

final size = the total size of the cache at the end (this can be impacted by eviction and be less than what you loaded)

TPS = total TPS during the run

MAX GET TIME = The maximum amount of time it took to get an entry

An interesting design choice of this pounder is that the threads you define do BOTH reading and writing. So the writers can starve out the readers in a hotset test. I could have gone he other way and then the readers could starve out the writers giving overly generous TPS.

What's Next

The biggest gap right now is that I didn't implement a way to specify a rate. It always goes full throttle. I'll try to add that when I get some time if people think it's useful.

Please give me lots of feedback so we can improve both the pounder and Ehcache

Learn more at:

The source for the pounder

http://terracotta.org/bigmemory

My other blog on BigMemory
Check the Ehcache BigMemory docs

Server Array BigMemory FAQ

A Little Bit About BigMemory for Ehcache and Terracotta ...

Big Memory?

In talking to our users it is clear that applications are getting more and more data hungry. According to IDC, data requirements are growing at an annual rate of 60 percent. This trend is driven further by cloud computing platforms, company consolidation and huge application platforms like Facebook. There is good news though. Server class machines purchased this year have a minimum of 8 Gig of RAM and likely have 32 Gig. Cisco is now selling mainstream UCS boxes with over 380 Gig of RAM (which I have tried and is amazing). On EC2 you can borrow 68.4 Gig machines for 2 dollars an hour (I have also tried this and it is also pretty amazing). Memory has gotten big and extremely cheap compared to things like developer time and user satisfaction.

Unfortunately a problem exists as well. For Java/JVM applications it is becoming an ever increasing challenge to use all that data and memory. At the same time that the data / memory explosion is occurring the amount of heap a Java process can effectively use has stayed largely unchanged. This is due to the ever increasing Garbage Collection pauses that occur as a Java heap gets large. We see this issue at our customers but we also see here at Terracotta tuning our products and the products we use like third party app servers, bug tracking systems CMS's and the like. How many times have you heard "run lots of JVM's" or "don't grow the heap" from your vendor's and/or devs?

So we set out to first identify the problem as it exists today, both in the wild and in-house. We then created a solution, first for us (an internal customer) and then for all of the millions of nodes of Ehcache out there (all of you)

3 Big Problems Seen by Java Applications

My Application is too slow

My application can't keep up with my users. I've got 10's of gigs of data in my database but it's over loaded and or too slow to service my needs. Either due to the complicated nature of my queriers or the volume of those queries. I want my data closer to the application so of course I start caching. Caching helps, but I want to cache more. My machine has 16 gigs of RAM but if I grow my heap that large, I get too many Java GC pauses.

My Application's latencies aren't predictable

On average my Java application is plenty fast but I see pauses that are unacceptable to my users. I can't meet my SLA's due to the size of my heap combined with Java GC pauses.

My software/deployment is too complicated

I've solved the Java GC problem. I run with many JVM's with heap sizes of 1-2 gigs. I partition my data and or loadbalance to get the performance and availability I need but my setup is complicated to manage because I need so many JVM's and I need to make sure the right data is in the right places. I fill up all 64 Gig's of RAM on my machine but it's too hard and fragile.


The other problem

Like many vendors, in the past we told our users to keep the heaps down under 6 gig. This forced our customers to not completely leverage the memory and or cpu on the machines they purchased and or stack JVM's on a machine. The prior is expensive and inefficient and the latter fragile and complex.


Here is a quick picture of what people do with their Java Applications today:

Base Case - Small heap JVM on a big machine because GC pauses are a problem
Big heap - That has long GC's that are complicated to manage
Stacked small JVM heaps - This in combination with various sharding, load balancing and clustering techniques is often used. This is complicated to manage and if all the nodes GC at the same time this can lead to availability problems.



What kind of solution would help?

Here's what we believe are the requirements for a stand-alone caching solution that attacks the above problems.

1. Hold a large dataset in memory without impacting GC (10s-100s of Gig) - The more data that is cached the less you have to go to your external data source and or disk the faster the app goes
2. Be Fast - needs to meet the SLA
3. Stay Fast - Don't fragment, don't slowdown as the data is changed over time
4. Concurrent - Scales with cpu and threads. No lock contention
5. Predictable - can't have pauses if I want to make my SLA
6. Needs to be 100 percent Java, work on your JVM on your OS
7. Restartable - A big cache like this needs to be restartable because it takes too long to build
8. Should just Snap-in and work - not a lot of complexity

What have we built?

First we built a core piece of technology, BigMemory, an off-heap, direct memory buffer store, with a highly optimized memory manager that meets and or exceeds requirements 1-6 above. This piece of technology is currently being applied in two ways:

1) Terracotta Server Array - We sold it to our built-in customer, the Terracotta Server Team, who can now create individual nodes of our L2 caches that can hold a hundred million entries, leverage 10's of gigs of memory, pause free and with linear TPS. This leverages entire machines (even big ones) with a single JVM for higher availability, a simpler deployment model, 8x improved density and rock steady latencies.

2) Ehcache - We've added BigMemory and a new disk store to Enterprise Ehcache to create a new tiered store adding in requirements 7-8 from above (snap-in simplicity and restart-ability). The Ehcache world at large can benefit from this store just as much as the Terracotta products do.

Check out the diagram below.




Typically, using either of the BigMemory backed products, you shrink your heap and grow your cache. By doing so SLA's are easier to meet because GC pauses pretty much go away and you are able to keep a huge chunk of data in memory.


Summing up

Memory is cheap and growing. Data is important and growing just as fast. Java's GC pauses are preventing applications from keeping up with your hardware. So do what every other layer of your software and hardware stack does: cache. But in Java, the large heaps needed to hold your cache can hurt performance due to GC pauses. So use a tiered cache with BigMemory that leverages your whole machine and keeps your data as close to where it is needed as possible. That's what Terracotta is doing for it's products. Do so simply, i.e. snap it in to Ehcache and have large caches without the pauses caused by GC. As a result create a simpler architecture with improved performance/density and better SLA's.


Learn more at http://terracotta.org/bigmemory
Check the Ehcache BigMemory docs

Server Array BigMemory FAQ