Caching for Java Applications

Introduction

A caching API may help to increase application performance and scalability when developing Java applications. This article explores ways for improving performance, concurrence and scalability in Java applications through caching.

Notion of Cache

A cache is an area of local memory that holds a copy of frequently accessed data that is otherwise expensive to get or compute. Examples of such data include a result of a query to a database, a disk file or a report. A typical interface for a Java cache provides access to the data using a unique key:

public interface Cache {  
  Object get(final Object key);  
  Object put(final Object key, final Object value);  
} 

A cache works as follows: An application requests data from cache using a key. If the key is not found, the application retrieves the data from a slow data source and puts it into the cache. The next request for a key is serviced from the cache.

Improving Performance with Caching

Using a caching API may provide significant performance improvement for a Java application, often in orders of large magnitude. The performance improvement comes from serving hard to get data from the local memory of the application.

For example, consider an application that shows a 50-line system status report displayed on a web page after the user logs into the system. For each line it sends a complex SQL query to a database. To execute each query and to fetch results over the network takes 100 milliseconds on average. The total average time to collect all data for the page is about 5 seconds. Getting the same results from a cache takes about 5 microseconds on a modern 2GHz CPU. The performance improvement for this particular use scenario is 1,000,000!

Requirement for Temporal and Spatial Locality

A cache uses a part of the application's memory. That is why the size of the cache has to be small. A special algorithm should be used to remove (evict) data from the cache that has low chances of access. Such algorithm is called an eviction policy.

To benefit from caching, the access to data should display properties of temporal and spatial locality. Temporal locality, also known as locality in time, is a data access pattern such as if a data entry is accessed, it will be accessed again soon. Spatial locality, also known as locality in space, is a data access pattern such as if a data entry is accessed, data entries whose addresses are close are accessed again soon.

The data from the example in the beginning of the article satisfies the requirement of temporal and spatial locality . Users log into the system around the same time and the number of items from the reports that are accessed in rapid succession is small.

Data that does not satisfy the requirement of temporal and spatial locality of access leads to faster eviction of cache elements and as a result will lower the number of cache hits and increase cache maintenance.

Cache Performance Characteristics

The main performance characteristic of a cache is a hit/miss ratio. The hit/miss ratio is calculated as number of cache hits divided by number of cache misses. The hit/miss ratio is calculated using hit and miss counters accumulated over a period of time. A high hit/miss ratio means that a cache performs well. A low hit/miss ratio means that the cache is applied to data that should not be cached. Also, the low hit/miss ratio may mean that a cache is too small to capture temporal locality of data access.

Cache Eviction Policy

A cache eviction policy is an algorithm according to which an existing element is removed from a cache when a new element is added. The eviction policy is applied to ensure that the size of the cache does not exceed a maximum size. Least Recently Used (LRU) is one of the most popular among a number of eviction policies. LRU earned its popularity for being the best in capturing temporal and spatial locality of data access.

A minor disadvantage of LRU is its sensitivity to a full scan. The sensitivity manifests itself in evicting accumulated frequently accessed cache elements when accessing data that does not satisfy the requirement of temporal locality. This disadvantage is minor because LRU recovers from full scans quickly.

A typical implementation of a cache with the LRU eviction policy consists of a map and a linked list. The map stores cached elements. The linked list keeps tracks of the least recently used cache elements. When a cache element is updated, it is removed from the list and added to the top of the list. The new elements are added to the top of the list as well. If the cache grows bigger than its maximum size, an element is removed from the bottom of the list and from the map. This way the least recently used elements are evicted first.

While simple cache in Java can be implemented in a few lines of code, it would be missing important features that a cache needs to have in order to be usable in a real application. Please see the section Caching Products for Java for a detailed discussion.

Putting Object to Cache

Putting an object to the cache is simple:

    
        final Cacheonix cacheManager = Cacheonix.getInstance();
        final Map cache = cacheManager.getCache("invoce.cache");
        
        // Put object to the cache
        final Object key = "key";
        final Object value = "value";
        cache.put(key, value);
        
        // Get object from the cache
        final Object myObject = cache.get(key);

 

Common Cache Use Scenarios

Common cache use scenarios include an application cache, a second level (L2) cache and a hybrid cache.

Application Cache

An application cache is a cache that an application accesses directly. An application benefits from using a cache by keeping most frequently accessed data in memory.

The following communication diagram illustrates using an application cache:

Application Cache

Level-2 Cache

One of the major use scenarios for a cache is a level-2 (L2) cache . An L2 cache provides caching services to an object-relational mapping (ORM) framework or a data mapping (DM) framework such as Hibernate or iBatis respectively. An L2 cache hides the complexity of the caching logic from an application.

An L2 cache improves performance of an ORM or DM framework by reducing unnecessary trips to the database. .

The following communication diagram illustrates using an L2 cache:

Level 2 (L2) Cache


The application does not access cache directly in this use scenario. Instead, the application utilizes a high level interface provided by an ORM or a DM framework. The framework uses cache for caching its internal data structures such as mapped objects and database query results. If the cached data is not available, the framework retrieves it from the database and puts it into the cache.

Data Grid

A data grid is a relaible distrbuted cache that uses an external data source to retrieve data that is not present in the cache and an external data store to write the updates. An application using a data grid benefits from simplified programming of cache access.

This use scenario is different from the application or the second-level cache when an application or a data access framework is responsible for populating the cache in case of cache misses.

The following communication diagram illustrates using a data grid:

In-Memory Data Grid


Caching Anti-Patterns

Caching provides such a great improvement of performance that it is often used without limit. An anti-pattern Cache Them All is characterized by caching all data, without regard to temporal or spatial locality of data access.

Cache Them All degrades application performance instead of improving it. The degradation of performance is caused by the overhead of maintaining a cache without benefiting from reduced cost of access to frequently used data.

To avoid the pitfall of the Cache Them All, only data that is hard to get and shows temporal and spatial locality of access should be cached.

Next

See Also

Share This Article