Caching - Part 2

In the first part, I have discussed the caching implementation in Client Tier. In this article, I will continue to discuss caching on business and database tier.

Caching on business logic tier

Caching on business tier is reliable in term of cache control. Developer can decide when to flush the cache, the maximum size of cache and how should the cache be updated. However, clustering make cache management more difficult on server side. Here are some of the concerns that you need to tackle

1/ Passive Caching or Active Caching

If you choose to implement passive caching, there is no need to frequently update the cache. The cache is build as wrapper around a service, intercept any call to it. If the cache does not contain wanted information, it simply let the service handle the request and memorize the response. If there is a identical method invoke next time, the cached response will be served.

Active caching take initiative of generating the expected response before any method invoke. It also take responsibility of regularly refreshing the cache. It is possible to do so if the data size is small enough.

The biggest advantage of active caching is prevention of slow response on first hit. However, if the calls are not predictable or not repetitive enough, the system may waste resources maintaining the unused cached records.

Passive Caching is also ineffective if the calls are not repetitive but it do not waste effort on caching unused records. It is also more scalable due to the ability of caching a small fraction of frequently used data. However, for passive caching to be effective, the size of cache should be comfortable larger than the amount of concurrent users to avoid purging useful records.

Active Caching is normally implemented manually by developers. Passive Caching can be supported by framework (for example, putting annotation on top of getter method) or done manually. The benefit of manual caching come from ability to manually refresh the cache if there is data change (for example, invalidate all the cache of getter methods if there is a call to update data). However, this is not popular any more due to the fact that we can not reliably detect data change in cluster environment (thinking of what happen if update request come to other server).

2/ How to maintain cache synchronization on clustering environment

Caching is not so challenging in the past, when clustering is not the norm. Distributed caching add more complexity into implementation as developers need to maintain the synchronization of data. Apparently, caching add some delay when displaying data. If the cached resources are not synced among servers, users may see different values of data within one session, which is awkward.

Here are some of methods that can be used to rectify the issue:

• Using stickiness session. If we can guaranteed that all requests from user will be forward to the same server, there is no need to synchronized data among servers. This is not preferred as non-stickiness session is better for scalability and silent fail-over.
• Using the distributed cache like MemCached or Terracotta. Distributed cache is slower because the records is download through internal network but it still helps on boosting scalability as there is no single point of data access. There are some major different between MemCached and TerraCotta on the link below. Generally, TerraCotta is preferred when you need to manipulate data. It also help to synchronized records between nodes, which is more scalable than MemCached. 

http://debasishg.blogspot.sg/2008/09/memcached-and-terracotta-alternatives.html

•  For Active Caching, we can sync the effort to refresh caches of all the servers. The easiest way is to have a central server pushing cached records to all nodes. With this design, there is no duplicated reading of data and all the nodes will serve identical records.

3/ Memory

Memory is the next thing you need to worry about caching. Generally, caching will not cause memory leak because the oldest records will be flushed to slower storage like hard disk or totally discarded. However, you may not want that to happen because it decrease the performance and efficiency of caching. Caching is optimized when the records are often purged due to expiration rather than memory constraint.

Cache Eviction is unwanted but when it happen, there are 3 popular choices of how to choose record to purge. They are Least Recently Used, Least Frequently Used and First In First Out. If you want to know further, can take a look at EhCache documentation for Cache Conviction at

http://ehcache.org/documentation/apis/cache-eviction-algorithms

If you are interested to build your own Conviction Algorithm, can refer to Wikipedia for some well-known techniques

http://en.wikipedia.org/wiki/Cache_algorithms

In the past, heap space is used to store records but recently, EhCache introduce off-heap store. Off-heap store reside on RAM but out of JVM. Off-heap store is slower than on-heap store as EhCache need to serialize object before writing it to the store and de-serialize it when retrieving back the object. However, as it reside outside heap space, it is invulnerable to intermittent performance issue caused by garbage collection.

Data Structure also plays a big role in caching performance. For a performance sensitive application like caching, it is highly recommended to implement your own data structure instead of using built-in implementation provided by Java. Your data structure need to reflect the business requirement of how your data will be queried.

• If the data queries are random, you may be interested in Random Sampling Technique. For example, can take a look at class SelectableConcurentHashMap from EhCache. It is a good example of organizing data so that random record retrieval is faster. 

• If the related or similar data queries come together, you can introduce insertion order to the data inside hash map so that access to least recently accessed data is faster. Detail explanation of this technique is out of scope for this article but audiences can take a look at Java LinkedHashMap to see a sample.

4/ Business Layer

Business Logic tier is rarely flat. In a simplest application, you still have Service on top of DAO objects. In more complex scenarios, the system may include complex services that make use of several other services and DAOs. On the top, the MVC controller is in charge of invoking service and rendering response.

The above diagram is a realistic sample of how your application may be built. There are many place to implement caching but as we already mentioned in the earlier article, too many layers of caching is ineffective.

As usual, the decision come from business requirement rather than developer preference. Caching at higher layer provide more performance gain but also increase the amount of possible method invoke. Let make a simplest assumption of each DAO only have 3 possible values of data, then caching at top layer require 27 records while caching at lowest layer only require 9 records.

If some how, due to business query, the queries for DAOs are always related, we may have much less than 27 possible records to cache. In this case, caching at higher layer make more sense.