Java high-performance cache library Caffeine

/ jason | 张小凯

Original link: https://jasonkayzk.github.io/2023/03/28/Java%E9%AB%98%E6%80%A7%E8%83%BD%E7%BC%93%E5%AD%98 %E5%BA%93Caffeine/

Caffeine is a high-performance local cache library based on Java, improved by Guava;

This article introduces how to use Caffeine cache in Java and how to integrate Caffeine cache in SpringBoot;

source code:

Java high-performance cache library Caffeine

Introduction to Caffeine

Caffeine is a high-performance local cache library for Java. Its official description points out that its cache hit rate is close to the optimal value.

In fact, a local cache like Caffeine is very similar to ConcurrentMap: it supports concurrency and supports data access with O(1) time complexity. The main difference between the two is:

  • ConcurrentMap will store all stored data until you explicitly remove it;
  • Caffeine will automatically remove “infrequently used” data through the given configuration to keep the memory occupied reasonably.

So a better way to understand it is:

Cache is a Map with storage and removal strategies

Caffeine provides the following functions:

 - automatic loading of entries into the cache, optionally asynchronously# 自动加载条目到缓存中，支持异步加载- size-based eviction when a maximum is exceeded based on frequency and recency# 根据频率和最近访问情况，支持将缓存数量设为移除策略- time-based expiration of entries, measured since last access or last write# 根据最近访问和修改时间，支持将时间设为移除策略- asynchronously refresh when the first stale request for an entry occurs# 过期条目再次访问时异步加载- keys automatically wrapped in weak references# key自动包装为弱引用- values automatically wrapped in weak or soft references# value自动包装为弱引用/软引用- notification of evicted (or otherwise removed) entries# 条目移除通知- writes propagated to an external resource# 对外部资源的写入- accumulation of cache access statistics# 累计缓存使用统计

Basic use of Caffeine

Add dependencies to the project:

 <dependency> <groupId>com.github.ben-manes.caffeine</groupId> <artifactId>caffeine</artifactId> <version>2.9.3</version></dependency>

This article is based on version 2.9.3;

cache type

Caffeine provides four types of Cache, corresponding to four loading strategies:

  • Cache;
  • LoadingCache;
  • AsyncCache;
  • AsyncLoadingCache;

Let’s look at it separately;

Cache

The most common type of cache, without specifying the loading method, needs to manually call put() to load;

It should be noted that: put() method will overwrite the existing key, which is consistent with the performance of Map;

When getting the cache value, if you want to atomically write the value to the cache when the cache value does not exist, you can call get(key, k -> value) method, which will avoid write competition;

Calling invalidate() method will manually remove the cache;

In the case of multi-threading, when using get(key, k -> value) , if another thread calls this method to compete at the same time, the latter thread will be blocked until the previous thread updates the cache;

And if another thread calls getIfPresent() method, it will return null immediately and will not be blocked;

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/type/CacheDemo.java

 public class CacheDemo { public static void main(String[] args) { Cache<String, String> cache = Caffeine.newBuilder().build(); System.out.println(cache.getIfPresent("123")); // null System.out.println(cache.get("123", k -> "456")); // 456 System.out.println(cache.getIfPresent("123")); // 456 cache.put("123", "789"); System.out.println(cache.getIfPresent("123")); // 789 }}

LoadingCache

LoadingCache is an automatic loading cache;

The difference from ordinary caches is: when the cache does not exist/the cache has expired, if get() method is called, CacheLoader.load() method will be automatically called to load the latest value;

Calling getAll() method will traverse all keys and call get() unless CacheLoader.loadAll() method is implemented.

When using LoadingCache, you need to specify CacheLoader, and implement load() method in it for automatic loading when the cache is missing.

In the case of multi-threading, when two threads call get() at the same time, the latter thread will be blocked until the previous thread updates the cache.

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/type/LoadingCacheDemo.java

 public class LoadingCacheDemo { public static void main(String[] args) { LoadingCache<String, String> cache = Caffeine.newBuilder() .build(new CacheLoader<String, String>() { @Override // 该方法必须实现public String load(@NonNull String k) throws Exception { return "456"; } @Override // 如果需要批量加载public @NonNull Map<String, String> loadAll(@NonNull Iterable<? extends String> keys) throws Exception { return new HashMap<String, String>() { }; } }); System.out.println(cache.getIfPresent("123")); // null System.out.println(cache.get("123")); // 456 System.out.println(cache.getAll(Arrays.asList("123", "456"))); // Map<String, String> }}

AsyncCache

AsyncCache is a variant of Cache, and its response results are all CompletableFuture. In this way, AsyncCache adapts to the asynchronous programming model;

By default, the cache calculation uses ForkJoinPool.commonPool() as the thread pool. If you want to specify the thread pool, you can override and implement Caffeine.executor(Executor) method.

synchronous() provides the ability to block until the asynchronous cache is generated, and it will return as Cache.

In the case of multi-threading, when two threads call get(key, k -> value) at the same time, the same CompletableFuture object will be returned. Since the returned result itself does not block, you can choose to block waiting or non-blocking according to the business design.

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/type/AsyncCacheDemo.java

 public class AsyncCacheDemo { public static void main(String[] args) throws ExecutionException, InterruptedException { String key = "123"; AsyncCache<String, String> cache = Caffeine.newBuilder().buildAsync(); CompletableFuture<String> completableFuture = cache.get(key, k -> "456"); System.out.println(completableFuture.get()); // 阻塞，直至缓存更新完成}}

AsyncLoadingCache

Obviously this is a functional combination of Loading Cache and Async Cache. AsyncLoadingCache supports automatic loading of the cache in an asynchronous manner.

Similar to LoadingCache, you also need to specify CacheLoader, and implement load() method in it for automatic loading when the cache is missing. This method will be automatically submitted in ForkJoinPool.commonPool() thread pool. If you want to specify an Executor, you can implement AsyncCacheLoader().asyncLoad() method.

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/type/AsyncLoadingCacheDemo.java

 public class AsyncLoadingCacheDemo { public static void main(String[] args) throws ExecutionException, InterruptedException { String key = "123"; AsyncLoadingCache<String, String> cache = Caffeine.newBuilder() .buildAsync(new AsyncCacheLoader<String, String>() { @Override // 自定义线程池加载public @NonNull CompletableFuture<String> asyncLoad(@NonNull String key, @NonNull Executor executor) { return CompletableFuture.completedFuture("456"); } });// .buildAsync(new CacheLoader<String, String>() {// @Override// // OR，使用默认线程池加载（二者选其一）// public String load(@NonNull String key) throws Exception {// return "456";// }// }); CompletableFuture<String> completableFuture = cache.get(key); // CompletableFuture<String> System.out.println(completableFuture.get());; // 阻塞，直至缓存更新完成}}

Eviction strategy

The eviction policy is specified when creating the cache;

Commonly used are: capacity-based eviction and time-based eviction;

  • Capacity-based eviction: the maximum cache capacity needs to be specified; when the cache capacity reaches the maximum, Caffeine will use the LRU strategy to eliminate the cache;
  • Time-based eviction: It can be set to automatically evict after the last access/write to a cache after a specified time;

Eviction strategies can be used in combination. After any eviction strategy takes effect, the cache entry will be evicted;

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/evict/EvictDemo.java

 public class EvictDemo { public static void main(String[] args) { // 创建一个最大容量为10的缓存Cache<String, String> cache1 = Caffeine.newBuilder(). maximumSize(10).build(); // 创建一个写入5s后过期的缓存Cache<String, String> cache2 = Caffeine.newBuilder(). expireAfterWrite(5, TimeUnit.SECONDS).build(); // 创建一个访问1s后过期的缓存Cache<String, String> cache3 = Caffeine.newBuilder(). expireAfterAccess(1, TimeUnit.SECONDS).build(); }}

refresh mechanism

Imagine such a situation: when the cache is running, some cache values ​​need to be refreshed periodically to ensure that the information can be correctly synchronized to the cache;

Of course we can use the time-based eviction strategy expireAfterWrite() , but the problem is that once the cache expires, the calling thread will be blocked the next time the cache is reloaded;

Using the refresh mechanism refreshAfterWrite() , Caffeine will immediately return the old value when the key is allowed to be accessed for the first time after refreshing, and at the same time refresh the cache value asynchronously, which prevents the caller from being blocked due to cache eviction;

It should be noted that the refresh mechanism only supports LoadingCache and AsyncLoadingCache;

By overriding CacheLoader.reload() method, the old cache value will be involved in the refresh;

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/refresh/RefreshDemo.java

 public class RefreshDemo { public static void main(String[] args) { LoadingCache<String, String> cache1 = Caffeine.newBuilder(). refreshAfterWrite(10, TimeUnit.MINUTES). build(RefreshDemo::create); } private static String create(String k) { return k; }}

statistics

Caffeine has a built-in data collection function, and the data collection can be turned on through Caffeine.recordStats() method;

In this way, Cache.stats() method will return some statistical indicators of the current cache, for example:

  • hitRate : The hit rate of the query cache
  • evictionCount : the number of caches evicted
  • averageLoadPenalty : the average time it takes for new values ​​to be loaded

cache/caffeine/basic/src/main/java/io/github/jasonkayzk/record/RecordDemo.java

 public class RecordDemo { public static void main(String[] args) { // 获取统计指标Cache<String, String> cache = Caffeine.newBuilder(). recordStats().build(); System.out.println(cache.stats()); System.out.println(cache.estimatedSize()); }}

output:

 CacheStats{hitCount=0, missCount=0, loadSuccessCount=0, loadFailureCount=0, totalLoadTime=0, evictionCount=0, evictionWeight=0}0

Integrating Caffeine in SpringBoot

SpringBoot cache manager

Spring has introduced support for Cache since 3.1. org.springframework.cache.Cache and org.springframework.cache.CacheManager interfaces are defined to unify different caching technologies and support the use of JCache(JSR-107) annotations to simplify development.

  • The Cache interface includes a collection of various operations of the cache. When the cache is actually operated, it is operated through these interfaces.
  • Under the Cache interface, Spring provides various xxxCache implementations. Since SpringBoot 2.x officially replaces Guava with Caffeine as the default cache component, what we need to use here is the CaffeineCache class. If you need to customize the Cache implementation, you only need to implement the Cache interface.
  • CacheManager defines the creation, configuration, acquisition, management and control of multiple uniquely named Cache. These Cache exist in the context of CacheManager.

Create a cache manager:

 @Beanpublic CacheManager cacheManager() { SimpleCacheManager cacheManager = new SimpleCacheManager(); ArrayList<CaffeineCache> caches = new ArrayList<>(); // String cacheName(): 创建缓存名称// Cache<Object, Object> generateCache(): 创建一个Caffeine缓存caches.add(new CaffeineCache(cacheName(), generateCache())); cacheManager.setCaches(caches); return cacheManager;}

In this way, multiple caches can be added to the cache manager at the same time. It should be noted that SimpleCacheManager can only use Cache and LoadingCache, and asynchronous cache will not be supported .

Use @Cacheable related annotations

@Cacheable Related Comments

After adding the cache manager, we can easily use @Cacheable related annotations to manage the cache. In order to use this annotation, the following dependencies need to be introduced:

 <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-cache</artifactId></dependency>

Commonly used annotations related to @Cacheable include:

  • @Cacheable : Indicates that the method supports caching. When the annotated method is called, if the corresponding key already exists in the cache, the method body will not be executed, but will be returned directly from the cache. When the method returns null, no cache operation will be performed.
  • @CachePut : Indicates that after the method is executed, its value will be updated to the cache as the latest result. This method is executed every time .
  • @CacheEvict : Indicates that after the method is executed, the cache clearing operation will be triggered.
  • @Caching : Used to combine the first three annotations, for example:
 @Caching(cacheable = @Cacheable("users"), evict = {@CacheEvict("cache2"), @CacheEvict(value = "cache3", allEntries = true)})public User find(Integer id) { return null;}

This type of annotation can also be marked on a class, indicating that all methods of this class support the corresponding cache annotation.

Common Annotation Properties

The commonly used annotation attributes @Cacheable are as follows:

  • cacheNames/value : The name of the cache component, that is, the name of the cache in the cacheManager.
  • key : The key used when caching data. By default, method parameter values ​​are used, and can also be written using SpEL expressions.
  • keyGenerator : use one of the two keys.
  • cacheManager : Specifies the cache manager to use.
  • condition : Check before the method execution starts, and cache if the condition is met
  • unless : Check after the method is executed, and do not cache if it matches unless
  • sync : Whether to use synchronous mode. If synchronous mode is used, when multiple threads load a key at the same time, other threads will be blocked.

Here is an example of annotation usage:

 @Cacheable(value = "UnitCache", key = "#unitType + T(top.kotoumi.constants.Constants).SPLIT_STR + #unitId", condition = "#unitType != 'weapon'")public Unit getUnit(String unitType, String unitId) { return getUnit(unitType, unitId);}

The cache used by this method is UnitCache, and the manually specified cache key is the splicing result of #unitType + Constants.SPLIT_STR + #unitId . This cache will take effect when #unitType != 'weapon' .

Cache Synchronization Mode

@Cacheable annotation supports configuration synchronization mode. Under different Caffeine configurations, observe whether the synchronization mode is enabled.

Caffeine cache type Whether to enable synchronization Multi-threaded reading of non-existent/evicted keys Multi-threaded read key to be refreshed
Cache no Execute the annotated method independently –
Cache yes Thread 1 executes the annotated method, and thread 2 is blocked until the cache update is completed –
LoadingCache no Thread 1 executes load() , and thread 2 is blocked until the cache update is completed Thread 1 returns immediately with the old value and updates the cached value asynchronously; thread 2 returns immediately without updating.
LoadingCache yes Thread 1 executes the annotated method, and thread 2 is blocked until the cache update is completed Thread 1 returns immediately with the old value and updates the cached value asynchronously; thread 2 returns immediately without updating.

As can be seen from the above summary, whether sync is turned on or off, the performance in Cache and LoadingCache is inconsistent:

  • In Cache, sync indicates whether all threads need to wait synchronously
  • In LoadingCache, sync indicates whether to execute the annotated method when reading a non-existent/evicted key

In fact, there is no lock processing in the reading process of Cache AOP, and the actual form of this parameter is determined by the cache implementation. When using Caffeine Cache, you can quickly find a suitable combination according to the above table.

Caffeine integration practice

Introduce related dependencies:

 <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-cache</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </dependency></dependencies>

This part is mainly to cache User by using Caffeine;

User structure

The User POJO structure is as follows:

cache/caffeine/spring-boot/src/main/java/io/github/jasonkayzk/entity/User.java

 @Data@NoArgsConstructor@AllArgsConstructorpublic class User { private String id; private String userType; private String password;}

caching component

For a cache, we mainly care about the following aspects:

  • Cache name: used in @Cacheable annotation;
  • Maximum capacity/cache expiration time: used by eviction strategy;
  • Update time: update policy usage;
  • Update method: used for the implementation of CacheLoader.load() ;

Therefore, a basic configuration class can be defined so that all the cache configurations we actually use inherit from it:

cache/caffeine/spring-boot/src/main/java/io/github/jasonkayzk/cache/BaseCaffeineCacheConfig.java

 @Datapublic abstract class BaseCaffeineCacheConfig { /** * 缓存名称*/ private String name = "caffeine"; /** * 默认最大容量，大于0生效*/ private int maxSize = 100; /** * 缓存过期时间（秒），大于0生效*/ private int expireDuration = -1; /** * 缓存刷新时间（秒），大于0生效，且表示这是一个LoadingCache，否则表示是一个普通Cache */ private int refreshDuration = -1; private Cache<Object, Object> cache; /** * 获取特定缓存值* @param key key * @return 缓存值*/ public abstract Object getValue(Object key);}

Actual cache class:

cache/caffeine/spring-boot/src/main/java/io/github/jasonkayzk/cache/UserCache.java

 @Service@Getter@EqualsAndHashCode(callSuper = true)public class UserCache extends BaseCaffeineCacheConfig { /** * 缓存名称*/ private final String name = "UserCache"; /** * 默认最大容量，大于0生效*/ private final int maxSize = 100; /** * 缓存过期时间（秒），大于0生效*/ private final int expireDuration = 86400; /** * 缓存刷新时间（秒），大于0生效，且表示这是一个LoadingCache，否则表示是一个普通Cache */ private final int refreshDuration = 600; /** * 获取特定缓存值* * @param key key * @return 缓存值*/ public Object getValue(Object key) { String[] param = ((String) key).split(Constants.SPLIT_STR); return getUser(param[0], param[1]); } @Cacheable(value = "UserCache", key = "#userType + T(io.github.jasonkayzk.consts.Constants).SPLIT_STR + #userId", condition = "#userType != 'root'") public User getUser(String userType, String userId) { if ("1".equals(userId) && "admin".equals(userType)) { return new User("1", "admin", "admin-password"); } else { return new User("999", "visitor", "no-password"); } } @CacheEvict(value = "UserCache", key = "#userType + T(io.github.jasonkayzk.consts.Constants).SPLIT_STR + #userId", condition = "#userType != 'root'") public void deleteUser(String userType, String userId) { }}

Here we annotate this class as @Service to facilitate subsequent callers to execute;

In addition, you can see that @Cacheable annotation is on the method used in the actual business, and its cache name is exactly the name specified by the name. In addition, we also need to make getValue() method call the annotated business method, which is to ensure the correct execution of load() method;

Register in SpringBoot:

cache/caffeine/spring-boot/src/main/java/io/github/jasonkayzk/cache/CaffeineCacheConfig.java

 @Slf4j@Configuration@EnableCachingpublic class CaffeineCacheConfig { Logger logger = LoggerFactory.getLogger(CaffeineCacheConfig.class); @Resource private UserCache userCache; @Bean public CacheManager cacheManager() { // 添加所有创建的缓存，这里仅添加一个用于示例List<BaseCaffeineCacheConfig> cacheConfigs = new ArrayList<>(); cacheConfigs.add(userCache); // 创建缓存管理器下的所有缓存SimpleCacheManager cacheManager = new SimpleCacheManager(); ArrayList<CaffeineCache> caches = new ArrayList<>(); for (BaseCaffeineCacheConfig config : cacheConfigs) { caches.add(new CaffeineCache(config.getName(), generateCache(config))); logger.info("registered cache: {}", config.getName()); } cacheManager.setCaches(caches); return cacheManager; } /** * 生成cache，并开启统计功能* * @param config 配置信息* @return cache */ private Cache<Object, Object> generateCache(BaseCaffeineCacheConfig config) { // 创建缓存Cache<Object, Object> cache; Caffeine<Object, Object> builder = Caffeine.newBuilder().recordStats(); if (config.getMaxSize() > 0) { builder.maximumSize(config.getMaxSize()); } if (config.getExpireDuration() > 0) { builder.expireAfterWrite(config.getExpireDuration(), TimeUnit.SECONDS); } if (config.getRefreshDuration() > 0) { // 创建LoadingCache，需要传入CacheLoader builder.refreshAfterWrite(config.getRefreshDuration(), TimeUnit.SECONDS); cache = builder.build(cacheLoader(config)); } else { // 创建普通Cache cache = builder.build(); } config.setCache(cache); return cache; } /** * 构造cache loader * * @param config 配置* @return cache loader */ private CacheLoader<Object, Object> cacheLoader(BaseCaffeineCacheConfig config) { // 使用配置类中的getValue()方法return config::getValue; }}

In the cache manager, use @Resource annotation to obtain an instance of the cache configuration class, and use the configuration to generate the corresponding Cache to be managed by SpringBoot;

Note: Since only LoadingCache can use refreshAfterWrite() method, it is necessary to determine whether to generate LoadingCache or ordinary Cache according to the incoming parameters;

In addition, it should be noted that if you need to enable the caching function, you need to use the @EnableCaching annotation on the SpringBoot startup class or any configuration class;

Use the cache in the controller

cache/caffeine/spring-boot/src/main/java/io/github/jasonkayzk/controller/UserController.java

 @Slf4j@RestController@RequestMapping("/user")public class UserController { @Resource private UserCache userCache; Logger logger = LoggerFactory.getLogger(UserController.class); @GetMapping("/{userType}/{userId}") public User getUser(@PathVariable String userType, @PathVariable String userId) { logger.info("getUser: userType: {}, userId: {}", userType, userId); userCache.getCache().asMap().forEach((key, user) -> logger.info(user.toString())); return userCache.getUser(userType, userId); } @DeleteMapping("/{userType}/{userId}") public void deleteUser(@PathVariable String userType, @PathVariable String userId) { userCache.deleteUser(userType, userId); } @GetMapping("/cachestat") public String cacheStat() { // 获取Cache实例，并调用stat()方法查看缓存情况return userCache.getCache().stats().toString(); }}

Note: @Cacheable annotation is made through the Spring AOP mechanism, so calls within the class will not be able to trigger cache operations and must be called externally;

At the same time, notice that a member variable is added to the cache configuration class, and the cache object is passed in generateCache() method:

 public abstract class BaseCaffeineCacheConfig { private Cache<Object, Object> cache;}private Cache<Object, Object> generateCache(BaseCaffeineCacheConfig config) { config.setCache(cache); return cache;}

You can call the relevant statistical methods;

test

Start the service, first visit: http://127.0.0.1:8848/user/root/1

Since in the configuration above:

 @Cacheable(value = "UserCache", key = "#userType + T(io.github.jasonkayzk.consts.Constants).SPLIT_STR + #userId", condition = "#userType != 'root'")

For userType==root , it will not be cached, so the default data will be returned:

 {"id":"999","userType":"visitor","password":"no-password"}

And visit: http://127.0.0.1:8848/user/cachestat

It shows that there is no cache at this time:

 CacheStats{hitCount=0, missCount=0, loadSuccessCount=0, loadFailureCount=0, totalLoadTime=0, evictionCount=0, evictionWeight=0}

Then, visit: http://127.0.0.1:8848/user/admin/1, return:

 {"id":"1","userType":"admin","password":"admin-password"}

Visit again: http://127.0.0.1:8848/user/visitor/1, return:

 {"id":"999","userType":"visitor","password":"no-password"}

Visit again: http://127.0.0.1:8848/user/cachestat to check the cache status:

 CacheStats{hitCount=0, missCount=2, loadSuccessCount=2, loadFailureCount=0, totalLoadTime=395541, evictionCount=0, evictionWeight=0}

appendix

source code:

Reference article:

This article is reproduced from: https://jasonkayzk.github.io/2023/03/28/Java%E9%AB%98%E6%80%A7%E8%83%BD%E7%BC%93%E5%AD%98 %E5%BA%93Caffeine/
This site is only for collection, and the copyright belongs to the original author.