Bloom filter with Redis: preventing cache penetration in Spring Boot

1. Overview

Cache penetration occurs when clients repeatedly request keys that don’t exist. Since the cache has no data, every request falls through to the database. This article explains why Bloom filter is a cheap and effective shield for this situation. You’ll also learn how to implement it on Redis using Redisson in a Spring Boot 3.x application.

By the end, you’ll know how to choose filter parameters, load initial data, integrate the filter into read flows, and write tests to verify its behavior.

2. What is cache penetration?

Suppose you have a ProductService using the familiar cache-aside pattern:

public Optional<Product> findById(long id) {
    RBucket<Product> bucket = redisson.getBucket("product:" + id, JsonJacksonCodec.INSTANCE);
    Product cached = bucket.get();
    if (cached != null) {
        return Optional.of(cached);
    }
    Optional<Product> fromDb = productRepository.findById(id); // query DB
    fromDb.ifPresent(p -> bucket.set(p, Duration.ofMinutes(30)));
    return fromDb;
}

This flow works fine with real data. Now see what happens when someone sends thousands of requests with id = -1, id = 999999999, or random ids:

• Redis returns null because the key doesn’t exist.
• Code falls through to the DB, which also finds nothing.
• Nothing is written to the cache, so the next request repeats the same process.

The result is that the cache becomes useless and the database bears the full load. This could be a client bug, a crawler, or a deliberate attack.

A common first solution is to cache null values with a short TTL. This works well when the set of bad keys is small. It fails when an attacker uses random ids: each id appears only once, and you’d have to store millions of meaningless empty keys. This is where Bloom filter shines.

3. How a Bloom filter works

A Bloom filter is a probabilistic data structure with two components:

• A bit array of m bits, initially all 0.
• k hash functions, each mapping an element to a position in the array.

Its two main operations are:

• add(x): compute k hashes of x and set the corresponding bits to 1.
• contains(x): compute k hashes of x. If any bit is 0, x is definitely not in the set. If all are 1, x might be in the set.

The key property is no false negatives, only false positives. When the filter says “not found,” you can be certain and return 404 without touching cache or DB. When it says “might exist,” you proceed with the normal flow. This small error rate is completely acceptable.

The filter size is computed from the expected number of elements n and desired false positive rate p:

m = -n * ln(p) / (ln 2)^2
k = (m / n) * ln 2

For example, with n = 1_000_000 products and p = 0.01 (1%):

m ≈ 9_585_059 bit ≈ 1.14 MB
k ≈ 7 hash functions

Just about 1 MB of RAM on Redis is enough to protect a million ids. This is much smaller than storing the ids themselves in a Set.

4. Setting up Redisson

Redisson provides RBloomFilter, implemented using pure Redis bitmaps. Therefore it runs on any Redis version without additional modules.

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson-spring-boot-starter</artifactId>
    <version>3.45.1</version>
</dependency>

The starter automatically reads connection configuration from spring.data.redis.* properties:

spring.data.redis.host=localhost
spring.data.redis.port=6379

After this step, you can inject a RedissonClient bean anywhere in your application.

5. Initializing the Bloom filter

Create the filter as a bean to be shared across the application:

@Configuration
public class BloomFilterConfig {

    public static final String PRODUCT_FILTER = "bf:product:ids";

    @Bean
    public RBloomFilter<Long> productIdFilter(RedissonClient redisson) {
        RBloomFilter<Long> filter = redisson.getBloomFilter(PRODUCT_FILTER);
        boolean created = filter.tryInit(1_000_000L, 0.01);
        if (!created) {
            // Filter already exists on Redis, keeping old configuration
            log.info("Bloom filter {} already exists, size={} bit, k={}",
                    PRODUCT_FILTER, filter.getSize(), filter.getHashIterations());
        }
        return filter;
    }

    private static final Logger log = LoggerFactory.getLogger(BloomFilterConfig.class);
}

The code above does the following:

• getBloomFilter(...) only creates a proxy object and doesn’t write anything to Redis yet.
• tryInit(1_000_000L, 0.01) computes m and k using the formula from section 3, then saves the configuration to an auxiliary hash on Redis. The method returns false if the filter was already initialized, for example by another instance of the application.
• getSize() and getHashIterations() tell you how many bits and hash functions the filter is actually using.

With the parameters above, the log will output something like:

Bloom filter bf:product:ids already exists, size=9585058 bit, k=7

Note: When the filter already exists, tryInit will not change the configuration. If you change expectedInsertions in code without deleting the old filter, the new configuration will be silently ignored.

6. Loading initial data

An empty filter will return “not found” for all ids, including real ones. Therefore you must load all existing ids before enabling the check.

@Component
public class ProductFilterWarmUp implements ApplicationRunner {

    private final ProductRepository productRepository;
    private final RBloomFilter<Long> productIdFilter;

    public ProductFilterWarmUp(ProductRepository productRepository,
                               RBloomFilter<Long> productIdFilter) {
        this.productRepository = productRepository;
        this.productIdFilter = productIdFilter;
    }

    @Override
    public void run(ApplicationArguments args) {
        if (productIdFilter.count() > 0) {
            return; // already loaded by another instance
        }
        List<Long> ids = productRepository.findAllIds();
        ids.forEach(productIdFilter::add);
        log.info("Loaded {} ids into Bloom filter", ids.size());
    }

    private static final Logger log = LoggerFactory.getLogger(ProductFilterWarmUp.class);
}

Here, ProductRepository.findAllIds() fetches only the id column, for example @Query("select p.id from Product p") if using JPA. count() returns an estimated element count based on set bits, so it’s good enough to check if the filter has data.

With millions of records, you should read in batches (using pagination or Stream with fetch size) instead of a single List. Each add command is a round-trip to Redis, so consider running warm-up as a separate job instead of slowing down application startup.

7. Integrating into ProductService

Now place the filter before the cache in the read flow:

@Service
public class ProductService {

    private final RedissonClient redisson;
    private final RBloomFilter<Long> productIdFilter;
    private final ProductRepository productRepository;

    public ProductService(RedissonClient redisson,
                          RBloomFilter<Long> productIdFilter,
                          ProductRepository productRepository) {
        this.redisson = redisson;
        this.productIdFilter = productIdFilter;
        this.productRepository = productRepository;
    }

    public Optional<Product> findById(long id) {
        if (!productIdFilter.contains(id)) {
            return Optional.empty(); // definitely doesn't exist
        }
        RBucket<Product> bucket = redisson.getBucket("product:" + id, JsonJacksonCodec.INSTANCE);
        Product cached = bucket.get();
        if (cached != null) {
            return Optional.of(cached);
        }
        Optional<Product> fromDb = productRepository.findById(id);
        fromDb.ifPresent(p -> bucket.set(p, Duration.ofMinutes(30)));
        return fromDb;
    }

    @Transactional
    public Product create(Product product) {
        Product saved = productRepository.save(product);
        productIdFilter.add(saved.id());
        return saved;
    }
}

Product here is a simple record record Product(Long id, String name, BigDecimal price). Let’s go through each step:

• contains(id) returns false means the id is definitely not in the system. The request is blocked immediately with the cost of one Redis command, without hitting the database.
• If the filter returns true, the request proceeds through the cache-aside flow as before. About 1% of non-existent ids still slip through as false positives. The database handles this small amount easily.
• In create, the new id is added to the filter. If you forget this step, the newly created product will “disappear” for users.

Regarding write order, if add runs within a transaction but the commit fails, the filter will contain a non-existent id. The consequence is just an additional false positive, which is harmless. The opposite case is more problematic: if you only add after commit, there will be a brief window where the product exists in the DB but the filter still returns “not found.” That would be a false negative of your own making. Therefore, prioritize adding to the filter before or at the same time as writing to the DB.

8. Testing filter behavior

To test against a real Redis, add Testcontainers:

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>1.20.4</version>
    <scope>test</scope>
</dependency>

The test below verifies two core properties: no false negatives, and false positive rate close to the configured value.

@Testcontainers
class ProductBloomFilterTest {

    @Container
    static GenericContainer<?> redis =
            new GenericContainer<>("redis:7.4-alpine").withExposedPorts(6379);

    static RedissonClient client;
    static RBloomFilter<Long> filter;

    @BeforeAll
    static void setUp() {
        Config config = new Config();
        config.useSingleServer()
              .setAddress("redis://" + redis.getHost() + ":" + redis.getMappedPort(6379));
        client = Redisson.create(config);

        filter = client.getBloomFilter("test:product:ids");
        filter.tryInit(5_000L, 0.01);
        LongStream.rangeClosed(1, 5_000).forEach(filter::add);
    }

    @AfterAll
    static void tearDown() {
        client.shutdown();
    }

    @Test
    void givenAddedIds_whenContains_thenAlwaysTrue() {
        boolean allPresent = LongStream.rangeClosed(1, 5_000).allMatch(filter::contains);

        assertThat(allPresent).isTrue();
    }

    @Test
    void givenUnknownIds_whenContains_thenFalsePositiveRateIsLow() {
        long falsePositives = LongStream.rangeClosed(5_001, 10_000)
                .filter(filter::contains)
                .count();

        double rate = falsePositives / 5_000.0;
        assertThat(rate).isLessThan(0.02);
    }
}

The first test asserts that all added ids are recognized. The second test counts how many ids not added are still reported as “might exist” by the filter. With p = 0.01, the measured rate typically hovers around 1%. The 2% threshold keeps the test from being flaky.

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

9. Alternative: native Redis Bloom commands

If you use Redis Stack or Redis 8+ (which have probabilistic data types built-in), you can use BF.* commands directly:

redis-cli BF.RESERVE bf:product:ids 0.01 1000000
redis-cli BF.MADD bf:product:ids 1 2 3
redis-cli BF.EXISTS bf:product:ids 2
redis-cli BF.EXISTS bf:product:ids 42

OK
1) (integer) 1
2) (integer) 1
3) (integer) 1
(integer) 1
(integer) 0

The native version has a few advantages. Each BF.EXISTS command computes hashes only once on the server side. BF.MADD allows adding multiple elements in one command. Also, the filter can auto-scale when exceeding capacity. On the other hand, Redisson’s RBloomFilter works on plain Redis, including managed services without module support. Choose based on your infrastructure.

10. Common pitfalls and when not to use

Note: Classic Bloom filters don’t support deletion. Turning off a bit might “lose” other elements sharing that bit, creating false negatives.

A few points to remember:

• Data deletion: when a product is deleted, its id remains in the filter. This is safe because it only becomes a false positive. However, if deletion rate is high, the filter will become “dirty” over time. The solution is to periodically rebuild the filter into a new key (e.g., bf:product:ids:v2) and switch with RENAME. Another option is Cuckoo filter (CF.*), which supports deletion.
• Exceeding capacity: if you actually have 5 million elements but the filter was declared for 1 million, the false positive rate will spike. Monitor count() against getExpectedInsertions() and rebuild before hitting the threshold.
• Forgetting to load or incomplete loading: an empty or incomplete filter will block even valid requests. You can use a feature flag to enable checks only after warm-up completes.
• Multiple write paths: if data is created via batch jobs, CSV imports, or other services, all write paths must update the filter. A safer approach is to update via CDC or events.
• Don’t use when: the key set is small (a few thousand elements, a Redis Set is enough), keys can’t be enumerated upfront (e.g., free-text search results), or the system can’t tolerate sync lag between DB and filter.

Combining Bloom filter with null-value caching at short TTL is a reasonable multi-layer defense. Bloom filter blocks most garbage ids, while null cache handles the few false positives that get called repeatedly.

11. Conclusion

Cache penetration renders caching useless when requests target non-existent keys. Bloom filter solves this at very low cost: about 1 MB for a million ids, one Redis command per check, and guaranteed no false negatives.

In this article, you saw how to compute filter size and initialize it with Redisson’s RBloomFilter. You also learned to load initial data, place the filter before cache in ProductService, and write tests validating its behavior with Testcontainers. Finally, always plan to periodically rebuild the filter and keep all data write paths in sync with it.