Skip to content

feat: Added a generic LRUCache interface and a default implementation #482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 5, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions core-api/src/main/java/com/optimizely/ab/internal/DefaultLRUCache.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
/**
*
* Copyright 2022, Optimizely
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.optimizely.ab.internal;

import com.optimizely.ab.annotations.VisibleForTesting;

import java.util.*;

public class DefaultLRUCache<T> implements LRUCache<T> {

private final Object lock = new Object();

private Integer maxSize;

private Long timeoutMillis;

final LinkedList<ItemWrapper> linkedList = new LinkedList<>();

final Map<String, ItemWrapper> hashMap = new HashMap<>();

public DefaultLRUCache() {
this.maxSize = DEFAULT_MAX_SIZE;
this.timeoutMillis = (long) (DEFAULT_TIMEOUT_SECONDS * 1000);
}

public void setMaxSize(Integer size) {
Integer sizeToSet = size;
if (size < 0 ) {
sizeToSet = DEFAULT_MAX_SIZE;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either way is fine, but let's use "sizeToSet = 0" for all SDKs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Defaulting to zero now when max size is negative

}

synchronized (lock) {
if (linkedList.size() > sizeToSet) {
// If cache is bigger than the new maxSize, remove additional items from the end.
int itemsToTrim = linkedList.size() - sizeToSet;
for (int i = 0; i < itemsToTrim; i++) {
ItemWrapper item = linkedList.removeLast();
hashMap.remove(item.key);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this support for dynamic cache size changes? We can allow them cache size setting only when app starts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to only allow when maxSize passed in is less than the current size of the cache.

this.maxSize = sizeToSet;
}
}

public void setTimeout(Long timeoutSeconds) {
this.timeoutMillis = timeoutSeconds * 1000;
}

public void save(String key, T value) {
if (maxSize == 0) {
// Cache is disabled when maxSize = 0
return;
}
synchronized (lock) {
ItemWrapper item = new ItemWrapper(key, value);

if (hashMap.containsKey(key)) {
ItemWrapper oldItem = hashMap.get(key);
linkedList.remove(oldItem);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this linear search? It may be a factor in large cache.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good catch. Because i was passing the item itself and it uses a doubly linked list, i thought the item should have references to its neighbors and should be able to cleanly remove itself in O(1) but it looks like the actual node with references is managed internally and remove actually searched for the item linearly. I then went ahead and redid the whole implementation using LunkedHashMap

linkedList.addFirst(item);
hashMap.replace(key, item);
return;
}

if (linkedList.size() == maxSize) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (linkedList.size() == maxSize) {
if (linkedList.size() >= maxSize) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the whole implementation. It does not apply anymore.

ItemWrapper removedItem = linkedList.removeLast();
hashMap.remove(removedItem.key);
}

linkedList.addFirst(item);
hashMap.put(key, item);
}
}

public T lookup(String key) {
if (maxSize == 0) {
// Cache is disabled when maxSize = 0
return null;
}

synchronized (lock) {
if (hashMap.containsKey(key)) {
ItemWrapper item = hashMap.get(key);
Long nowMs = new Date().getTime();

// ttl = 0 means items never expire.
if (timeoutMillis == 0 || (nowMs - item.timestamp < timeoutMillis)) {
linkedList.remove(item);
linkedList.addFirst(item);
return item.value;
}

// If item is stale, remove it.
linkedList.remove(item);
hashMap.remove(item.key);
}
return null;
}
}

public T peek(String key) {
if (maxSize == 0) {
// Cache is disabled when maxSize = 0
return null;
}

if (hashMap.containsKey(key)) {
return hashMap.get(key).value;
}

return null;
}

public void reset() {
synchronized (lock) {
linkedList.clear();
hashMap.clear();
}
}

@VisibleForTesting
class ItemWrapper {
public String key;
public T value;
public Long timestamp;

public ItemWrapper(String key, T value) {
this.key = key;
this.value = value;
this.timestamp = new Date().getTime();
}
}
}
28 changes: 28 additions & 0 deletions core-api/src/main/java/com/optimizely/ab/internal/LRUCache.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/**
*
* Copyright 2022, Optimizely
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.optimizely.ab.internal;

public interface LRUCache<T> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public interface LRUCache<T> {
public interface Cache<T> {

This interface can be for generic caches, not only for LRUCache. Not sure about the best name to avoid conflicts., but "LRU" should be dropped. I think this is for internal use (not for public custom segments cache). If this interface is for public, I have a few other suggestions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing it to Cache for now as per your suggestion. We can discuss this further in subsequent PRs when things become more clear.

int DEFAULT_MAX_SIZE = 10000;
int DEFAULT_TIMEOUT_SECONDS = 600;
void save(String key, T value);
T lookup(String key);
T peek(String key);
void reset();
void setMaxSize(Integer size);
void setTimeout(Long timeoutSeconds);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
/**
*
* Copyright 2022, Optimizely
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.optimizely.ab.internal;

import org.junit.Test;

import java.util.Arrays;
import java.util.List;

import static org.junit.Assert.*;

public class DefaultLRUCacheTest {

@Test
public void createSaveAndLookupOneItem() {
LRUCache<String> cache = new DefaultLRUCache<>();
assertNull(cache.lookup("key1"));
cache.save("key1", "value1");
assertEquals("value1", cache.lookup("key1"));
}

@Test
public void saveAndLookupMultipleItems() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test looks good. It will be good to have the same ordering switch tests for save ops as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>();

cache.save("user1", Arrays.asList("segment1", "segment2"));
cache.save("user2", Arrays.asList("segment3", "segment4"));
cache.save("user3", Arrays.asList("segment5", "segment6"));

assertEquals("user3", cache.linkedList.get(0).key);
assertEquals("user2", cache.linkedList.get(1).key);
assertEquals("user1", cache.linkedList.get(2).key);

assertEquals(Arrays.asList("segment1", "segment2"), cache.lookup("user1"));

// Lookup should move user1 to top of the list and push down others.
assertEquals("user1", cache.linkedList.get(0).key);
assertEquals("user3", cache.linkedList.get(1).key);
assertEquals("user2", cache.linkedList.get(2).key);

assertEquals(Arrays.asList("segment3", "segment4"), cache.lookup("user2"));

// Lookup should move user2 to top of the list and push down others.
assertEquals("user2", cache.linkedList.get(0).key);
assertEquals("user1", cache.linkedList.get(1).key);
assertEquals("user3", cache.linkedList.get(2).key);

assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3"));

// Lookup should move user3 to top of the list and push down others.
assertEquals("user3", cache.linkedList.get(0).key);
assertEquals("user2", cache.linkedList.get(1).key);
assertEquals("user1", cache.linkedList.get(2).key);
}

@Test
public void whenCacheIsDisabled() {
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>();
cache.setMaxSize(0);
cache.save("user1", Arrays.asList("segment1", "segment2"));
cache.save("user2", Arrays.asList("segment3", "segment4"));
cache.save("user3", Arrays.asList("segment5", "segment6"));

assertNull(cache.lookup("user1"));
assertNull(cache.lookup("user2"));
assertNull(cache.lookup("user3"));
}

@Test
public void whenItemsExpire() throws InterruptedException {
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>();
cache.setTimeout(1L);
cache.save("user1", Arrays.asList("segment1", "segment2"));
assertEquals(Arrays.asList("segment1", "segment2"), cache.lookup("user1"));
assertEquals(1, cache.linkedList.size());
assertEquals(1, cache.hashMap.size());
Thread.sleep(1000);
assertNull(cache.lookup("user1"));
assertEquals(0, cache.linkedList.size());
assertEquals(0, cache.hashMap.size());
}

@Test
public void whenCacheReachesMaxSize() {
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>();
cache.setMaxSize(2);

cache.save("user1", Arrays.asList("segment1", "segment2"));
cache.save("user2", Arrays.asList("segment3", "segment4"));
cache.save("user3", Arrays.asList("segment5", "segment6"));

assertEquals(2, cache.linkedList.size());
assertEquals(2, cache.hashMap.size());

assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3"));
assertEquals(Arrays.asList("segment3", "segment4"), cache.lookup("user2"));
assertNull(cache.lookup("user1"));
}

@Test
public void whenMaxSizeIsReducedInBetween() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified the test to check the changed implementation

DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>();
cache.save("user1", Arrays.asList("segment1", "segment2"));
cache.save("user2", Arrays.asList("segment3", "segment4"));
cache.save("user3", Arrays.asList("segment5", "segment6"));

assertEquals(Arrays.asList("segment1", "segment2"), cache.lookup("user1"));
assertEquals(Arrays.asList("segment3", "segment4"), cache.lookup("user2"));
assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3"));

assertEquals(3, cache.linkedList.size());
assertEquals(3, cache.hashMap.size());

cache.setMaxSize(1);

assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3"));
assertNull(cache.lookup("user1"));
assertNull(cache.lookup("user2"));

assertEquals(1, cache.linkedList.size());
assertEquals(1, cache.hashMap.size());
}

@Test
public void saveAndPeekItems() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A test for reset()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>();

cache.save("user1", Arrays.asList("segment1", "segment2"));
cache.save("user2", Arrays.asList("segment3", "segment4"));
cache.save("user3", Arrays.asList("segment5", "segment6"));

assertEquals("user3", cache.linkedList.get(0).key);
assertEquals("user2", cache.linkedList.get(1).key);
assertEquals("user1", cache.linkedList.get(2).key);

assertEquals(Arrays.asList("segment1", "segment2"), cache.peek("user1"));

// Peek should not alter the order of array
assertEquals("user3", cache.linkedList.get(0).key);
assertEquals("user2", cache.linkedList.get(1).key);
assertEquals("user1", cache.linkedList.get(2).key);

assertEquals(Arrays.asList("segment3", "segment4"), cache.peek("user2"));

// Peek should not alter the order of array
assertEquals("user3", cache.linkedList.get(0).key);
assertEquals("user2", cache.linkedList.get(1).key);
assertEquals("user1", cache.linkedList.get(2).key);

assertEquals(Arrays.asList("segment5", "segment6"), cache.peek("user3"));

// Peek should not alter the order of array
assertEquals("user3", cache.linkedList.get(0).key);
assertEquals("user2", cache.linkedList.get(1).key);
assertEquals("user1", cache.linkedList.get(2).key);
}
}