-
Notifications
You must be signed in to change notification settings - Fork 32
feat: Added a generic LRUCache interface and a default implementation #482
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
21d74d4
8fbd398
46f0004
3fda8cb
1763ac2
1611a86
c63affd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,147 @@ | ||||||
/** | ||||||
* | ||||||
* Copyright 2022, Optimizely | ||||||
* | ||||||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||||||
* you may not use this file except in compliance with the License. | ||||||
* You may obtain a copy of the License at | ||||||
* | ||||||
* http://www.apache.org/licenses/LICENSE-2.0 | ||||||
* | ||||||
* Unless required by applicable law or agreed to in writing, software | ||||||
* distributed under the License is distributed on an "AS IS" BASIS, | ||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||
* See the License for the specific language governing permissions and | ||||||
* limitations under the License. | ||||||
*/ | ||||||
package com.optimizely.ab.internal; | ||||||
|
||||||
import com.optimizely.ab.annotations.VisibleForTesting; | ||||||
|
||||||
import java.util.*; | ||||||
|
||||||
public class DefaultLRUCache<T> implements LRUCache<T> { | ||||||
|
||||||
private final Object lock = new Object(); | ||||||
|
||||||
private Integer maxSize; | ||||||
|
||||||
private Long timeoutMillis; | ||||||
|
||||||
final LinkedList<ItemWrapper> linkedList = new LinkedList<>(); | ||||||
|
||||||
final Map<String, ItemWrapper> hashMap = new HashMap<>(); | ||||||
|
||||||
public DefaultLRUCache() { | ||||||
this.maxSize = DEFAULT_MAX_SIZE; | ||||||
this.timeoutMillis = (long) (DEFAULT_TIMEOUT_SECONDS * 1000); | ||||||
} | ||||||
|
||||||
public void setMaxSize(Integer size) { | ||||||
Integer sizeToSet = size; | ||||||
if (size < 0 ) { | ||||||
sizeToSet = DEFAULT_MAX_SIZE; | ||||||
} | ||||||
|
||||||
synchronized (lock) { | ||||||
if (linkedList.size() > sizeToSet) { | ||||||
// If cache is bigger than the new maxSize, remove additional items from the end. | ||||||
int itemsToTrim = linkedList.size() - sizeToSet; | ||||||
for (int i = 0; i < itemsToTrim; i++) { | ||||||
ItemWrapper item = linkedList.removeLast(); | ||||||
hashMap.remove(item.key); | ||||||
} | ||||||
} | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need this support for dynamic cache size changes? We can allow them cache size setting only when app starts. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I changed it to only allow when maxSize passed in is less than the current size of the cache. |
||||||
this.maxSize = sizeToSet; | ||||||
} | ||||||
} | ||||||
|
||||||
public void setTimeout(Long timeoutSeconds) { | ||||||
this.timeoutMillis = timeoutSeconds * 1000; | ||||||
} | ||||||
|
||||||
public void save(String key, T value) { | ||||||
if (maxSize == 0) { | ||||||
// Cache is disabled when maxSize = 0 | ||||||
return; | ||||||
} | ||||||
synchronized (lock) { | ||||||
ItemWrapper item = new ItemWrapper(key, value); | ||||||
|
||||||
if (hashMap.containsKey(key)) { | ||||||
ItemWrapper oldItem = hashMap.get(key); | ||||||
linkedList.remove(oldItem); | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Isn't this linear search? It may be a factor in large cache. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a good catch. Because i was passing the item itself and it uses a doubly linked list, i thought the item should have references to its neighbors and should be able to cleanly remove itself in O(1) but it looks like the actual node with references is managed internally and |
||||||
linkedList.addFirst(item); | ||||||
hashMap.replace(key, item); | ||||||
return; | ||||||
} | ||||||
|
||||||
if (linkedList.size() == maxSize) { | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changed the whole implementation. It does not apply anymore. |
||||||
ItemWrapper removedItem = linkedList.removeLast(); | ||||||
hashMap.remove(removedItem.key); | ||||||
} | ||||||
|
||||||
linkedList.addFirst(item); | ||||||
hashMap.put(key, item); | ||||||
} | ||||||
} | ||||||
|
||||||
public T lookup(String key) { | ||||||
if (maxSize == 0) { | ||||||
// Cache is disabled when maxSize = 0 | ||||||
return null; | ||||||
} | ||||||
|
||||||
synchronized (lock) { | ||||||
if (hashMap.containsKey(key)) { | ||||||
ItemWrapper item = hashMap.get(key); | ||||||
Long nowMs = new Date().getTime(); | ||||||
|
||||||
// ttl = 0 means items never expire. | ||||||
if (timeoutMillis == 0 || (nowMs - item.timestamp < timeoutMillis)) { | ||||||
linkedList.remove(item); | ||||||
linkedList.addFirst(item); | ||||||
return item.value; | ||||||
} | ||||||
|
||||||
// If item is stale, remove it. | ||||||
linkedList.remove(item); | ||||||
hashMap.remove(item.key); | ||||||
} | ||||||
return null; | ||||||
} | ||||||
} | ||||||
|
||||||
public T peek(String key) { | ||||||
if (maxSize == 0) { | ||||||
// Cache is disabled when maxSize = 0 | ||||||
return null; | ||||||
} | ||||||
|
||||||
if (hashMap.containsKey(key)) { | ||||||
return hashMap.get(key).value; | ||||||
} | ||||||
|
||||||
return null; | ||||||
} | ||||||
|
||||||
public void reset() { | ||||||
synchronized (lock) { | ||||||
linkedList.clear(); | ||||||
hashMap.clear(); | ||||||
} | ||||||
} | ||||||
|
||||||
@VisibleForTesting | ||||||
class ItemWrapper { | ||||||
public String key; | ||||||
public T value; | ||||||
public Long timestamp; | ||||||
|
||||||
public ItemWrapper(String key, T value) { | ||||||
this.key = key; | ||||||
this.value = value; | ||||||
this.timestamp = new Date().getTime(); | ||||||
} | ||||||
} | ||||||
} |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,28 @@ | ||||||
/** | ||||||
* | ||||||
* Copyright 2022, Optimizely | ||||||
* | ||||||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||||||
* you may not use this file except in compliance with the License. | ||||||
* You may obtain a copy of the License at | ||||||
* | ||||||
* http://www.apache.org/licenses/LICENSE-2.0 | ||||||
* | ||||||
* Unless required by applicable law or agreed to in writing, software | ||||||
* distributed under the License is distributed on an "AS IS" BASIS, | ||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||
* See the License for the specific language governing permissions and | ||||||
* limitations under the License. | ||||||
*/ | ||||||
package com.optimizely.ab.internal; | ||||||
|
||||||
public interface LRUCache<T> { | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
This interface can be for generic caches, not only for LRUCache. Not sure about the best name to avoid conflicts., but "LRU" should be dropped. I think this is for internal use (not for public custom segments cache). If this interface is for public, I have a few other suggestions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changing it to Cache for now as per your suggestion. We can discuss this further in subsequent PRs when things become more clear. |
||||||
int DEFAULT_MAX_SIZE = 10000; | ||||||
int DEFAULT_TIMEOUT_SECONDS = 600; | ||||||
void save(String key, T value); | ||||||
T lookup(String key); | ||||||
T peek(String key); | ||||||
void reset(); | ||||||
void setMaxSize(Integer size); | ||||||
void setTimeout(Long timeoutSeconds); | ||||||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,171 @@ | ||
/** | ||
* | ||
* Copyright 2022, Optimizely | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package com.optimizely.ab.internal; | ||
|
||
import org.junit.Test; | ||
|
||
import java.util.Arrays; | ||
import java.util.List; | ||
|
||
import static org.junit.Assert.*; | ||
|
||
public class DefaultLRUCacheTest { | ||
|
||
@Test | ||
public void createSaveAndLookupOneItem() { | ||
LRUCache<String> cache = new DefaultLRUCache<>(); | ||
assertNull(cache.lookup("key1")); | ||
cache.save("key1", "value1"); | ||
assertEquals("value1", cache.lookup("key1")); | ||
} | ||
|
||
@Test | ||
public void saveAndLookupMultipleItems() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This test looks good. It will be good to have the same ordering switch tests for save ops as well. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done! |
||
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>(); | ||
|
||
cache.save("user1", Arrays.asList("segment1", "segment2")); | ||
cache.save("user2", Arrays.asList("segment3", "segment4")); | ||
cache.save("user3", Arrays.asList("segment5", "segment6")); | ||
|
||
assertEquals("user3", cache.linkedList.get(0).key); | ||
assertEquals("user2", cache.linkedList.get(1).key); | ||
assertEquals("user1", cache.linkedList.get(2).key); | ||
|
||
assertEquals(Arrays.asList("segment1", "segment2"), cache.lookup("user1")); | ||
|
||
// Lookup should move user1 to top of the list and push down others. | ||
assertEquals("user1", cache.linkedList.get(0).key); | ||
assertEquals("user3", cache.linkedList.get(1).key); | ||
assertEquals("user2", cache.linkedList.get(2).key); | ||
|
||
assertEquals(Arrays.asList("segment3", "segment4"), cache.lookup("user2")); | ||
|
||
// Lookup should move user2 to top of the list and push down others. | ||
assertEquals("user2", cache.linkedList.get(0).key); | ||
assertEquals("user1", cache.linkedList.get(1).key); | ||
assertEquals("user3", cache.linkedList.get(2).key); | ||
|
||
assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3")); | ||
|
||
// Lookup should move user3 to top of the list and push down others. | ||
assertEquals("user3", cache.linkedList.get(0).key); | ||
assertEquals("user2", cache.linkedList.get(1).key); | ||
assertEquals("user1", cache.linkedList.get(2).key); | ||
} | ||
|
||
@Test | ||
public void whenCacheIsDisabled() { | ||
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>(); | ||
cache.setMaxSize(0); | ||
cache.save("user1", Arrays.asList("segment1", "segment2")); | ||
cache.save("user2", Arrays.asList("segment3", "segment4")); | ||
cache.save("user3", Arrays.asList("segment5", "segment6")); | ||
|
||
assertNull(cache.lookup("user1")); | ||
assertNull(cache.lookup("user2")); | ||
assertNull(cache.lookup("user3")); | ||
} | ||
|
||
@Test | ||
public void whenItemsExpire() throws InterruptedException { | ||
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>(); | ||
cache.setTimeout(1L); | ||
cache.save("user1", Arrays.asList("segment1", "segment2")); | ||
assertEquals(Arrays.asList("segment1", "segment2"), cache.lookup("user1")); | ||
assertEquals(1, cache.linkedList.size()); | ||
assertEquals(1, cache.hashMap.size()); | ||
Thread.sleep(1000); | ||
assertNull(cache.lookup("user1")); | ||
assertEquals(0, cache.linkedList.size()); | ||
assertEquals(0, cache.hashMap.size()); | ||
} | ||
|
||
@Test | ||
public void whenCacheReachesMaxSize() { | ||
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>(); | ||
cache.setMaxSize(2); | ||
|
||
cache.save("user1", Arrays.asList("segment1", "segment2")); | ||
cache.save("user2", Arrays.asList("segment3", "segment4")); | ||
cache.save("user3", Arrays.asList("segment5", "segment6")); | ||
|
||
assertEquals(2, cache.linkedList.size()); | ||
assertEquals(2, cache.hashMap.size()); | ||
|
||
assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3")); | ||
assertEquals(Arrays.asList("segment3", "segment4"), cache.lookup("user2")); | ||
assertNull(cache.lookup("user1")); | ||
} | ||
|
||
@Test | ||
public void whenMaxSizeIsReducedInBetween() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can remove this test There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Modified the test to check the changed implementation |
||
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>(); | ||
cache.save("user1", Arrays.asList("segment1", "segment2")); | ||
cache.save("user2", Arrays.asList("segment3", "segment4")); | ||
cache.save("user3", Arrays.asList("segment5", "segment6")); | ||
|
||
assertEquals(Arrays.asList("segment1", "segment2"), cache.lookup("user1")); | ||
assertEquals(Arrays.asList("segment3", "segment4"), cache.lookup("user2")); | ||
assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3")); | ||
|
||
assertEquals(3, cache.linkedList.size()); | ||
assertEquals(3, cache.hashMap.size()); | ||
|
||
cache.setMaxSize(1); | ||
|
||
assertEquals(Arrays.asList("segment5", "segment6"), cache.lookup("user3")); | ||
assertNull(cache.lookup("user1")); | ||
assertNull(cache.lookup("user2")); | ||
|
||
assertEquals(1, cache.linkedList.size()); | ||
assertEquals(1, cache.hashMap.size()); | ||
} | ||
|
||
@Test | ||
public void saveAndPeekItems() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A test for reset()? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done! |
||
DefaultLRUCache<List<String>> cache = new DefaultLRUCache<>(); | ||
|
||
cache.save("user1", Arrays.asList("segment1", "segment2")); | ||
cache.save("user2", Arrays.asList("segment3", "segment4")); | ||
cache.save("user3", Arrays.asList("segment5", "segment6")); | ||
|
||
assertEquals("user3", cache.linkedList.get(0).key); | ||
assertEquals("user2", cache.linkedList.get(1).key); | ||
assertEquals("user1", cache.linkedList.get(2).key); | ||
|
||
assertEquals(Arrays.asList("segment1", "segment2"), cache.peek("user1")); | ||
|
||
// Peek should not alter the order of array | ||
assertEquals("user3", cache.linkedList.get(0).key); | ||
assertEquals("user2", cache.linkedList.get(1).key); | ||
assertEquals("user1", cache.linkedList.get(2).key); | ||
|
||
assertEquals(Arrays.asList("segment3", "segment4"), cache.peek("user2")); | ||
|
||
// Peek should not alter the order of array | ||
assertEquals("user3", cache.linkedList.get(0).key); | ||
assertEquals("user2", cache.linkedList.get(1).key); | ||
assertEquals("user1", cache.linkedList.get(2).key); | ||
|
||
assertEquals(Arrays.asList("segment5", "segment6"), cache.peek("user3")); | ||
|
||
// Peek should not alter the order of array | ||
assertEquals("user3", cache.linkedList.get(0).key); | ||
assertEquals("user2", cache.linkedList.get(1).key); | ||
assertEquals("user1", cache.linkedList.get(2).key); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
either way is fine, but let's use "sizeToSet = 0" for all SDKs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! Defaulting to zero now when max size is negative