Skip to content

Commit 797c387

Browse files
matthewelwellgazreese
matthewelwell
and
gazreese
authored
Feature/real time flags (#28)
* Feature/real time flags (#5) * Compiles and test, need to add some tests then get it into a sample app * Should be testing but not getting the errors through Fuse so can't run into the logic * Probably gone as far as I can with 2.x fuel, let's try the 3.x * Move to Retrofit - seems to be going well so far, test runs * Tidying up, setTrait test not working * All the tests are passing so will finish the retrofit migration * Updated some of the logic and added setTraits * Checkpoint commit before trying generic converter * Generics working fine * All passing for flags and such with the new generic caching * Mostly swapped to Retrofit, now need to do the analytics * Analytics now over to retrofit * Add caching for the getFlags endpoint * Get rid of the last of Fuel * Another clear-out and all working fine on the tests * Now using Retrofit cache, remove the old stuff * Now just using HTTP caching * Delete the old caching logic * Finishing off, should be done for defaults and caching * Remove unneeded todo * Remove some more code * Still just playing around with it * Move cache configuration to its own data class * Tidy up the cache config and the tests * Update the comments * Now covers the caching tests * Tidy up some more of the tests * Some more tidying up * Default to caching disabled * Last few PR comments * Split the read and write timeout for HTTP * Initial basic implementation, let's try to get things hooked up to the server * Seems to be generally working * Checkpoint commit, seems to be generally working now just need to get the flags on update * Checkpoint commit before making the changes OK'd by Matthew to move the update clock into the event service * Ensure that the event source just reconnects if it loses the connection * Events and timers now all hooked-up and working in the manual integration test * Got the integration test working * Tidy everything up and move sensitive data to environment variables * Add a new test to cover the event stream going through a reconnect cycle * Added test for the live stream of flags, tidied up the imports and various thing, changed the logic a bit for when we need to do updates from events * Update FlagsmithClient/src/test/java/com/flagsmith/RealTimeUpdatesIntegrationTests.kt Co-authored-by: Matthew Elwell <[email protected]> * Update environment variables in the github actions * Add some error checking on the environment variables so it's a bit more obvious what's going on if we don't configure properly * Noddy change to get the tests to run again * Try printing out unsuccessful responses in the integration tests * Push more more non-empty checks * Feature/real time flags (#5) (#21) * Feature/real time flags (#5) * Compiles and test, need to add some tests then get it into a sample app * Should be testing but not getting the errors through Fuse so can't run into the logic * Probably gone as far as I can with 2.x fuel, let's try the 3.x * Move to Retrofit - seems to be going well so far, test runs * Tidying up, setTrait test not working * All the tests are passing so will finish the retrofit migration * Updated some of the logic and added setTraits * Checkpoint commit before trying generic converter * Generics working fine * All passing for flags and such with the new generic caching * Mostly swapped to Retrofit, now need to do the analytics * Analytics now over to retrofit * Add caching for the getFlags endpoint * Get rid of the last of Fuel * Another clear-out and all working fine on the tests * Now using Retrofit cache, remove the old stuff * Now just using HTTP caching * Delete the old caching logic * Finishing off, should be done for defaults and caching * Remove unneeded todo * Remove some more code * Still just playing around with it * Move cache configuration to its own data class * Tidy up the cache config and the tests * Update the comments * Now covers the caching tests * Tidy up some more of the tests * Some more tidying up * Default to caching disabled * Last few PR comments * Split the read and write timeout for HTTP * Initial basic implementation, let's try to get things hooked up to the server * Seems to be generally working * Checkpoint commit, seems to be generally working now just need to get the flags on update * Checkpoint commit before making the changes OK'd by Matthew to move the update clock into the event service * Ensure that the event source just reconnects if it loses the connection * Events and timers now all hooked-up and working in the manual integration test * Got the integration test working * Tidy everything up and move sensitive data to environment variables * Add a new test to cover the event stream going through a reconnect cycle * Added test for the live stream of flags, tidied up the imports and various thing, changed the logic a bit for when we need to do updates from events * Update FlagsmithClient/src/test/java/com/flagsmith/RealTimeUpdatesIntegrationTests.kt Co-authored-by: Matthew Elwell <[email protected]> * Update environment variables in the github actions * Add some error checking on the environment variables so it's a bit more obvious what's going on if we don't configure properly * Noddy change to get the tests to run again * Try printing out unsuccessful responses in the integration tests * Push more more non-empty checks --------- Co-authored-by: Matthew Elwell <[email protected]> * Split the actions into push and pull-request versions * Remove the old script * Update the timeouts on the integration tests and make them a little more robust * Fix up some merge issues * Use the flow value directly in the unit tests and simplify the flow updates in the SDK so they're always passive * Quick update to see if we're getting an error back when pushing the flag * Bump to 4c runner * Move to Strings as the server prefers them. Increase the timeouts and make the check loops more CPU friendly * Updates to cover most of the review comments * Make some of the Flagsmith class external again, move the test-only portions of the API into the test folder * Suppress a case warning in the test code * Remove integration test env vars now that we know that they passed with them included * Exclude integration tests in the Kover run * Disable coverage on PR workflow * Remove some duplicated logic * Update construction of event source URL * Add some constants and rearrange a bit of code * Return the env key interceptor to its original position * Various updates to the integration tests to cover Matthew's comments * Tidied up the integration tests, also added a bit more to one of the timeouts as it was taking a while for the realtime updates to come though * Tidied some of the println() statements into the asserts, commented on the reasoning for some of the timeouts and delays --------- Co-authored-by: Gareth Reese <[email protected]> Co-authored-by: Gareth Reese <[email protected]>
1 parent 9aab12d commit 797c387

15 files changed

+564
-35
lines changed

Diff for: ‎.github/workflows/verify.yml renamed to ‎.github/workflows/verify-pull-request.yml

+3-8
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,19 @@
1-
name: Verify
1+
name: Verify pull request to main
22

33
on:
4-
push:
5-
branches:
6-
- "main"
74
pull_request:
85
branches:
96
- "main"
107

118
jobs:
129
test:
1310
name: Run Unit Tests
14-
runs-on: ubuntu-latest
11+
runs-on: General-Purpose-4c-Runner
1512

1613
steps:
1714
- uses: actions/checkout@v3
1815
- name: Preconfigure gradle
1916
uses: ./.github/actions/prepare-gradle
2017

2118
- name: Run unit tests
22-
run: ./gradlew check -x koverVerify
23-
- name: Verify code coverage level
24-
run: ./gradlew koverVerify
19+
run: ./gradlew check -x koverVerify -P excludeIntegrationTests

Diff for: ‎.github/workflows/verify-push-main.yml

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: Verify on push to main
2+
3+
on:
4+
push:
5+
branches:
6+
- "main"
7+
8+
jobs:
9+
test:
10+
name: Run Unit Tests
11+
runs-on: General-Purpose-4c-Runner
12+
13+
env:
14+
INTEGRATION_TESTS_ENVIRONMENT_KEY: NTtWcerSBE5yj7a5optMSk
15+
INTEGRATION_TESTS_FEATURE_NAME: integration-test-feature
16+
INTEGRATION_TESTS_FEATURE_STATE_ID: 321715
17+
INTEGRATION_TESTS_API_TOKEN: ${{ secrets.INTEGRATION_TESTS_API_TOKEN }}
18+
19+
steps:
20+
- uses: actions/checkout@v3
21+
- name: Preconfigure gradle
22+
uses: ./.github/actions/prepare-gradle
23+
24+
- name: Run unit tests
25+
run: ./gradlew check -x koverVerify
26+
- name: Verify code coverage level
27+
run: ./gradlew koverVerify

Diff for: ‎FlagsmithClient/build.gradle.kts

+16-1
Original file line numberDiff line numberDiff line change
@@ -68,11 +68,16 @@ android {
6868

6969
dependencies {
7070
implementation("com.google.code.gson:gson:2.10")
71+
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4")
7172

7273
// HTTP Client
7374
implementation("com.squareup.retrofit2:retrofit:2.9.0")
7475
implementation("com.squareup.retrofit2:converter-gson:2.9.0")
7576

77+
// Server Sent Events
78+
implementation("com.squareup.okhttp3:okhttp-sse:4.11.0")
79+
testImplementation("com.squareup.okhttp3:okhttp-sse:4.11.0")
80+
7681
testImplementation("junit:junit:4.13.2")
7782
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.6.4")
7883
testImplementation("org.mock-server:mockserver-netty-no-dependencies:5.14.0")
@@ -84,7 +89,8 @@ dependencies {
8489
kover {
8590
filters {
8691
classes {
87-
excludes += listOf("${android.namespace}.BuildConfig")
92+
excludes += listOf("${android.namespace}.BuildConfig", "com.flagsmith.test.*")
93+
8894
}
8995
}
9096
verify {
@@ -101,6 +107,15 @@ kover {
101107
}
102108

103109
tasks.withType(Test::class) {
110+
// if the excludeIntegrationTests property is set
111+
// then exclude tests with IntegrationTest in the name
112+
// i.e. `gradle :FlagsmithClient:testDebugUnitTest --tests "com.flagsmith.*" -P excludeIntegrationTests`
113+
if (project.hasProperty("excludeIntegrationTests")) {
114+
exclude {
115+
it.name.contains("IntegrationTest")
116+
}
117+
}
118+
104119
testLogging {
105120
events(
106121
TestLogEvent.FAILED,

Diff for: ‎FlagsmithClient/src/main/java/com/flagsmith/Flagsmith.kt

+69-11
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,19 @@
11
package com.flagsmith
22

33
import android.content.Context
4-
import com.flagsmith.entities.*
4+
import android.util.Log
5+
import com.flagsmith.entities.Flag
6+
import com.flagsmith.entities.Identity
7+
import com.flagsmith.entities.IdentityFlagsAndTraits
8+
import com.flagsmith.entities.Trait
9+
import com.flagsmith.entities.TraitWithIdentity
510
import com.flagsmith.internal.FlagsmithAnalytics
11+
import com.flagsmith.internal.FlagsmithEventService
12+
import com.flagsmith.internal.FlagsmithEventTimeTracker
613
import com.flagsmith.internal.FlagsmithRetrofitService
714
import com.flagsmith.internal.enqueueWithResult
15+
import kotlinx.coroutines.flow.MutableStateFlow
16+
import okhttp3.Cache
817

918
/**
1019
* Flagsmith
@@ -21,27 +30,69 @@ import com.flagsmith.internal.enqueueWithResult
2130
class Flagsmith constructor(
2231
private val environmentKey: String,
2332
private val baseUrl: String = "https://edge.api.flagsmith.com/api/v1/",
33+
private val eventSourceBaseUrl: String = "https://realtime.flagsmith.com/",
2434
private val context: Context? = null,
2535
private val enableAnalytics: Boolean = DEFAULT_ENABLE_ANALYTICS,
36+
private val enableRealtimeUpdates: Boolean = false,
2637
private val analyticsFlushPeriod: Int = DEFAULT_ANALYTICS_FLUSH_PERIOD_SECONDS,
2738
private val cacheConfig: FlagsmithCacheConfig = FlagsmithCacheConfig(),
2839
private val defaultFlags: List<Flag> = emptyList(),
2940
private val requestTimeoutSeconds: Long = 4L,
3041
private val readTimeoutSeconds: Long = 6L,
31-
private val writeTimeoutSeconds: Long = 6L
32-
) {
33-
private val retrofit: FlagsmithRetrofitService = FlagsmithRetrofitService.create(
34-
baseUrl = baseUrl, environmentKey = environmentKey, context = context, cacheConfig = cacheConfig,
35-
requestTimeoutSeconds = requestTimeoutSeconds, readTimeoutSeconds = readTimeoutSeconds, writeTimeoutSeconds = writeTimeoutSeconds)
42+
private val writeTimeoutSeconds: Long = 6L,
43+
override var lastFlagFetchTime: Double = 0.0 // from FlagsmithEventTimeTracker
44+
) : FlagsmithEventTimeTracker {
45+
private lateinit var retrofit: FlagsmithRetrofitService
46+
private var cache: Cache? = null
47+
private var lastUsedIdentity: String? = null
3648
private val analytics: FlagsmithAnalytics? =
3749
if (!enableAnalytics) null
3850
else if (context != null) FlagsmithAnalytics(context, retrofit, analyticsFlushPeriod)
3951
else throw IllegalArgumentException("Flagsmith requires a context to use the analytics feature")
4052

53+
private val eventService: FlagsmithEventService? =
54+
if (!enableRealtimeUpdates) null
55+
else FlagsmithEventService(eventSourceBaseUrl = eventSourceBaseUrl, environmentKey = environmentKey) { event ->
56+
if (event.isSuccess) {
57+
lastEventUpdate = event.getOrNull()?.updatedAt ?: lastEventUpdate
58+
59+
// Check whether this event is anything new
60+
if (lastEventUpdate > lastFlagFetchTime) {
61+
// First evict the cache otherwise we'll be stuck with the old values
62+
cache?.evictAll()
63+
lastFlagFetchTime = lastEventUpdate
64+
65+
// Now we can get the new values, which will automatically be emitted to the flagUpdateFlow
66+
getFeatureFlags(lastUsedIdentity) { res ->
67+
if (res.isFailure) {
68+
Log.e(
69+
"Flagsmith",
70+
"Error getting flags in SSE stream: ${res.exceptionOrNull()}"
71+
)
72+
} else {
73+
Log.i("Flagsmith", "Got flags due to SSE event: $event")
74+
}
75+
}
76+
}
77+
}
78+
}
79+
80+
// The last time we got an event from the SSE stream or via the API
81+
private var lastEventUpdate: Double = 0.0
82+
83+
/** Stream of flag updates from the SSE stream if enabled */
84+
val flagUpdateFlow = MutableStateFlow<List<Flag>>(listOf())
85+
4186
init {
4287
if (cacheConfig.enableCache && context == null) {
4388
throw IllegalArgumentException("Flagsmith requires a context to use the cache feature")
4489
}
90+
val pair = FlagsmithRetrofitService.create<FlagsmithRetrofitService>(
91+
baseUrl = baseUrl, environmentKey = environmentKey, context = context, cacheConfig = cacheConfig,
92+
requestTimeoutSeconds = requestTimeoutSeconds, readTimeoutSeconds = readTimeoutSeconds,
93+
writeTimeoutSeconds = writeTimeoutSeconds, timeTracker = this, klass = FlagsmithRetrofitService::class.java)
94+
retrofit = pair.first
95+
cache = pair.second
4596
}
4697

4798
companion object {
@@ -50,12 +101,19 @@ class Flagsmith constructor(
50101
}
51102

52103
fun getFeatureFlags(identity: String? = null, result: (Result<List<Flag>>) -> Unit) {
104+
// Save the last used identity as we'll refresh with this if we get update events
105+
lastUsedIdentity = identity
106+
53107
if (identity != null) {
54108
retrofit.getIdentityFlagsAndTraits(identity).enqueueWithResult { res ->
109+
flagUpdateFlow.tryEmit(res.getOrNull()?.flags ?: emptyList())
55110
result(res.map { it.flags })
56111
}
57112
} else {
58-
retrofit.getFlags().enqueueWithResult(defaults = defaultFlags, result = result)
113+
retrofit.getFlags().enqueueWithResult(defaults = defaultFlags) { res ->
114+
flagUpdateFlow.tryEmit(res.getOrNull() ?: emptyList())
115+
result(res)
116+
}
59117
}
60118
}
61119

@@ -78,18 +136,19 @@ class Flagsmith constructor(
78136
fun getTrait(id: String, identity: String, result: (Result<Trait?>) -> Unit) =
79137
retrofit.getIdentityFlagsAndTraits(identity).enqueueWithResult { res ->
80138
result(res.map { value -> value.traits.find { it.key == id } })
81-
}
139+
}.also { lastUsedIdentity = identity }
82140

83141
fun getTraits(identity: String, result: (Result<List<Trait>>) -> Unit) =
84142
retrofit.getIdentityFlagsAndTraits(identity).enqueueWithResult { res ->
85143
result(res.map { it.traits })
86-
}
144+
}.also { lastUsedIdentity = identity }
87145

88146
fun setTrait(trait: Trait, identity: String, result: (Result<TraitWithIdentity>) -> Unit) =
89147
retrofit.postTraits(TraitWithIdentity(trait.key, trait.value, Identity(identity))).enqueueWithResult(result = result)
90148

91149
fun getIdentity(identity: String, result: (Result<IdentityFlagsAndTraits>) -> Unit) =
92150
retrofit.getIdentityFlagsAndTraits(identity).enqueueWithResult(defaults = null, result = result)
151+
.also { lastUsedIdentity = identity }
93152

94153
private fun getFeatureFlag(
95154
featureId: String,
@@ -101,6 +160,5 @@ class Flagsmith constructor(
101160
analytics?.trackEvent(featureId)
102161
foundFlag
103162
})
104-
}
105-
163+
}.also { lastUsedIdentity = identity }
106164
}

Diff for: ‎FlagsmithClient/src/main/java/com/flagsmith/entities/FlagEvent.kt

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
package com.flagsmith.entities
2+
3+
import com.google.gson.annotations.SerializedName
4+
5+
internal data class FlagEvent (
6+
@SerializedName(value = "updated_at") val updatedAt: Double
7+
)

Diff for: ‎FlagsmithClient/src/main/java/com/flagsmith/entities/Trait.kt

-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@ package com.flagsmith.entities
22

33

44
import com.google.gson.annotations.SerializedName
5-
import java.io.Reader
65

76
data class Trait(
87
val identifier: String? = null,

Diff for: ‎FlagsmithClient/src/main/java/com/flagsmith/internal/FlagsmithEventService.kt

+87
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
package com.flagsmith.internal
2+
3+
import android.util.Log
4+
import com.flagsmith.entities.FlagEvent
5+
import com.google.gson.Gson
6+
import kotlinx.coroutines.flow.MutableStateFlow
7+
import okhttp3.OkHttpClient
8+
import okhttp3.Request
9+
import okhttp3.Response
10+
import okhttp3.sse.EventSource
11+
import okhttp3.sse.EventSourceListener
12+
import okhttp3.sse.EventSources
13+
import java.util.concurrent.TimeUnit
14+
15+
internal class FlagsmithEventService constructor(
16+
private val eventSourceBaseUrl: String?,
17+
private val environmentKey: String,
18+
private val updates: (Result<FlagEvent>) -> Unit
19+
) {
20+
private val sseClient = OkHttpClient.Builder()
21+
.addInterceptor(FlagsmithRetrofitService.envKeyInterceptor(environmentKey))
22+
.connectTimeout(6, TimeUnit.SECONDS)
23+
.readTimeout(10, TimeUnit.MINUTES)
24+
.writeTimeout(10, TimeUnit.MINUTES)
25+
.build()
26+
27+
private val completeEventSourceUrl: String = eventSourceBaseUrl + "sse/environments/" + environmentKey + "/stream"
28+
29+
private val sseRequest = Request.Builder()
30+
.url(completeEventSourceUrl)
31+
.header("Accept", "application/json")
32+
.addHeader("Accept", "text/event-stream")
33+
.build()
34+
35+
private var currentEventSource: EventSource? = null
36+
37+
var sseEventsFlow = MutableStateFlow(FlagEvent(updatedAt = 0.0))
38+
private set
39+
40+
private val sseEventSourceListener = object : EventSourceListener() {
41+
override fun onClosed(eventSource: EventSource) {
42+
super.onClosed(eventSource)
43+
Log.d(TAG, "onClosed: $eventSource")
44+
45+
// This isn't uncommon and is the nature of HTTP requests, so just reconnect
46+
initEventSource()
47+
}
48+
49+
override fun onEvent(eventSource: EventSource, id: String?, type: String?, data: String) {
50+
super.onEvent(eventSource, id, type, data)
51+
Log.d(TAG, "onEvent: $data")
52+
if (type != null && type == "environment_updated" && data.isNotEmpty()) {
53+
val flagEvent = Gson().fromJson(data, FlagEvent::class.java)
54+
sseEventsFlow.tryEmit(flagEvent)
55+
updates(Result.success(flagEvent))
56+
}
57+
}
58+
59+
override fun onFailure(eventSource: EventSource, t: Throwable?, response: Response?) {
60+
super.onFailure(eventSource, t, response)
61+
Log.d(TAG, "onFailure: ${t?.message}")
62+
if (t != null)
63+
updates(Result.failure(t))
64+
else
65+
updates(Result.failure(Throwable("Unknown error")))
66+
}
67+
68+
override fun onOpen(eventSource: EventSource, response: Response) {
69+
super.onOpen(eventSource, response)
70+
Log.d(TAG, "onOpen: $eventSource")
71+
}
72+
}
73+
74+
init {
75+
initEventSource()
76+
}
77+
78+
private fun initEventSource() {
79+
currentEventSource?.cancel()
80+
currentEventSource = EventSources.createFactory(sseClient)
81+
.newEventSource(request = sseRequest, listener = sseEventSourceListener)
82+
}
83+
84+
companion object {
85+
private const val TAG = "FlagsmithEventService"
86+
}
87+
}

Diff for: ‎FlagsmithClient/src/main/java/com/flagsmith/internal/FlagsmithEventTimeTracker.kt

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
package com.flagsmith.internal
2+
3+
/** Internal interface for objects that track the last time the flags were fetched from the server */
4+
interface FlagsmithEventTimeTracker {
5+
/** The last time the flags were fetched from the server, as a Unix epoch */
6+
var lastFlagFetchTime: Double
7+
}

0 commit comments

Comments
 (0)