-
Notifications
You must be signed in to change notification settings - Fork 74
FFL-1621 Fix: set currentState in a blocking call #3068
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
🎯 Code Coverage 🔗 Commit SHA: 9d4291c | Docs | Datadog PR Page | Was this helpful? Give us feedback! |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #3068 +/- ##
===========================================
+ Coverage 71.40% 71.50% +0.10%
===========================================
Files 881 881
Lines 32460 32471 +11
Branches 5466 5475 +9
===========================================
+ Hits 23178 23218 +40
+ Misses 7729 7717 -12
+ Partials 1553 1536 -17
🚀 New features to boost your workflow:
|
| currentState = newState | ||
| subscription.notifyListeners { | ||
| onStateChanged(newState) | ||
| executorService.executeSafe( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
enqueue safe executions of the listeners instead of safe execution of the queue. Now listeners cannot not crash the rest
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we move synchronized protection inside DDCoreSubscriptionImpl?, it seems to me the job of DDCoreSubscriptionImpl to guarantee the thread-safety no matter where it is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The DDCoreSubscription ensures atomicity wrt modifying the subscriber list during a notification loop. Our synchronized here is layering atomicity linking the notification mechanics of DDCoreSubscription with mutation of the currentState. This coupling is a specific use case of the DDCoreSubscription or another layer on top.
DDCoreSubscription is still susceptible to listeners crashing the notification loop. We could put a try-catch there, but it might be better to keep the onus on the caller (which easy to do via the block argument passed to notifyListeners.
wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need do executorService.executeSafe instead of simply calling onStateChanged(newState)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
originally, it was to ensure fairness of state changes - which we've actually lost here with the synchronized block (switched to a ReentrantLock now)
The executor lets us run the subscription consumer off-thread and not block the caller to setEvaluationContext on potential long running consumers. Would it be better to leave the thread mgmt to the caller here? Would the caller need to run their consumer specifically in the UI thread (or some other specific thread?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be better to leave the thread mgmt to the caller here?
I would say yes, it is up to the caller to do it unless we are picky about listener performance and for us it is critical for it to be fast. And then we can wrap simply onStateChanged calls in try-catch if we want to continue work after consumer threw exception. But then this exception will be handled, so maybe it is actually better to let it propagate and crash the app (since it is exception from the customer code).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am in favour of the more straightforward approach here where we put onus for threading and exception handling on the user. Fail-fast prevents us from hiding exceptions from the dev and clear docs should point them in the right direction.
...dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/FlagsStateManager.kt
Outdated
Show resolved
Hide resolved
typotter
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @ambushwork ptal.
...dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/FlagsStateManager.kt
Outdated
Show resolved
Hide resolved
| currentState = newState | ||
| subscription.notifyListeners { | ||
| onStateChanged(newState) | ||
| executorService.executeSafe( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The DDCoreSubscription ensures atomicity wrt modifying the subscriber list during a notification loop. Our synchronized here is layering atomicity linking the notification mechanics of DDCoreSubscription with mutation of the currentState. This coupling is a specific use case of the DDCoreSubscription or another layer on top.
DDCoreSubscription is still susceptible to listeners crashing the notification loop. We could put a try-catch there, but it might be better to keep the onus on the caller (which easy to do via the block argument passed to notifyListeners.
wdyt?
| * The fair parameter ensures that threads acquire the lock in the order they requested it. | ||
| */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
threads acquire the lock in the order they requested it
Does it really matter actually? I'm wondering if it is better to use ReadWriteLock since probably the number of read operations will dominate over the number of write operations, or is it a wrong assumption?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. We could use a ReentrantReadWriteLock to allow reads and still provide ordering for writes (to maintain the ordering of state change notifications sent to listeners).
In practice, I think the direct reads will not be that high, as the use case will typically be to, once on app load:
- check the current state
- listen to state changes (typically, to then load flag-critical parts of the app when the provider is READY)
At any rate, it makes sense to allow the current state to be read concurrently with notification operations. Thanks!
| currentState = newState | ||
| subscription.notifyListeners { | ||
| onStateChanged(newState) | ||
| executorService.executeSafe( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be better to leave the thread mgmt to the caller here?
I would say yes, it is up to the caller to do it unless we are picky about listener performance and for us it is critical for it to be fast. And then we can wrap simply onStateChanged calls in try-catch if we want to continue work after consumer threw exception. But then this exception will be handled, so maybe it is actually better to let it propagate and crash the app (since it is exception from the customer code).
typotter
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy new year!
Addressed the executor and lock suggestions. PTAL/
| * The fair parameter ensures that threads acquire the lock in the order they requested it. | ||
| */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. We could use a ReentrantReadWriteLock to allow reads and still provide ordering for writes (to maintain the ordering of state change notifications sent to listeners).
In practice, I think the direct reads will not be that high, as the use case will typically be to, once on app load:
- check the current state
- listen to state changes (typically, to then load flag-critical parts of the app when the provider is READY)
At any rate, it makes sense to allow the current state to be read concurrently with notification operations. Thanks!
| currentState = newState | ||
| subscription.notifyListeners { | ||
| onStateChanged(newState) | ||
| executorService.executeSafe( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am in favour of the more straightforward approach here where we put onus for threading and exception handling on the user. Fail-fast prevents us from hiding exceptions from the dev and clear docs should point them in the right direction.
…manage threads and catch exceptions
0e8fb0a to
3e50ece
Compare
...dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/FlagsStateManager.kt
Outdated
Show resolved
Hide resolved
...dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/FlagsStateManager.kt
Outdated
Show resolved
Hide resolved
| ) { | ||
| val state = currentState | ||
| listener.onStateChanged(state) | ||
| stateLock.write { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in fact both addListener and removeListener don't need to be wrapped in the lock, because DDCoreSubscription is using copy-on-write under the hood, so changes to the subscription won't affect ongoing subscription.notifyListeners call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We want to ensure that the listener gets the current state here before any other state is broadcast, so we lock the current state until done. However, if we notify listener with the current state before adding it to the listeners, we can get by without the lock.
dropped the lock for removeListener as well - DDCoreSubscription is indeed sufficient here
...dk-android-flags/src/test/kotlin/com/datadog/android/flags/internal/FlagsStateManagerTest.kt
Outdated
Show resolved
Hide resolved
Co-authored-by: Nikita Ogorodnikov <4046447+0xnm@users.noreply.github.com>
0xnm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm. Maybe you would like to get also a look from @aleksandr-gringauz before merging.
|
Thanks Nikita! |
|
@codex review |
|
I think I found one small problem (not 100% sure), let's see if codex finds it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3ad003c3de
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| override fun addListener(listener: FlagsStateListener) { | ||
| listener.onStateChanged(currentState) | ||
| subscription.addListener(listener) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Register listener after state read under lock
In addListener the new listener is invoked before registration and without acquiring stateLock even though currentState is only written under that lock and is no longer @Volatile. If updateState is writing at the same time, this path can read a stale state and deliver it before the listener is ever added, so an in-flight transition (e.g., Ready) is missed and the subscriber remains out of sync. The initial callback should be issued while holding the read lock and after the listener is added to avoid lost updates.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. This is it.
I think it is better to put these 2 lines under the read-lock:
stateLock.read {
listener.onStateChanged(currentState)
subscription.addListener(listener)
}
Because otherwise in case of concurrent addListener(listener_1) and updateState(new_state) listener_1 might end up not with the latest state in the end.
The order could be the following:
addListener(listener_1)andupdateState(state_new)are called concurrently from different threads.- read of
currentStateoccurs inaddListenerand returnsstate_old. updateStatedoes its thing, updatingcurrentState=state_newand notifying listeners. But notlistener_1, because it hasn't been added yet.listener.onStateChanged(state_old)insideaddListeneris called.listener_1ends up withstate_old.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reverting back to the read lock here!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rewrapped the state access + addListener in the read lock plus added a test. ptal
...dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/FlagsStateManager.kt
Outdated
Show resolved
Hide resolved
|
pushed a Small fix. PTAL for final merge |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
Cherry-picked from PR #3068 to enable OpenFeature provider implementation. Adds state observation capabilities to FlagsClient with thread-safe synchronous state management using ReentrantReadWriteLock. Key changes: - FlagsStateManager: Synchronous state updates with read/write lock - State notification: Listeners called directly (no executor) - Thread safety: Caller manages threading and exception handling - Test updates: Enhanced coverage for state management Original PR: #3068 Branch: typo/FFL-1621-sdk-android
What does this PR do?
This PR changes the way that the current state's access and mutation is handled to ensure the ordering of state changes delivered to listeners. It also emphasizes the need for usercode listeners to catch their own exceptions lest they bubble and crash the app.
Motivation
As noted in #2998, the
currentStatewas updated synchronously erroneously and prevented other components from realizing that state directly after updatingThe decision was made to bubble usercode exceptions in order to fail fast and surface those errors to the dev rather than catching and logging them in our dispatch handler.
Additional Notes
Anything else we should know when reviewing?
Review checklist (to be filled by reviewers)