test(e2e): Adds Feedback Widget Maestro E2E tests #4604

antonis · 2025-02-26T18:02:04Z

📢 Type of change

Bugfix
New feature
Enhancement
Refactoring

Based on: #4726

PR Chain:

📜 Description

Adds Feedback Widget Maestro E2E tests for the happy path flow:

Show feedback button
Open the feedback widget
Fill and submit the form

💡 Motivation and Context

Part of #4302

💚 How did you test it?

CI (submitted E2E test feedback), Manual

Note: To run Android locally you might need to change the build architecture android.patch

📝 Checklist

I added tests to verify changes
No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
I updated the docs if needed.
I updated the wizard if needed.
All tests passing
No breaking changes

🔮 Next steps

Add more tests outside the happy path

#skip-changelog

Co-authored-by: Krystof Woldrich <[email protected]>

…implementation

antonis · 2025-04-28T08:37:12Z

dev-packages/e2e-tests/maestro/feedback.yml

+
+- runFlow:
+    when:
+      visible: 'Engine: Hermes for RN 0.77.1' # Skipping for 0.65.3


As discussed and documented the Modal implementation is fully supported on React Native 0.71 and up.
To avoid complicating the e2e-tests/cli.mjs implementation to independently run each .yml test in order to separate the feedback.yml execution I skipped the test with Maestro. We can probably iterate by injecting the <FeedbackWidget> form in the test App if we want to cover 0.65.3 with the E2E tests.

Thank you for the note. The Modal is fully supported in 0.71 in the new architecture,

but for the legacy architecture, the test should pass (the modal should work) even for 0.65.3.

If it doesn't it might be unrelated to the modal, but we can discuss it.

but for the legacy architecture, the test should pass (the modal should work) even for 0.65.3.

I haven't been able to successfully run the tests on 0.65.3 legacy architecture either. From the logs it seems that the modal does not open but I'll need to investigate this further to understand why this is happening.

iOSLogs

14:46:27.222 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on id: feedback-button COMPLETED 14:46:27.222 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Assert that id: form-title is visible RUNNING 14:46:27.690 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:27.690 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:28.428 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:28.431 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:28.514 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:28.514 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:28.830 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:28.831 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:28.902 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:28.902 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:29.190 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:29.190 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:29.270 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:29.270 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:29.618 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:29.618 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:29.679 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:29.689 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:29.948 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:29.948 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:30.048 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:30.049 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:30.806 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:30.817 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:30.902 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:30.908 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:31.247 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:31.249 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:31.400 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:31.402 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:31.761 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:31.775 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:32.044 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:32.048 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:32.336 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:32.371 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:32.446 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:32.447 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:32.696 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:32.696 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:32.780 [ INFO] ios.xctest.XCTestIOSDevice.invoke: Device info DeviceInfo(widthPixels=1179, heightPixels=2556, widthPoints=393, heightPoints=852) 14:46:32.780 [ INFO] ios.IOSDevice.viewHierarchy: Requesting view hierarchy of the screen 14:46:33.078 [TRACE] ios.xctest.XCTestIOSDevice.invoke: Depth received: 14 14:46:33.079 [ INFO] ios.IOSDevice.viewHierarchy: Depth of the screen is 14 14:46:33.080 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Assert that id: form-title is visible FAILED 14:46:33.137 [ INFO] maestro.Maestro.takeScreenshot: Taking screenshot 14:46:33.154 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink 14:46:33.637 [ERROR] maestro.orchestra.Orchestra.executeCommands: [Command execution] CommandFailed: Assertion is false: id: form-title is visible 14:46:33.665 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Run feedback/happyFlow.yml FAILED

Android logs

14:42:41.038 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on "Feedback" COMPLETED 14:42:41.038 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on id: feedback-button RUNNING 14:42:41.130 [ INFO] maestro.Maestro.tap-BUbHBYE: Tapping on element: UiElement(treeNode=TreeNode(attributes={text=Report a Bug, accessibilityText=, hintText=, ignoreBoundsFiltering=false, resource-id=feedback-button, clickable=false, bounds=[171,569][274,594], enabled=true, focused=false, checked=false, scrollable=false, selected=false, class=android.widget.TextView}, children=[], clickable=false, enabled=true, focused=false, checked=false, selected=false), bounds=Bounds(x=171, y=569, width=103, height=25)) 14:42:42.045 [ INFO] maestro.Maestro.tap-BUbHBYE: Refreshed element 14:42:42.046 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Tapping at (222, 581) using hierarchy based logic for wait 14:42:42.046 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to byte array 14:42:42.046 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink 14:42:43.411 [ INFO] maestro.Maestro.hierarchyBasedTap-hbl3e4M: Something have changed in the UI judging by view hierarchy. Proceed. 14:42:43.412 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Tap on id: feedback-button COMPLETED 14:42:43.412 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Assert that id: form-title is visible RUNNING 14:42:48.608 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Assert that id: form-title is visible FAILED 14:42:48.609 [ INFO] maestro.Maestro.takeScreenshot: Taking screenshot 14:42:48.609 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink 14:42:48.709 [ERROR] maestro.orchestra.Orchestra.executeCommands: [Command execution] CommandFailed: Assertion is false: id: form-title is visible 14:42:48.709 [ INFO] maestro.cli.runner.TestSuiteInteractor.invoke: Run feedback/happyFlow.yml FAILED

Investigating this further it seems that the issue is not related to the Modal but to the 'backgroundColor' not supported by native animations and causing a crash in older RN versions.

I've enabled the test for all RN versions with 9137fb3 and disabled the native driver for the backgroundColor animation with 1a68b93 and cb20fbe

Note: I've also opened a PR on main targetting the current beta feedback widget release. This also checks the RN version and enables the native driver for supported versions.

Android crash logs

Error: Style property 'backgroundColor' is not supported by native animated module This error is located at: in c in f in RCTView in Unknown in __Sentry.TouchEventBoundary in Unknown in RCTView in Unknown in RCTView in Unknown in u Error: Style property 'backgroundColor' is not supported by native animated module This error is located at: in c in f in RCTView in Unknown in __Sentry.TouchEventBoundary in Unknown in RCTView in Unknown in RCTView in Unknown in u, stack: validateStyles@221:4389 value@250:2140 value@226:2615 value@249:2877 value@226:2615 value@249:2274 value@249:1839 value@225:1674 value@250:1925 value@225:1674 value@224:3074 value@225:1674 value@241:639 r@242:1432 value@242:1611 value@220:2510 s@230:1060 start@230:1119 <unknown>@230:2366 forEach@-1 start@230:2247 value@736:3755 zl@59:65656 Va@59:81394 Va@-1 <unknown>@189:3836 Ha@59:79751 Ia@59:75906 Ia@-1 <unknown>@59:26275 <unknown>@189:3836 ht@59:26222 pt@59:26157 Ee@59:93057 Te@59:13008 Ce@59:13397 receiveTouches@59:14183 value@40:3537 <unknown>@40:730 value@40:2527 value@40:702 value@-1 Creating new dialog from context: com.rndiffapp.MainActivity@81c704c@136081484 FATAL EXCEPTION: main (Ask Gemini) Process: com.rndiffapp, PID: 6044 com.facebook.react.bridge.JSApplicationIllegalArgumentException: disconnectAnimatedNodeFromView: Animated node with tag [6] does not exist at com.facebook.react.animated.NativeAnimatedNodesManager.disconnectAnimatedNodeFromView(NativeAnimatedNodesManager.java:411) at com.facebook.react.animated.NativeAnimatedModule$18.execute(NativeAnimatedModule.java:772) at com.facebook.react.animated.NativeAnimatedModule.executeAllOperations(NativeAnimatedModule.java:258) at com.facebook.react.animated.NativeAnimatedModule.access$400(NativeAnimatedModule.java:85) at com.facebook.react.animated.NativeAnimatedModule$3.execute(NativeAnimatedModule.java:287) at com.facebook.react.uimanager.UIViewOperationQueue$UIBlockOperation.execute(UIViewOperationQueue.java:579) at com.facebook.react.uimanager.UIViewOperationQueue$1.run(UIViewOperationQueue.java:914) at com.facebook.react.uimanager.UIViewOperationQueue.flushPendingBatches(UIViewOperationQueue.java:1025) at com.facebook.react.uimanager.UIViewOperationQueue.access$2600(UIViewOperationQueue.java:47) at com.facebook.react.uimanager.UIViewOperationQueue$DispatchUIFrameCallback.doFrameGuarded(UIViewOperationQueue.java:1085) at com.facebook.react.uimanager.GuardedFrameCallback.doFrame(GuardedFrameCallback.java:29) at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175) at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85) at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1229) at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1239) at android.view.Choreographer.doCallbacks(Choreographer.java:899) at android.view.Choreographer.doFrame(Choreographer.java:827) at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1214) at android.os.Handler.handleCallback(Handler.java:942) at android.os.Handler.dispatchMessage(Handler.java:99) at android.os.Looper.loopOnce(Looper.java:201) at android.os.Looper.loop(Looper.java:288) at android.app.ActivityThread.main(ActivityThread.java:7872) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936) No tap action can be set up failed to get buffer, invalid process token Failed to transact (-32) No tap action can be set up

antonis · 2025-04-28T09:43:26Z

Note that End-to-End Tests / metrics (new, ios) (pull_request) already fail in the feature branch PR and will be handled with #4784

dev-packages/e2e-tests/maestro/feedback/happyFlow.yml

krystofwoldrich · 2025-04-28T14:59:37Z

packages/core/src/js/feedback/FeedbackWidget.tsx

              />
            )}
          </View>

          {config.showName && (
          <>
-            <Text style={styles.label}>
+            <Text style={styles.label} testID='name-label' accessibilityLabel={text.nameLabel}>


Do we need the additional testId and accessibilityLabel?

I would prefer not using the testIDs if not needed to avoid conflicting with user apps.

If we need them then lets prefix them with something unique like sentry-feedback-*.

Updated with 8176d4f:

I've cleaned up all the unneeded test-id and used the sentry-feedback-* prefix for those used.

Removed the unneeded accessibility labels.

I would prefer not using the testIDs if not needed to avoid conflicting with user apps.

I ended up using both testID's and text lookups since:

On iOS legacy text lookups fail

On Android RN 0.65.3 input fields lookups fail with testID's

This reverts commit e20aff2.

krystofwoldrich · 2025-04-30T16:29:20Z

dev-packages/e2e-tests/maestro/feedback.yml

+
+# The following tests are happy path tests for the feedback widget on both iOS and Android.
+# They verify that the feedback form can be opened, filled out, and submitted successfully.
+# The tests are separate because iOS tests work better with `testID` and Android tests work better with `text`.


The Android Test is super clean, sad that is doesn't work for iOS.

Is it flaky or the elements are not found?

On iOS legacy text lookups fail consistently :(

antonis · 2025-04-30T16:33:10Z

Thank you for all the feedback @krystofwoldrich 🙇
This is now ready for another pass 🙏

# Conflicts: # dev-packages/e2e-tests/patch-scripts/rn.patch.app.js # packages/core/src/js/feedback/FeedbackWidgetProvider.tsx

lucas-zimerman

Nice E2E Tests! LGTM

antonis and others added 30 commits November 27, 2024 14:00

Update the client implementation to use the new capture feedback js api

817eac8

Updates SDK API

5370a99

Adds new feedback button in the sample

42e2fa1

Adds changelog

514b102

Removes unused mock

0935bbd

Update CHANGELOG.md

9ea5496

Co-authored-by: Krystof Woldrich <[email protected]>

Directly use captureFeedback from sentry/core

da0d4ac

Use import from core

3e36c6d

Fixes imports order lint issue

5f3df64

Fixes build issue

71b28e8

Adds captureFeedback tests from sentry-javascript

f9d2b59

Update CHANGELOG.md

d05d531

Only deprecate client captureUserFeedback

2bb104b

Add simple form UI

4339274

Adds basic form functionality

6ce799b

Update imports

f2cefc6

Update imports

694ee33

Merge branch 'main' into antonis/3859-newCaptureFeedbackAPI-Form

18b1c33

Remove useState hook to avoid multiple react instances issues

034efde

Move types and styles in different files

67a492d

Removes attachment button to be added back separately along with the …

8eaa61d

…implementation

Add basic field validation

0b88cc5

Adds changelog

ae11b8d

Updates changelog

7f2ca06

Updates changelog

064b6c4

Merge branch 'main' into antonis/3859-newCaptureFeedbackAPI-Form

e2add4a

Trim whitespaces from the submitted feedback

e21718a

Adds tests

407f179

Merge branch 'main' into antonis/3859-newCaptureFeedbackAPI-Form

14ac005

Renames FeedbackFormScreen to FeedbackForm

ddade00

Update snapshots

211170d

antonis commented Apr 28, 2025

View reviewed changes

antonis marked this pull request as ready for review April 28, 2025 08:37

antonis requested review from krystofwoldrich and lucas-zimerman as code owners April 28, 2025 08:37

krystofwoldrich reviewed Apr 28, 2025

View reviewed changes

dev-packages/e2e-tests/maestro/feedback/happyFlow.yml Outdated Show resolved Hide resolved

krystofwoldrich reviewed Apr 28, 2025

View reviewed changes

antonis added 4 commits April 29, 2025 09:15

Merge branch 'feedback-ui-2' into antonis/feedback-e2e

882c22e

Cleanup testIds and accessibility labels

8176d4f

Enable for all RN versions

9137fb3

Temp: remove modal supported check

e20aff2

antonis marked this pull request as draft April 30, 2025 11:17

antonis added 3 commits April 30, 2025 14:40

Do not use native driver for backgroundOpacity animation

1a68b93

Revert "Temp: remove modal supported check"

910d86e

This reverts commit e20aff2.

Do not use native driver for backgroundOpacity animation

cb20fbe

antonis mentioned this pull request Apr 30, 2025

fix(feedback): Disable native driver for backgroundColor animation #4794

Merged

10 tasks

antonis added 2 commits April 30, 2025 17:40

Use text instead of test ids

e122ade

Separate iOS and Android tests

b7fee86

krystofwoldrich reviewed Apr 30, 2025

View reviewed changes

antonis marked this pull request as ready for review April 30, 2025 16:32

antonis requested a review from krystofwoldrich April 30, 2025 16:39

Merge branch 'feedback-ui-2' into antonis/feedback-e2e

842824c

# Conflicts: # dev-packages/e2e-tests/patch-scripts/rn.patch.app.js # packages/core/src/js/feedback/FeedbackWidgetProvider.tsx

antonis mentioned this pull request May 1, 2025

test(e2e): Adds Feedback Widget Capture Screenshot Maestro E2E tests #4795

Draft

10 tasks

Merge branch 'feedback-ui-2' into antonis/feedback-e2e

888baca

lucas-zimerman approved these changes May 7, 2025

View reviewed changes

Merge branch 'feedback-ui-2' into antonis/feedback-e2e

93ef135

antonis merged commit 92c4d2d into feedback-ui-2 May 9, 2025
69 checks passed

antonis deleted the antonis/feedback-e2e branch May 9, 2025 07:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(e2e): Adds Feedback Widget Maestro E2E tests #4604

test(e2e): Adds Feedback Widget Maestro E2E tests #4604

antonis commented Feb 26, 2025 •

edited

Loading

antonis Apr 28, 2025

krystofwoldrich Apr 28, 2025

antonis Apr 29, 2025 •

edited

Loading

antonis Apr 30, 2025 •

edited

Loading

antonis commented Apr 28, 2025

krystofwoldrich Apr 28, 2025

antonis Apr 29, 2025

antonis Apr 30, 2025

krystofwoldrich Apr 30, 2025

antonis Apr 30, 2025

antonis commented Apr 30, 2025

lucas-zimerman left a comment

test(e2e): Adds Feedback Widget Maestro E2E tests #4604

test(e2e): Adds Feedback Widget Maestro E2E tests #4604

Conversation

antonis commented Feb 26, 2025 • edited Loading

📢 Type of change

📜 Description

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

🔮 Next steps

antonis Apr 28, 2025

Choose a reason for hiding this comment

krystofwoldrich Apr 28, 2025

Choose a reason for hiding this comment

antonis Apr 29, 2025 • edited Loading

Choose a reason for hiding this comment

antonis Apr 30, 2025 • edited Loading

Choose a reason for hiding this comment

antonis commented Apr 28, 2025

krystofwoldrich Apr 28, 2025

Choose a reason for hiding this comment

antonis Apr 29, 2025

Choose a reason for hiding this comment

antonis Apr 30, 2025

Choose a reason for hiding this comment

krystofwoldrich Apr 30, 2025

Choose a reason for hiding this comment

antonis Apr 30, 2025

Choose a reason for hiding this comment

antonis commented Apr 30, 2025

lucas-zimerman left a comment

Choose a reason for hiding this comment

antonis commented Feb 26, 2025 •

edited

Loading

antonis Apr 29, 2025 •

edited

Loading

antonis Apr 30, 2025 •

edited

Loading