fix: switchTo - within block #3892

kobenguyent · 2023-09-26T09:05:42Z

Motivation/Description of the PR

Resolves within()/I.switchTo() fails in specific case: Either name or url matcher should be specified #3880
Resolves within()/I.switchTo(): Can't find text using "I.see(text)" in iframe #3891
Resolves within()/I.switchTo(): Context of a 2nd iframe is used though I switched to 1st iframe #3894

Applicable helpers:

Playwright

Type of change

🐛 Bug fix

Checklist:

Tests have been added
Documentation has been added (Run npm run docs)
Lint checking (Run npm run lint)
Local tests are passed (Run npm test)

mirao · 2023-10-05T18:15:09Z

@kobenguyent I replaced Playwright.js from 3.5.5 by content of Playright.js from the branch CodeceptJS/fix-switchTo
The cases from the three issues I reported work fine with your changes.

However one of my tests (using the helper Pixelmatcher methods in an iframe) is failing with the error context.waitForSelector is not a function (it passes in 3.5.3)

1) PlayDndGame
       Verify the game (stone) is not finished:
     context.waitForSelector is not a function
      at Playwright.waitForVisible (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_e6w65ce3sp4lg2q5fzcoe2hnqu/node_modules/codeceptjs/lib/helper/Playwright.js:2384:28)
      at async PixelmatchHelper._takeElementScreenshot (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_6hq34cvqihrbyrcnb4yjxphwgm/node_modules/codeceptjs-pixelmatchhelper/index.js:423:3)
      at async PixelmatchHelper.takeScreenshot (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_6hq34cvqihrbyrcnb4yjxphwgm/node_modules/codeceptjs-pixelmatchhelper/index.js:398:4)
  
  Scenario Steps:
  - I.checkVisualDifferences("Lesson_Path_Initial_chromium_headless", ".konvajs-content", 0.001) at verifyArea (/home/mirao/workspace/codeceptjs/tests/fragments/Konvajs.ts:15:7)

The scenario is

Scenario("Verify the game (stone) is not finished", () => {
    within(
        {
            frame: HtmlBodyPage.iFrame, // "(//iframe)[last()]"
        },
        () => {
            konvajsFragment.verifyLessonPathArea("Lesson_Path_Initial");
        }
    );
});

mirao · 2023-10-05T18:25:27Z

However one of my tests (using the helper Pixelmatcher methods in an iframe)

It looks like a different issue than the original ones. However it's related to within() (or maybe to iframe) because when I use other test using PixelmatchHelper.takeScreenshot and not using iframe, it works fine.
I'll try to create a simple test case how to reproduce on a public page.

kobenguyent · 2023-10-06T12:53:06Z

my simple test looked good! Not sure if you have any special setup. 🤔

Scenario.only('visual test',
    async ({I}) => {
        I.amOnPage('http://127.0.0.1:8000/iframe')
        within({frame: '(//iframe)[last()]'}, async () => {
            I.waitForVisible('h3', 5);
            await I.takeScreenshot("screenshot.png");
            await I.checkVisualDifferences("screenshot.png");
            const result = await I.getVisualDifferences("screenshot.png");
        });
    });

***************************************
CodeceptJS v3.5.5 #StandWithUkraine
Using test root "/Users/t/Downloads/highlight_error_sample"
Helpers: Playwright, PixelmatchHelper
Plugins: screenshotOnFail, tryTo

regression --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  visual test
    I am on page "http://127.0.0.1:8000/iframe"
    › [Browser:Error] Failed to load resource: the server responded with a status of 404 (Not Found)
    Within "{"frame":"(//iframe)[last()]"}" ""
      I wait for visible "h3", 5
      I take screenshot "screenshot.png"
      › Screenshot is saving to /Users/t/Downloads/highlight_error_sample/output/~4f1rp.temp.png
      I check visual differences "screenshot.png"
      › Check differences in screenshot.png ...
      › Load image from /Users/t/Downloads/highlight_error_sample/output/screenshot.png ...
      › Load image from /Users/t/Downloads/highlight_error_sample/tests/screenshots/base/screenshot.png ...
      › Difference: 0% | 0 / 921600 pixels
      I get visual differences "screenshot.png"
      › Check differences in screenshot.png ...
      › Load image from /Users/t/Downloads/highlight_error_sample/output/screenshot.png ...
      › Load image from /Users/t/Downloads/highlight_error_sample/tests/screenshots/base/screenshot.png ...
  ✔ OK in 523ms


  OK  | 1 passed   // 2s

mirao · 2023-10-07T15:55:30Z

@kobenguyent This test passes in 3.5.3 and 3.5.5, but doesn't pass in the branch CodeceptJS/fix-switchTo. So it seems to be broken by your latest changes in CodeceptJS/fix-switchTo.
It can be reproduced with I.takeScreenshot() if you pass an element you want to screenshot.

Config:

export const config: CodeceptJS.MainConfig = {
    tests: "./*_test.ts",
    output: "./output",
    helpers: {
        Playwright: {
            browser: "chromium",
            url: "http://localhost",
            show: true,
        },
        PixelmatchHelper: {
            require: "codeceptjs-pixelmatchhelper",
            dirExpected: "./screenshots/base/",
            dirDiff: "./screenshots/diff/",
        },
    },
    name: "my",
};

Test:

Feature("My");

Scenario("test something", ({ I }) => {
    I.amOnPage(
        "https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_iframe"
    );
    I.click("#accept-choices");
    I.wait(5); // Wait for Google ads to be loaded
    within(
        {
            frame: "iframe#iframeResult",
        },
        () => {
            I.takeScreenshot("screenshot.png", "actual", "h1");
        }
    );
});

Output:

$ codeceptjs run --debug
CodeceptJS v3.5.5 #StandWithUkraine
Using test root "/home/mirao/workspace/codeceptjs/tests/my"
Helpers: Playwright, PixelmatchHelper
Plugins: screenshotOnFail

My --
 › [Session] Starting singleton browser session
  test something
    I am on page "https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_iframe"
    I click "#accept-choices"
    I wait 5
    › [Browser:Error] Failed to load resource: the server responded with a status of 400 (Bad Request)
    › [Browser:TimeEnd] a: 0.006103515625 msa: 0.006103515625 ms
    Within "{"frame":"iframe#iframeResult"}" ""
      I take screenshot "screenshot.png", "actual", "h1"
      › [Browser:Error] Failed to load resource: the server responded with a status of 400 (Bad Request)
      › [Browser:Error] Failed to load resource: the server responded with a status of 400 ()
  › <screenshotOnFail> Test failed, try to save a screenshot
  › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/my/output/test_something.failed.png
  ✖ FAILED in 12032ms


-- FAILURES:

  1) My
       test something:
     context.waitForSelector is not a function
      at Playwright.waitForVisible (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_e6w65ce3sp4lg2q5fzcoe2hnqu/node_modules/codeceptjs/lib/helper/Playwright.js:2384:28)
      at async PixelmatchHelper._takeElementScreenshot (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_6hq34cvqihrbyrcnb4yjxphwgm/node_modules/codeceptjs-pixelmatchhelper/index.js:423:3)
      at async PixelmatchHelper.takeScreenshot (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_6hq34cvqihrbyrcnb4yjxphwgm/node_modules/codeceptjs-pixelmatchhelper/index.js:398:4)
  
  Scenario Steps:
  - I.takeScreenshot("screenshot.png", "actual", "h1") at ./My_test.ts:14:15
  - I.wait(5) at Test.<anonymous> (./My_test.ts:8:7)
  - I.click("#accept-choices") at Test.<anonymous> (./My_test.ts:7:7)
  - I.amOnPage("https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_iframe") at Test.<anonymous> (./My_test.ts:4:7)
  
  Artifacts:
  - screenshot: /home/mirao/workspace/codeceptjs/tests/my/output/test_something.failed.png


  FAIL  | 0 passed, 1 failed   // 13s

kobenguyent · 2023-10-09T07:31:36Z

May you check your code to see if you are using the correct branch?

CodeceptJS v3.5.5 #StandWithUkraine
Using test root "/Users/t/Downloads/highlight_error_sample"
Helpers: Playwright, PixelmatchHelper
Plugins: screenshotOnFail, tryTo

My --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  test something
    I am on page "https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_iframe"
    I click "#accept-choices"
    I wait 5
    › [Browser:Error] Failed to load resource: the server responded with a status of 400 (Bad Request)
    Within "{"frame":"iframe#iframeResult"}" ""
      I take screenshot "screenshot.png", "actual", "h1"
  ✔ OK in 6154ms


  OK  | 1 passed   // 9s

mirao · 2023-10-09T13:02:41Z

@kobenguyent Maybe it's a deployment issue on my side. I copied "Playwright.js" from current 3.x (with all your changes) to my project (with CodeceptJS 3.5.5) and the test is still failing (context.waitForSelector is not a function)
How do you test it in your environment?

kobenguyent · 2023-10-09T13:06:00Z

"codeceptjs": "github:codeceptjs/codeceptjs#fix-switchTo",

I installed the branch directly.

mirao · 2023-10-09T17:43:05Z

Tried again, with latest "3.x" in a fork of the repo codeceptjs/CodeceptJS.
I created the test directly in that project (in "~/workspace/CodeceptJS/test/my"). But still I'm getting context.waitForSelector is not a function.
When I switch my repo to the commit 206d1df, the test passes.
Yes, still it could be an issue in my forked repo or some misconfiguration, it's hard to say. Maybe I could test it correctly if you released the version as a some beta version or something like that.

kobenguyent · 2023-10-10T04:43:28Z

Perhaps your issue would be fixed with this #3918

BenjaminHoch-GoTo · 2023-10-10T07:47:27Z

could you please let me know, in which version this fix will be included? We are pinned to 3.5.3 because of this. 3.5.6?

kobenguyent · 2023-10-10T07:59:52Z

hey @BenjaminHoch-GoTo sorry for the confusion!
so if you use switchTo/Within and there is no waitForVisible/waitForInvisible/waitToHide inside that block, you could try 3.5.6, otherwise this would fix that issue but waiting for next release unfortunately #3918

BenjaminHoch-GoTo · 2023-10-10T08:51:27Z

Yeah i just tried out 3.5.6 and still get "context.waitForSelector is not a function" . We'll wait then.

mirao · 2023-10-10T12:26:53Z

Perhaps your issue would be fixed with this #3918

Yes, just tried my production test that originally failed on context.waitForSelector is not a function in 3.5.6.
It passes with the changes from #3918 ✔️
But I noticed that now the test takes quite long time. It seems that the action I take screenshot "screenshot.png", "actual", "h1" is now waiting for waitForTimeout always. So e.g. in our project it means additional 30 sec for every I.takeScreenshot("picture", "actual", "element") which is annoying.
It's fast with 3.5.3

This is how the simple test looks with 20 sec. timeout:

My --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  test something
    I am on page "https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_iframe"
    I click "#accept-choices"
    I wait 5
    › [Browser:Error] Failed to load resource: the server responded with a status of 400 (Bad Request)
    › [Browser:Error] %cAdConsent display: inline-block; color: #fff; background: #ff0000; padding: 1px 4px; border-radius: 3px; ERROR: Unknown __tcfapi() API call 'getInAppTCData'%cAdConsent display: inline-block; color: #fff; background: #ff0000; padding: 1px 4px; border-radius: 3px; ERROR: Unknown __tcfapi() API call 'getInAppTCData'
    Within "{"frame":"iframe#iframeResult"}" ""
      I take screenshot "screenshot.png", "actual", "h1"
      › [Browser:Error] Failed to load resource: the server responded with a status of 400 (Bad Request)
  ✔ OK in 26648ms


  OK  | 1 passed   // 28s

kobenguyent · 2023-10-10T14:21:42Z

please recheck #3918 if you have some time @mirao pushed some fixes for the waiting mechanism. Thank you!

mirao · 2023-10-10T16:31:07Z

@kobenguyent Looks good, it works well as with 3.5.3. Thanks!

mirao · 2023-11-07T14:36:15Z

But I noticed that now the test takes quite long time. It seems that the action I take screenshot "screenshot.png", "actual", "h1" is now waiting for waitForTimeout always

I have suspicion that I.waitForText() has the same bug. I'm testing now CodeceptJS 3.5.7 and I.waitForText("mytext") waits for 60 sec (my value in waitForTimeout) though text is already found.
I'm going to create an issue for it.

kobenguyent · 2023-11-07T14:40:52Z

I guess this covers your issue. #3984

mirao · 2023-11-07T14:54:21Z

@kobenguyent Confirmed. It fixes my issue, thanks.

fix: switchTo

cbb069c

kobenguyent added bug playwright labels Sep 26, 2023

mirao mentioned this pull request Sep 26, 2023

within()/I.switchTo(): Context of a 2nd iframe is used though I switched to 1st iframe #3894

Closed

Merge branch '3.x' into fix-switchTo

c27a955

Arhell approved these changes Sep 30, 2023

View reviewed changes

Arhell requested a review from DavertMik September 30, 2023 00:50

kobenguyent added 11 commits September 30, 2023 06:36

fix: switch to

9a2e3ba

fix: failed acceptance tests

d067ae7

fix: within block

8004bee

fix: within tests

c69dd1c

fix: within tests

70cc4d4

fix: within tests

8b1ad76

fix: within block

4bd0ed9

fix within block

81e4fbe

fix switchTo

e9f45a1

fix: puppeteer test

7fa87ca

fix: switchTo

5af37df

kobenguyent changed the title ~~fix: switchTo~~ fix: switchTo - within block Oct 4, 2023

kobenguyent merged commit 1832ced into 3.x Oct 9, 2023

kobenguyent deleted the fix-switchTo branch October 9, 2023 08:25

kobenguyent restored the fix-switchTo branch October 9, 2023 13:05

kobenguyent deleted the fix-switchTo branch October 9, 2023 13:54

BenjaminHoch-GoTo mentioned this pull request Nov 20, 2023

<iframe> was expected, prevents further execution #4012

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: switchTo - within block #3892

fix: switchTo - within block #3892

kobenguyent commented Sep 26, 2023 •

edited

Loading

mirao commented Oct 5, 2023

mirao commented Oct 5, 2023

kobenguyent commented Oct 6, 2023

mirao commented Oct 7, 2023

kobenguyent commented Oct 9, 2023

mirao commented Oct 9, 2023

kobenguyent commented Oct 9, 2023

mirao commented Oct 9, 2023

kobenguyent commented Oct 10, 2023

BenjaminHoch-GoTo commented Oct 10, 2023 •

edited

Loading

kobenguyent commented Oct 10, 2023

BenjaminHoch-GoTo commented Oct 10, 2023

mirao commented Oct 10, 2023

kobenguyent commented Oct 10, 2023

mirao commented Oct 10, 2023

mirao commented Nov 7, 2023

kobenguyent commented Nov 7, 2023

mirao commented Nov 7, 2023

fix: switchTo - within block #3892

fix: switchTo - within block #3892

Conversation

kobenguyent commented Sep 26, 2023 • edited Loading

Motivation/Description of the PR

Type of change

Checklist:

mirao commented Oct 5, 2023

mirao commented Oct 5, 2023

kobenguyent commented Oct 6, 2023

mirao commented Oct 7, 2023

kobenguyent commented Oct 9, 2023

mirao commented Oct 9, 2023

kobenguyent commented Oct 9, 2023

mirao commented Oct 9, 2023

kobenguyent commented Oct 10, 2023

BenjaminHoch-GoTo commented Oct 10, 2023 • edited Loading

kobenguyent commented Oct 10, 2023

BenjaminHoch-GoTo commented Oct 10, 2023

mirao commented Oct 10, 2023

kobenguyent commented Oct 10, 2023

mirao commented Oct 10, 2023

mirao commented Nov 7, 2023

kobenguyent commented Nov 7, 2023

mirao commented Nov 7, 2023

kobenguyent commented Sep 26, 2023 •

edited

Loading

BenjaminHoch-GoTo commented Oct 10, 2023 •

edited

Loading