Skip to content

[Feature]: Playwright.Xunit should have a V3 variant #3093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alexaka1 opened this issue Jan 2, 2025 · 5 comments · May be fixed by #3097
Open

[Feature]: Playwright.Xunit should have a V3 variant #3093

alexaka1 opened this issue Jan 2, 2025 · 5 comments · May be fixed by #3097

Comments

@alexaka1
Copy link

alexaka1 commented Jan 2, 2025

🚀 Feature Request

With XUnit v3 released (https://xunit.net/releases/v3/1.0.0), Playwright should support it the same way it supports NUnit, MSTest etc.

Example

No response

Motivation

It will allow Playwright to utilize the new TestContext of XUnit v3, such as TestContext.Current.TestStatus for enabling tracing on failed tests (currently not possible with xunit v2).

@mxschmitt mxschmitt linked a pull request Jan 23, 2025 that will close this issue
@mxschmitt
Copy link
Member

mxschmitt commented Feb 10, 2025

Linking xunit/xunit#3146 and #3097. Looks like this requires architectural changes.

@omni-htg
Copy link

Should proposals be set up over a new PR, or continue over what #3097 has achieved so far? Thanks!

@alexaka1
Copy link
Author

alexaka1 commented Mar 20, 2025

Sorry guys, I kept meaning to post my workaround but completely forgot it. This is what I went with, works perfectly for my project:

using System.Collections.Concurrent;
using System.Globalization;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.Playwright.TestAdapter;
using Xunit;

// ReSharper disable once CheckNamespace
namespace Microsoft.Playwright.Xunit;

public class PageTest : ContextTest
{
    public IPage Page { get; private set; } = null!;

    public override async ValueTask InitializeAsync()
    {
        await base.InitializeAsync().ConfigureAwait(false);
        Page = await Context.NewPageAsync().ConfigureAwait(false);
    }
}

public class ContextTest : BrowserTest
{
    public IBrowserContext Context { get; private set; } = null!;

    public override async ValueTask InitializeAsync()
    {
        await base.InitializeAsync().ConfigureAwait(false);
        Context = await NewContext(ContextOptions()).ConfigureAwait(false);
    }

    public virtual BrowserNewContextOptions ContextOptions()
    {
        return new BrowserNewContextOptions
        {
            Locale = "en-US",
            ColorScheme = ColorScheme.Light,
        };
    }
}

public class BrowserTest : PlaywrightTest
{
    private readonly List<IBrowserContext> _contexts = new();
    public IBrowser Browser { get; internal set; } = null!;

    /// <summary>
    ///     Creates a new context and adds it to the list of contexts to be disposed.
    /// </summary>
    /// <param name="options"></param>
    /// <returns></returns>
    public async Task<IBrowserContext> NewContext(BrowserNewContextOptions? options = null)
    {
        var context = await Browser.NewContextAsync(options).ConfigureAwait(false);
        _contexts.Add(context);
        return context;
    }

    public override async ValueTask InitializeAsync()
    {
        await base.InitializeAsync().ConfigureAwait(false);
        var service = await BrowserService.Register(this, BrowserType).ConfigureAwait(false);
        Browser = service.Browser;
    }

    public override async ValueTask DisposeAsync()
    {
        if (TestOk)
        {
            foreach (var context in _contexts)
            {
                await context.CloseAsync().ConfigureAwait(false);
            }
        }

        _contexts.Clear();
        Browser = null!;
        await base.DisposeAsync().ConfigureAwait(false);
    }
}

public class PlaywrightTest : WorkerAwareTest
{
    public string BrowserName { get; internal set; } = null!;

    public IPlaywright Playwright { get; private set; } = null!;
    public IBrowserType BrowserType { get; private set; } = null!;

    private static readonly Task<IPlaywright> _playwrightTask = Microsoft.Playwright.Playwright.CreateAsync();

    public override async ValueTask InitializeAsync()
    {
        await base.InitializeAsync().ConfigureAwait(false);
        Playwright = await _playwrightTask.ConfigureAwait(false);
        BrowserName = PlaywrightSettingsProvider.BrowserName;
        BrowserType = Playwright[BrowserName];
        Playwright.Selectors.SetTestIdAttribute("data-testid");
    }

    public static void SetDefaultExpectTimeout(float timeout)
    {
        Assertions.SetDefaultExpectTimeout(timeout);
    }

    public ILocatorAssertions Expect(ILocator locator)
    {
        return Assertions.Expect(locator);
    }

    public IPageAssertions Expect(IPage page)
    {
        return Assertions.Expect(page);
    }

    public IAPIResponseAssertions Expect(IAPIResponse response)
    {
        return Assertions.Expect(response);
    }
}

public class WorkerAwareTest : ExceptionCapturer
{
    private Worker _currentWorker = null!;

    public int WorkerIndex { get; internal set; }
    private static readonly ConcurrentStack<Worker> _allWorkers = new();

    public async Task<T> RegisterService<T>(string name, Func<Task<T>> factory) where T : class, IWorkerService
    {
        if (!_currentWorker.Services.ContainsKey(name))
        {
            _currentWorker.Services[name] = await factory().ConfigureAwait(false);
        }

        return (_currentWorker.Services[name] as T)!;
    }

    public override async ValueTask InitializeAsync()
    {
        await base.InitializeAsync().ConfigureAwait(false);
        if (!_allWorkers.TryPop(out _currentWorker!))
        {
            _currentWorker = new Worker();
        }

        WorkerIndex = _currentWorker.WorkerIndex;
        if (PlaywrightSettingsProvider.ExpectTimeout.HasValue)
        {
            Assertions.SetDefaultExpectTimeout(PlaywrightSettingsProvider.ExpectTimeout.Value);
        }
    }

    public override async ValueTask DisposeAsync()
    {
        if (TestOk)
        {
            foreach (var kv in _currentWorker.Services)
            {
                await kv.Value.ResetAsync().ConfigureAwait(false);
            }

            _allWorkers.Push(_currentWorker);
        }
        else
        {
            foreach (var kv in _currentWorker.Services)
            {
                await kv.Value.DisposeAsync().ConfigureAwait(false);
            }

            _currentWorker.Services.Clear();
        }

        await base.DisposeAsync().ConfigureAwait(false);
    }

    internal class Worker
    {
        public Dictionary<string, IWorkerService> Services = [];
        public int WorkerIndex = Interlocked.Increment(ref _lastWorkedIndex);
        private static int _lastWorkedIndex;
    }
}

public interface IWorkerService
{
    public Task ResetAsync();
    public Task DisposeAsync();
}

/// <summary>
///     ExceptionCapturer is a best-effort way of detecting if a test did pass or fail in xUnit.
///     This class uses the AppDomain's FirstChanceException event to set a flag indicating
///     whether an exception has occurred during the test execution.
///     Note: There is no way of getting the test status in xUnit in the dispose method.
///     For more information, see: https://stackoverflow.com/questions/28895448/current-test-status-in-xunit-net
/// </summary>
public class ExceptionCapturer : IAsyncLifetime
{
    protected static bool TestOk { get => TestContext.Current.TestState?.Result is not TestResult.Failed; }

    public virtual ValueTask DisposeAsync()
    {
        return ValueTask.CompletedTask;
    }

    public virtual ValueTask InitializeAsync()
    {
        return ValueTask.CompletedTask;
    }
}

internal class BrowserService : IWorkerService
{
    public IBrowser Browser { get; }

    private BrowserService(IBrowser browser)
    {
        Browser = browser;
    }

    public Task DisposeAsync()
    {
        return Browser.CloseAsync();
    }

    public Task ResetAsync()
    {
        return Task.CompletedTask;
    }

    public static Task<BrowserService> Register(WorkerAwareTest test, IBrowserType browserType)
    {
        return test.RegisterService("Browser",
            async () => new BrowserService(await CreateBrowser(browserType).ConfigureAwait(false)));
    }

    private static async Task<IBrowser> CreateBrowser(IBrowserType browserType)
    {
        var accessToken = Environment.GetEnvironmentVariable("PLAYWRIGHT_SERVICE_ACCESS_TOKEN");
        var serviceUrl = Environment.GetEnvironmentVariable("PLAYWRIGHT_SERVICE_URL");

        if (string.IsNullOrEmpty(accessToken) || string.IsNullOrEmpty(serviceUrl))
        {
            return await browserType.LaunchAsync(PlaywrightSettingsProvider.LaunchOptions).ConfigureAwait(false);
        }

        var exposeNetwork = Environment.GetEnvironmentVariable("PLAYWRIGHT_SERVICE_EXPOSE_NETWORK") ?? "<loopback>";
        var os = Uri.EscapeDataString(Environment.GetEnvironmentVariable("PLAYWRIGHT_SERVICE_OS") ?? "linux");
        var runId = Uri.EscapeDataString(Environment.GetEnvironmentVariable("PLAYWRIGHT_SERVICE_RUN_ID") ??
#pragma warning disable RS0030
                                         DateTime.Now.ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ss",
#pragma warning restore RS0030
                                             CultureInfo.InvariantCulture));
        var apiVersion = "2023-10-01-preview";
        var wsEndpoint = $"{serviceUrl}?os={os}&runId={runId}&api-version={apiVersion}";
        var connectOptions = new BrowserTypeConnectOptions
        {
            Timeout = 3 * 60 * 1000,
            ExposeNetwork = exposeNetwork,
            Headers = new Dictionary<string, string>
            {
                ["Authorization"] = $"Bearer {accessToken}",
                ["x-playwright-launch-options"] = JsonSerializer.Serialize(PlaywrightSettingsProvider.LaunchOptions,
                    new JsonSerializerOptions
                    {
                        DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
                    }),
            },
        };

        return await browserType.ConnectAsync(wsEndpoint, connectOptions).ConfigureAwait(false);
    }
}

This is pretty much 1:1 the Xunit v2 version, except that the EceptionCapturer is no longer needed, but I kept it because I was lazy to redo the inheritance. And TestContext now has some data that can be used instead, like for TestOk. I just used Microsoft.Playwright.TestAdapter and xunit.v3.extensibility.core in a class lib, to share this with all my test projects.

@julealgon
Copy link

Is this being actively worked on? We are starting to use Playwright in our solution, and we've been using xunit v3 for all of our other tests thus far. I'd like to keep using that same version for the new UI tests as well but was surprised to find that there was no XunitV3 package for Playwright.

I see the v1.52 label there @mxschmitt . Is there an ETA for that release?

Thanks for posting your workaround @alexaka1 . We'll likely go with something like that as well while we wait for official support. I don't want to maintain 2 distinct xunit pipelines in our solution.

@kohestanimahdi
Copy link

Do you have any updates on this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants