Is there a test client? #332
Comments
|
There is currently no test client. I have one for the Flask integration with this package, it would be nice to have something similar that is generic. The best approximation is to use a real server with a real client, both possibly running within the same process. |
|
Thanks. I tried to implement the test in such a way, but cannot manage to have the ASGI server running in a separate thread or process while the test client connects to it. My test looks like this: @pytest.mark.asyncio
async def test_websocket():
config = uvicorn.Config(app.create_app(), host='localhost', port=8000)
server = uvicorn.Server(config)
loop = asyncio.get_running_loop()
serve_coroutine = await server.serve()
executor = concurrent.futures.ProcessPoolExecutor(
max_workers=5,
)
loop.run_in_executor(executor, serve_coroutine)
# this line is reached only when I press Ctrl-C and kill the server
async def connect_to_ws():
sio = socketio.Client()
sio.connect('http://localhost:8000')
# here would go assertions on the socket responses
k = await connect_to_ws()
loop.run_in_executor(executor, k)the def create_app():
app = Starlette(debug=True)
app.mount('/', StaticFiles(directory='static'), name='static')
sio = socketio.AsyncServer(async_mode='asgi')
extended_app = socketio.ASGIApp(sio, app)
# here define HTTP and Socketio handlers
return extended_appthe basic idea is to start the complete server and just connect to it, I assumed I could run the server and the test client in the same event loop but apparently when I run the server (that is indeed started and I can reach with the browser) it blocks the test code. Only when I use Ctrl-C to stop it the server is killed and the rest of the test runs but of course it doesn't find the server. Probably I'm missing something essential here, I expected the server and the test client to run concurrently on the same event loop without need for multithreading or multiprocessing. |
|
In your example you are using a process executor, so you are in fact using multiprocessing there. I think this can be done in a much simpler way. Here is a rough attempt that appears to be work well: import asyncio
import socketio
sio = socketio.AsyncServer(async_mode='asgi', monitor_clients=False)
app = socketio.ASGIApp(sio)
def start_server():
import asyncio
from uvicorn import Config, Server
config = Config(app, host='127.0.0.1', port=5000)
server = Server(config=config)
config.setup_event_loop()
loop = asyncio.get_event_loop()
server_task = server.serve()
asyncio.ensure_future(server_task)
return server_task
async def run_client():
client = socketio.AsyncClient()
await client.connect('http://localhost:5000')
await asyncio.sleep(5)
await client.disconnect()
start_server()
loop = asyncio.get_event_loop()
loop.run_until_complete(run_client())Hopefully this will get you started. |
|
Thanks a lot! Indeed from this example I was able to make it work :) For whoever will encounter the problem in the future, in case someone in the future is interested this is my implementation: The app: def create_app():
app = Starlette(debug=True)
app.mount('/', StaticFiles(directory='static'), name='static')
sio = socketio.AsyncServer(async_mode='asgi')
extended_app = socketio.ASGIApp(sio, app)
@sio.on('double')
async def double(sid, data):
logging.info(f"doubling for {sid}")
return 'DOUBLED:' + data * 2
# here add HTTP and WS handlers...
return extended_appThe test, based on import asyncio
import socketio
import uvicorn
import pytest
from myapp import app
def get_server():
config = uvicorn.Config(app.create_app(), host='localhost', port=8000)
server = uvicorn.Server(config=config)
config.setup_event_loop()
return server
@pytest.fixture
async def async_get_server():
server = get_server()
server_task = server.serve()
asyncio.ensure_future(server_task)
@pytest.mark.asyncio
async def test_websocket(async_get_server):
client = socketio.AsyncClient()
await client.connect('http://localhost:8000')
result = await client.call('double', 'hello')
assert result == 'DOUBLED:hellohello'
await client.disconnect()This work although it produces a lot of warnings (I suspect some problem with the logs). A weird thing I noticed is that to run this it is required to install |
|
The |
|
I'm at a point where having a test client with socketio would be really helpful for me too. Is there any update on the progress of this? |
|
@nbanmp I am not currently working on a test client. The main reason is that there is a real Python client now. Is there anything that prevents you from using the real client against your real server for tests? |
|
Thanks for the update. Running the real client against the real server has some issues, the main one for me is that it is more difficult to run multiple tests asynchronously. But also important is that, I would like my unit tests to be as independent as possible, and I was expecting to run the actual server for integration testing. |
|
In any case, the test client that exists in the Flask-SocketIO package is very limited, if/when I get to do a generic test client it would be based on the real client talking to the real server. It would make it easier to start/stop the server for each test, but other than that I expect it will be the real thing, not a fake. |
|
There should to be a way to gracefully shutdown the server given the setup above. This would need to cancel the |
|
@databasedav The service task does not need to run when testing. A testing set up can be made by subclassing the |
|
@miguelgrinberg I agree; I was just talking in the context of using a live server like discussed above |
|
@databasedav start your server with |
This definitely did the trick! For future readers, I also had to do the following:
If you also want to work with HTTP requests in the same client session, use I also used the following to shutdown after the test: I do still wonder if it's possible to set this up directly on ASGI level, instead of actually binding to ports and hostnames... |
|
Ok, here goes a complete example, using FastAPI as the primary ASGI app and socketio as the secondary one. The chat server echoes the message to all clients.
import os
import socketio
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
app = FastAPI()
path = os.path.dirname(__file__)
app.mount("/static", StaticFiles(directory=os.path.join(path, "static")), name="static")
sio = socketio.AsyncServer(async_mode='asgi')
app.mount('/sio', socketio.ASGIApp(sio)) # socketio adds automatically /socket.io/ to the URL.
@sio.on('connect')
def sio_connect(sid, environ):
"""Track user connection"""
print('A user connected')
@sio.on('disconnect')
def sio_disconnect(sid):
"""Track user disconnection"""
print('User disconnected')
@sio.on('chat message')
async def chat_message(sid, msg):
"""Receive a chat message and send to all clients"""
print(f"Server received: {msg}")
await sio.emit('chat message', msg)
from typing import List, Optional
# stdlib imports
import asyncio
# 3rd party imports
import pytest
import socketio
import uvicorn
# FastAPI imports
from fastapi import FastAPI
# project imports
from .. import main
PORT = 8000
# deactivate monitoring task in python-socketio to avoid errores during shutdown
main.sio.eio.start_service_task = False
class UvicornTestServer(uvicorn.Server):
"""Uvicorn test server
Usage:
@pytest.fixture
async def start_stop_server():
server = UvicornTestServer()
await server.up()
yield
await server.down()
"""
def __init__(self, app: FastAPI = main.app, host: str = '127.0.0.1', port: int = PORT):
"""Create a Uvicorn test server
Args:
app (FastAPI, optional): the FastAPI app. Defaults to main.app.
host (str, optional): the host ip. Defaults to '127.0.0.1'.
port (int, optional): the port. Defaults to PORT.
"""
self._startup_done = asyncio.Event()
super().__init__(config=uvicorn.Config(app, host=host, port=port))
async def startup(self, sockets: Optional[List] = None) -> None:
"""Override uvicorn startup"""
await super().startup(sockets=sockets)
self.config.setup_event_loop()
self._startup_done.set()
async def up(self) -> None:
"""Start up server asynchronously"""
self._serve_task = asyncio.create_task(self.serve())
await self._startup_done.wait()
async def down(self) -> None:
"""Shut down server asynchronously"""
self.should_exit = True
await self._serve_task
@pytest.fixture
async def startup_and_shutdown_server():
"""Start server as test fixture and tear down after test"""
server = UvicornTestServer()
await server.up()
yield
await server.down()
@pytest.mark.asyncio
async def test_chat_simple(startup_and_shutdown_server):
"""A simple websocket test"""
sio = socketio.AsyncClient()
future = asyncio.get_running_loop().create_future()
@sio.on('chat message')
def on_message_received(data):
print(f"Client received: {data}")
# set the result
future.set_result(data)
message = 'Hello!'
await sio.connect(f'http://localhost:{PORT}', socketio_path='/sio/socket.io/')
print(f"Client sends: {message}")
await sio.emit('chat message', message)
# wait for the result to be set (avoid waiting forever)
await asyncio.wait_for(future, timeout=1.0)
await sio.disconnect()
assert future.result() == messageHere goes a test run: Notes:
|
|
@erny some suggestions for improvement over the hardcoded sleeps (untested), which should make your test suite faster overall: |
|
Awesome, thanks @erny! Given that this appears to be figured out, I'm going to close this issue. |
|
@Korijn I included your improvements in the updated example. Thank you very much. |
|
@Korijn, @miguelgrinberg , I was not able to remove the |
You would need to wait in a loop for Usage examples: Also I guess you could still lower the interval quite a bit, like 0.001 or even lower maybe. |
|
I was thinking about defining the result as future, something a bit more
elegant, but I'm not sure if I'll be able to do it.
Regards
El mar., 20 oct. 2020 21:40, Korijn van Golen <[email protected]>
escribió:
… @Korijn <https://github.com/Korijn>, @miguelgrinberg
<https://github.com/miguelgrinberg> , I was not able to remove the
sio.sleep(0.1) after sio.emit. Is there any alternative?
You would need to wait in a loop for result.message_received to become
True, very similar to how wait_ready is defined. You could pass the
condition to wait for as an argument to make it reusable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#332 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAS24ELXQN6SWEQOBIQDLLSLXRUFANCNFSM4IKG6JCQ>
.
|
|
An idea would be to define a test server app with a catch-all event handler which writes all messages it receives to a list, and a helper to wait for a new message to come in which could also return that new message. |
|
Would it be possible for one of you guys that have been able to make this work to list the versions of the libraries/dependencies you're using? I attempted to replicate what's discussed here (#332 (comment)) using windows, but I keep having issues and I'm wondering if it's related to my python version or dependencies's. Thanks ! Update: INFO: Shutting down
INFO: Waiting for background tasks to complete. (CTRL+C to force quit)for a whole minute before going on with the next one. |
|
Put your code up somewhere so we can have a look 👍 |
|
Hi. Sorry for the late answer.
Of course, here we go: (I just put the uvicorn deps on the top and skipped the mypy and jupyterlab dependencies which are very long...)
I have no "Waiting for background tasks to complete." message. Running Searching inside the # Wait for existing tasks to complete.
if self.server_state.tasks and not self.force_exit:
msg = "Waiting for background tasks to complete. (CTRL+C to force quit)"
logger.info(msg)
while self.server_state.tasks and not self.force_exit:
await asyncio.sleep(0.1)It seems that I don't have a background task, but you do. What version of |
|
I came across this thread as I am also working on some tests in a project that uses python-socketio. To help, I have been trying to replicate the tests shared by @erny, but I'm having issues adapting them. I was hoping someone could take a look at this and provide some feedback? Also looping in @miguelgrinberg and @Korijn to see if you have any ideas! import socketio
import asyncio
import pytest
import pytest_asyncio
import uvicorn
import typing as t
class UvicornTestServer(uvicorn.Server):
def __init__(self, app, host: str = "127.0.0.1", port: int = 8000):
"""Create a Uvicorn test server
Args:
app: the ASGI app
host (str, optional): the host ip. Defaults to '127.0.0.1'.
port (int, optional): the port. Defaults to PORT.
"""
self._startup_done = asyncio.Event()
self._serve_task: t.Awaitable[t.Any] | None = None
super().__init__(config=uvicorn.Config(app, host=host, port=port))
async def startup(self, sockets: list[t.Any] | None = None) -> None:
"""Override uvicorn startup"""
await super().startup(sockets=sockets)
self.config.setup_event_loop()
self._startup_done.set()
async def up(self) -> None:
"""Start up server asynchronously"""
self._serve_task = asyncio.create_task(self.serve())
await self._startup_done.wait()
async def down(self) -> None:
"""Shut down server asynchronously"""
self.should_exit = True
if self._serve_task:
await self._serve_task
@pytest.fixture()
def sio() -> socketio.AsyncServer:
"""Create a Socket.IO server."""
sio = socketio.AsyncServer(
async_mode="asgi",
namespaces=["/chat"],
logger=True,
engineio_logger=True,
)
@sio.on("chat message", namespace="/chat")
async def chat_message(sid, msg):
"""Receive a chat message and send to all clients"""
print(f"Server received: {msg}")
await sio.emit("chat message", msg, namespace="/chat")
return sio
@pytest_asyncio.fixture(autouse=True)
async def startup_and_shutdown_server(
sio: socketio.AsyncServer,
) -> t.AsyncIterator[None]:
"""Start server as test fixture and tear down after test"""
server = UvicornTestServer(socketio.ASGIApp(sio, socketio_path="/socket.io"))
await server.up()
yield
await server.down()
@pytest_asyncio.fixture
async def client() -> t.AsyncIterator[socketio.AsyncClient]:
"""Create a Socket.IO client."""
client = socketio.AsyncClient(logger=True, engineio_logger=True, ssl_verify=False)
await client.connect("http://localhost:8000", namespaces=["/chat"])
yield client
# await client.wait()
# await client.disconnect()
@pytest.mark.asyncio
async def test_emit_event_from_client2(
client: socketio.AsyncClient,
) -> None:
"""Test emitting a message."""
future = asyncio.get_running_loop().create_future()
@client.on("chat message", namespace="/chat")
def on_message_received(data):
print(f"Client received: {data}")
# set the result
future.set_result(data)
message = "Hello!"
print(f"Client sends: {message}")
await client.emit("chat message", message, namespace="/chat")
# wait for the result to be set (avoid waiting forever)
await asyncio.wait_for(future, timeout=1.0)
await client.disconnect()
assert future.result() == messageHere are the logs |
|
@leverage-analytics My only suggestion would be to remove the pytest fixtures from the equation and instead try to make it work without all the magical stuff that makes fixtures work, and only then introduce fixtures one change at a time. I have shared working code above that does not use pytest that you can use as a start. |
|
@miguelgrinberg thanks for the input! I appreciate your time responding and maintaining the project. I will update when I have a chance. |
|
I resolved my issue! I reproduced both @miguelgrinberg's example in a fresh virtual environment, and it worked. I then attempted to run the same example using the virtual environment for my package (to check if there was a problem with dependency versions), and it failed. After some careful comparison of the installed packages across the two virtual environments, I found that uvloop was installed in my virtual environment (as a dependency of uvicorn), and uvicorn, as of version 0.34.0, defaults to an event loop provided by uvloop (if installed) instead of the default asyncio loop. I realized then there must have been a discrepancy between uvloop's handling of the loop compared to asyncio. Thankfully, the Edit: This example prints error Here is my updated example, including pytest and pytest-asyncio fixtures, for anyone who might be interested in the future! import asyncio
import typing as t
import pytest
import pytest_asyncio
import socketio
import uvicorn
class UvicornTestServer(uvicorn.Server):
def __init__(self, app, host: str = "127.0.0.1", port: int = 8000):
self._startup_done = asyncio.Event()
self._serve_task: t.Awaitable[t.Any] | None = None
super().__init__(
config=uvicorn.Config(app, host=host, port=port, loop="asyncio")
)
async def startup(self, sockets: list[t.Any] | None = None) -> None:
"""Override uvicorn startup."""
await super().startup(sockets=sockets)
self.config.setup_event_loop()
self._startup_done.set()
async def up(self) -> None:
"""Start up server asynchronously."""
self._serve_task = asyncio.create_task(self.serve())
await self._startup_done.wait()
async def down(self) -> None:
"""Shut down server asynchronously."""
self.should_exit = True
if self._serve_task:
await self._serve_task
@pytest_asyncio.fixture
async def sio() -> socketio.AsyncServer:
"""Create a Socket.IO server."""
sio = socketio.AsyncServer(
async_mode="asgi",
monitor_clients=False,
logger=True,
engineio_logger=True,
)
@sio.on("double")
async def double(sid, data):
print(f"doubling for {sid}")
return "DOUBLED:" + data * 2
return sio
@pytest_asyncio.fixture()
async def startup_and_shutdown_server(
sio: socketio.AsyncServer,
) -> t.AsyncIterator[None]:
"""Start server as test fixture and tear down after test."""
server = UvicornTestServer(socketio.ASGIApp(sio))
await server.up()
yield
await server.down()
tasks = [
task
for task in asyncio.all_tasks(loop=asyncio.get_running_loop())
if not task.done()
]
for task in tasks:
task.cancel()
try:
await asyncio.wait(tasks)
except asyncio.CancelledError:
# If the task was cancelled, it will raise a CancelledError
# We can ignore this error as it is expected
pass
@pytest_asyncio.fixture
async def client() -> t.AsyncIterator[socketio.AsyncClient]:
"""Create a Socket.IO client."""
client = socketio.AsyncClient(logger=True, engineio_logger=True)
await client.connect("http://localhost:8000")
yield client
@pytest.mark.asyncio
@pytest.mark.usefixtures("startup_and_shutdown_server")
async def test_client(client: socketio.AsyncClient) -> None:
result = await client.call("double", "hello")
assert result == "DOUBLED:hellohello"
await client.disconnect()@miguelgrinberg Thanks again for your help! |
Hello and thanks for the library!
I'm using it with Starlette and trying to implement some integration test. Is there a test client for socketio similar to the one they provide for the basic HTTP/websocket (here), or examples about how to implement such a test?
The text was updated successfully, but these errors were encountered: