-
-
Notifications
You must be signed in to change notification settings - Fork 158
Update of 5.6 to 5.7.1 causes lots of Timeout in include queries #1731
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Let me provide some more Context. JsonApiDotNetCore: 5.6.0 Query String of Api Call: filter=and(any(id,'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX','XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX','XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX',XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX','XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'),equals(isDeleted,'false'))&include=products.unitGroup.units Execution Time in Postman: 2.1s Logs:
After Version Update without changing anything else... Logs:
Please Notice that even EF Core has now provided new warnings after the update and If I were to show you the full query you could see that it trippled in its size... Since there is no EF Core update that our team did can you help us explain this? |
Are you using resource inheritance? It would help to share the before/after SQL for the request URL. |
Yes we are. products is a base resource with 5 or 6 children Resources |
@bkoelman Here is the generated Query in version 5.6.0:
Here is the generate query in version 5.7.1:
|
That looks awful :) Possibly caused by #1641. Can you try turning off pagination in options and see if that helps? Another thing to inspect (with pagination enabled) would be to compare |
@bkoelman Tried to apply your idea of turning off Pagination by setting DefaultPageSize = null, which wouldn't apply much in our scenarios since we are using this api to fill client Datagrids that use pagination. The change did affect the query but the end result is the same, in other words still produces a Timeout...
Also here is the QueryLayer.ToString() when pagination is not set to null but also is not specifically set in the query string. It appears to be correct in my opinion and similar to logs we were originally getting and we sent in previous messages
|
@bkoelman since it appears to be a bug of 5.7.0 and 5.7.1 versions may I suggest turning this issue into a bug and handling it this was? We have paused our version update until this is resolved. Please let us know if we can provide any further insight or assistance on resolving this since this is block us as well. |
I'm surprised by that query layer (assuming it originates from JADNC v5.7.1). I would have expected to see multiple field selectors (one per derived type), with pagination inside them. But it also has no includes, so it doesn't seem to represent the original problematic request with both a filter and includes. The initial log file contains "HTTP GET /v1/priceGroups", which doesn't have query string parameters, but they may have been trimmed off in logs for redaction. Can you confirm that the I don't know yet whether this is a bug or intended behavior. That doesn't mean I won't try to help you, but I need to understand the root cause first. Based on that, maybe JADNC can provide extra configuration settings to prevent this from happening, assuming the query becomes too complex for SQL Server to handle. But I'm just guessing now. It's hard for me to track what's happening, ie. I don't know the data model and its inheritance hierarchy. Would it be possible to create a minimal repro that I can debug, in the form of a git repo? Then I can analyze the query layer, the resulting LINQ expression, and its SQL, and compare those with the old version of JADNC. I also wonder if the problem is specific to SQL Server. Seeding the minimal repro with sample data should be easy with Bogus Faker, which we already use for integration tests.
Based on the conversations at dotnet/efcore#29182 and dotnet/efcore#29665, this warning may not be appropriate; the "solution" to activate query splitting may actually make the situation worse. I recommend ignoring this warning while trying to identify the root cause of the problem. |
Well, I suspect the printed I've merged a PR that logs both See the instructions at #1732 for how to activate it. It should become available shortly for download using https://github.com/json-api-dotnet/JsonApiDotNetCore?tab=readme-ov-file#trying-out-the-latest-build. |
@bkoelman here is a dummy repo that demonstrates issue: https://github.com/JohnStrim/JsonApiBugReport/tree/master |
SUMMARY
Update to this version has produced many Timeouts in EF Core
DETAILS
In our logic we are extensively using query strings for filtering, including relationships and paging which in previous versions used to perform very well (Generally under 1 sec).
When updating to version 5.7.1 we started having a lot of Timeout in Db Queries. To provide more context we were trying to include relationships that are 2 or 3 level deep from our original resource: (ex. include=products.unitGroup,products.unitGroup.units)
Also we noticed a big delay even in simple GET queries which used to return result under 200ms and now performance has up to 600ms with no appart explanation.
We've tried to do an EF Performance Update: (Include the UseQuerySplittingBehavior(QuerySplittingBehavior.SplitQuery)) but this seems to causes many Data is Null Exceptions when including relationships.
Is there any suggestions as far as configuration or minimum versions that we should be aware of to avoid this overhead?
VERSIONS USED
The text was updated successfully, but these errors were encountered: