-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Add searchable snapshots integration tests for URL repositories #70709
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add searchable snapshots integration tests for URL repositories #70709
Conversation
Pinging @elastic/es-distributed (Team:Distributed) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this PR! I've left one suggestion which I think will give us more confidence that the client is behaving well.
@@ -18,13 +18,15 @@ | |||
import java.nio.file.Paths; | |||
import java.util.HashMap; | |||
import java.util.Map; | |||
import java.util.regex.Matcher; | |||
import java.util.regex.Pattern; | |||
|
|||
/** | |||
* This {@link URLFixture} exposes a filesystem directory over HTTP. It is used in repository-url | |||
* integration tests to expose a directory created by a regular FS repository. | |||
*/ | |||
public class URLFixture extends AbstractHttpFixture { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a fixture. I was wondering if we could plug in Nginx or some other simple http server to really make sure it works with at least a proper http server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good idea, I'll add a real http server to the test suite 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've left two build-related comments. I think someone from @elastic/es-delivery should have a look here too.
} | ||
|
||
for (repoDir in project.ext.fsRepositoryDirs) { | ||
delete(repoDir.value as File) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's odd that we would delete / recreate the directory at configuration time of the build?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need a clean environment between test runs, I couldn't find a way to run the nginx container with a directory that's created on runtime. Maybe someone from @elastic/es-delivery can help us with that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can delete the directory contents in preprocessfixture
task if necessary. We definitely don't want to do this during configuration time.
setting 'xpack.searchable.snapshot.shared_cache.region_size', '256KB' | ||
} | ||
|
||
tasks.register("urlThirdPartyTest") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't a third-party test (it's running locally on the same machine) so why have this task?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that was a mistake.
@elasticmachine run elasticsearch-ci/eql-correctness |
@elasticmachine update branch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks for adding these tests!
This commit also increases the max number of pooled HTTP connections