Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: aquasecurity/esquery
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: deliveroo/os-query
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Able to merge. These branches can be automatically merged.
  • 17 commits
  • 53 files changed
  • 3 contributors

Commits on Sep 3, 2023

  1. adds log ranking

    alekseizorin committed Sep 3, 2023
    Copy the full SHA
    14c7166 View commit details

Commits on Sep 4, 2023

  1. Merge pull request #1 from deliveroo/ranking

    adds log ranking
    alekseizorin authored Sep 4, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    3b201c0 View commit details
  2. fix: ranking query

    alekseizorin committed Sep 4, 2023
    Copy the full SHA
    1923ac7 View commit details
  3. Merge pull request #2 from deliveroo/ranking

    fix: ranking query
    alekseizorin authored Sep 4, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    8c829ba View commit details

Commits on Sep 7, 2023

  1. Copy the full SHA
    36a27dd View commit details
  2. Copy the full SHA
    7ace685 View commit details
  3. Merge pull request #3 from deliveroo/modules

    es -> opensearch packages
    alekseizorin authored Sep 7, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    fb1fd68 View commit details

Commits on Sep 8, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    cb5e56d View commit details

Commits on Oct 12, 2023

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    d0f0612 View commit details
  2. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    bd6436a View commit details

Commits on Oct 16, 2023

  1. doc: cleanup (#7)

    alekseizorin authored Oct 16, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    e0f05d9 View commit details

Commits on Nov 30, 2023

  1. doc: codeowners (#8)

    alekseizorin authored Nov 30, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    6beb440 View commit details

Commits on Apr 15, 2024

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    dfb993a View commit details

Commits on May 2, 2024

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    91b3efc View commit details

Commits on Aug 27, 2024

  1. CBB-1170: add collapse support (#13)

    * CBB-1170: add collapse support
    
    * CBB-1170: add constructor
    mikesmitharoo authored Aug 27, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    b248120 View commit details
  2. CBB-1079: add source disable (#12)

    * CBB-1079: add source disable
    
    * CBB-1170: remove param for source disable
    mikesmitharoo authored Aug 27, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    274a03a View commit details

Commits on Sep 5, 2024

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    1783ebb View commit details
79 changes: 79 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
version: 2.1

commands:
configure_github_access:
steps:
- run:
name: Configure github access
command: git config --global url."https://${GITHUB_USERNAME}:${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"

restore_build_cache:
steps:
- restore_cache:
keys:
- v1.21-go-build-cache-{{ .Environment.CIRCLE_JOB }}-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}
- v1.21-go-build-cache-{{ .Environment.CIRCLE_JOB }}-{{ .Environment.CIRCLE_BRANCH }}
- v1.21-go-build-cache-{{ .Environment.CIRCLE_JOB }}-main
- v1.21-go-build-cache-{{ .Environment.CIRCLE_JOB }}-

save_build_cache:
steps:
- save_cache:
key: v1.21-go-build-cache-{{ .Environment.CIRCLE_JOB }}-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}
paths:
- /mnt/ramdisk/.cache/go-build
when: always

global_context: &global_context
context:
- org-global
- org-datadog-credentials
- GITHUB_CREDS

global_dockerhub_auth: &global_dockerhub_auth
auth:
username: $GLOBAL_DOCKERHUB_USERNAME
password: $GLOBAL_DOCKERHUB_PASSWORD

executors:
test-container:
docker:
- image: cimg/go:1.21
<<: *global_dockerhub_auth
environment:
GOCACHE: /mnt/ramdisk/.cache/go-build
TEST_RESULTS: /tmp/test-results

test_steps: &test_steps
steps:
- checkout
- restore_build_cache
- configure_github_access
- run:
name: Run tests
command: make test-ci
- store_artifacts:
path: /tmp/test-results
destination: raw-test-output
- store_artifacts:
path: /tmp/artifacts
- store_test_results:
path: /tmp/test-results
- persist_to_workspace:
root: /tmp/artifacts
paths:
- test_coverage_stats
- save_build_cache

jobs:
test:
executor: test-container
resource_class: small
<<: *test_steps

workflows:
version: 2
build_and_push:
jobs:
- test:
<<: *global_context
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @deliveroo/consumer-search
11 changes: 11 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
- package-ecosystem: "" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
49 changes: 49 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: "CodeQL - Minimal incremental analysis"

on:
push:
branches:
- "main" # Change this to the branch to default branch
- "!ignore/branch" # Ignore CodeQL scan for these branches
- "!test/*" # Ignore CodeQL scan for these branches
paths-ignore:
- "**/*.md"
- "**/*.txt"
pull_request:
branches:
- "main" # Change this to the branch to default branch
# If your project is not actively developed, consider scheduling CodeQL scans
#schedule:
# - cron: '44 23 * * 5' # Run CodeQL scan every Friday at 11:44 PM UTC

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
timeout-minutes: 30 # Set timeout to 30 minutes; Change if your project takes longer to scan
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: ["go"]

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# queries: security-extended,security-and-quality
# debug: true # Only use this for debugging. It will increase the runtime of the action and take up storage

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
Empty file.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
language: go
go:
- 1.13.x
- 1.21.x

script:
- go test -v -race ./...
34 changes: 34 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Go related variables.
GOBASE=$(shell pwd)
export GOBIN=$(GOBASE)/bin
export GOPRIVATE=github.com/deliveroo/*
export GOPROXY=https://proxy.golang.org,off

# Ensure that we use vendored binaries before consulting the system.
GOBIN=$(shell pwd)/bin
export PATH := $(GOBIN):$(PATH)

MODULE = $(shell env GO111MODULE=on go list -m)

# allows passing specific tags to go build (for example musl)
ifdef GO_BUILD_TAGS
GO_BUILD_TAGS_ARG += -tags $(GO_BUILD_TAGS)
endif

gocoverstats=$(GOBIN)/gocoverstats
$(gocoverstats):
GOBIN=$(GOBIN) go install $(GO_BUILD_TAGS_ARG) gitlab.com/fgmarand/gocoverstats@latest

.PHONY: test
test: ## Run tests
APP_ENV=test go test -race ./...

.PHONY: test-ci
test-ci: $(go-junit-report) $(gocoverstats) ## Run tests and output as junit-xml
mkdir -p /tmp/artifacts
mkdir -p /tmp/test-results
touch /tmp/test-results/go-test.out
trap "$(GOBIN)/go-junit-report </tmp/test-results/go-test.out > /tmp/test-results/go-test-report.xml" EXIT; \
APP_ENV=test go test ${GO_TEST_ARGS} $(GO_BUILD_TAGS_ARG) -coverprofile=/tmp/artifacts/coverage.txt -race ./... 2>&1 | tee /tmp/test-results/go-test.out
$(GOBIN)/gocoverstats -f /tmp/artifacts/coverage.txt > /tmp/artifacts/test_coverage_stats
go tool cover -html=/tmp/artifacts/coverage.txt -o /tmp/artifacts/coverage.html
80 changes: 37 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# esquery
# opensearch query builder

[![](https://img.shields.io/static/v1?label=godoc&message=reference&color=blue&style=flat-square)](https://godoc.org/github.com/aquasecurity/esquery) [![](https://img.shields.io/github/license/aquasecurity/esquery?style=flat-square)](LICENSE) [![Build Status](https://travis-ci.org/aquasecurity/esquery.svg?branch=master)](https://travis-ci.org/aquasecurity/esquery)
[![](https://img.shields.io/github/license/aquasecurity/esquery?style=flat-square)](LICENSE) [![Build Status](https://circleci.com/gh/deliveroo/os-query.svg?branch=main)](https://app.circleci.com/pipelines/github/deliveroo/os-query?branch=main)


**A non-obtrusive, idiomatic and easy-to-use query and aggregation builder for the [official Go client](https://github.com/elastic/go-elasticsearch) for [ElasticSearch](https://www.elastic.co/products/elasticsearch).**
**A non-obtrusive, idiomatic and easy-to-use query and aggregation builder for the [official Opensearch client](https://opensearch.org/docs/latest/clients/go/) for [Opensearch](https://opensearch.org/).**

## Table of Contents

@@ -34,65 +34,59 @@ This is an early release, API may still change.

`esquery` is a Go module. To install, simply run this in your project's root directory:

```bash
go get github.com/aquasecurity/esquery
```

## Usage

esquery provides a [method chaining](https://en.wikipedia.org/wiki/Method_chaining)-style API for building and executing queries and aggregations. It does not wrap the official Go client nor does it require you to change your existing code in order to integrate the library. Queries can be directly built with `esquery`, and executed by passing an `*elasticsearch.Client` instance (with optional search parameters). Results are returned as-is from the official client (e.g. `*esapi.Response` objects).
it provides a [method chaining](https://en.wikipedia.org/wiki/Method_chaining)-style API for building and executing queries and aggregations. It does not wrap the official Go client nor does it require you to change your existing code in order to integrate the library. Queries can be directly built, and executed by passing an `*opensearch.Client` instance (with optional search parameters). Results are returned as-is from the official client (e.g. `*opensearchapi.Response` objects).

Getting started is extremely simple:

```go
package main

import (
"context"
"log"

"github.com/aquasecurity/esquery"
"github.com/elastic/go-elasticsearch/v7"
"github.com/opensearch-project/opensearch-go/v2"

oq "github.com/deliveroo/os-query"
)

func main() {
// connect to an ElasticSearch instance
es, err := elasticsearch.NewDefaultClient()
if err != nil {
log.Fatalf("Failed creating client: %s", err)
}

// run a boolean search query
res, err := esquery.Search().
Query(
esquery.
Bool().
Must(esquery.Term("title", "Go and Stuff")).
Filter(esquery.Term("tag", "tech")),
).
Aggs(
esquery.Avg("average_score", "score"),
esquery.Max("max_score", "score"),
).
Size(20).
Run(
es,
es.Search.WithContext(context.TODO()),
es.Search.WithIndex("test"),
)
if err != nil {
log.Fatalf("Failed searching for stuff: %s", err)
}

defer res.Body.Close()

// ...
// connect to an Opensearch instance
client, err := opensearch.NewDefaultClient()
if err != nil {
log.Fatalf("Failed creating client: %s", err)
}

query := oq.Query(
oq.
Bool().
Must(oq.Term("title", "Go and Stuff")).
Filter(oq.Term("tag", "tech")),
).
Aggs(
oq.Avg("average_score", "score"),
oq.Max("max_score", "score"),
).
Size(20)

queryString, err := query.MarshalJSON()
if err != nil {
log.Fatal("Failed creating query")
}
// run a boolean search query
search := client.Search
res, err := search(search.WithQuery(string(queryString)))
if err != nil {
log.Fatal("Failed executing query")
}

defer res.Body.Close()
}
```

## Notes

* `esquery` currently supports version 7 of the ElasticSearch Go client.
* The library cannot currently generate "short queries". For example, whereas
ElasticSearch can accept this:

2 changes: 1 addition & 1 deletion aggregations_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package esquery
package query

import (
"testing"
7 changes: 4 additions & 3 deletions aggs_bucket.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
package esquery
package query

//----------------------------------------------------------------------------//

// TermsAggregation represents an aggregation of type "terms", as described in
// https://www.elastic.co/guide/en/elasticsearch/reference/current/
// search-aggregations-bucket-terms-aggregation.html
//
// search-aggregations-bucket-terms-aggregation.html
type TermsAggregation struct {
name string
field string
@@ -56,7 +57,7 @@ func (agg *TermsAggregation) Aggs(aggs ...Aggregation) *TermsAggregation {
return agg
}

// Order sets the sort for terms agg
// Order sets the sorts for terms agg
func (agg *TermsAggregation) Order(order map[string]string) *TermsAggregation {
agg.order = order
return agg
2 changes: 1 addition & 1 deletion aggs_filter.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package esquery
package query

type FilterAggregation struct {
name string
2 changes: 1 addition & 1 deletion aggs_filter_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package esquery
package query

import "testing"

Loading