Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: redis/redis-py
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v5.1.1
Choose a base ref
...
head repository: redis/redis-py
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v5.2.0
Choose a head ref
  • 6 commits
  • 22 files changed
  • 4 contributors

Commits on Oct 7, 2024

  1. Backport 5.1 changes into master (#3406)

    * Updated package version (#3389)
    
    * Fix bug with Redis Set commands returns List instead of Set (#3399)
    
    * Fix bug with Redis Set commands returns List instead of Set in RESP2
    
    * Removed fixture, codestyle fixes
    
    * Fixed tests for async
    
    * Fixed asyncio cluster test cases
    
    * Added Sets alignment for RESP2 and RESP3
    
    * Updated doctests
    
    * Fix bug with partial Hiredis availability (#3400)
    
    * Fix bug with partial Hiredis availability
    
    * Added yes flag
    
    * Codestyl fixes
    
    * Removed redundant check
    
    * Removed redundant checks associated with pack command
    
    * Updated condition to check the actual flag
    
    * Removed unused import
    
    * Fix bug with async pipeline and cluster fails with some commands (#3402)
    
    * Fix bug with async pipeline fails with some commands
    
    * Codestyl changes
    
    * Remove keys option in cluster context
    
    * Updated package version (#3403)
    vladvildanov authored Oct 7, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    700045c View commit details

Commits on Oct 9, 2024

  1. DOC-4200: add TCEs to the aggregation query page (#3381)

    dwdougherty authored Oct 9, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    17db62e View commit details

Commits on Oct 16, 2024

  1. DOC-4199: add TCEs to the combined query page (#3380)

    Co-authored-by: Vladyslav Vildanov <[email protected]>
    dwdougherty and vladvildanov authored Oct 16, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    d6ddb0d View commit details

Commits on Oct 21, 2024

  1. Pin pytest-profiling (#3417)

    uglide authored Oct 21, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    4c4d4af View commit details

Commits on Oct 22, 2024

  1. adds scorer to AggregateRequest (#3409)

    * adds scorer to AggregateRequest
    
    * fix linting
    
    * update tests for BM25
    
    * enum for aggregation scorer
    
    * update signature
    
    * revert back to string input
    
    ---------
    
    Co-authored-by: Vladyslav Vildanov <[email protected]>
    rbs333 and vladvildanov authored Oct 22, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    00f5be4 View commit details

Commits on Oct 24, 2024

  1. Updated package version (#3418)

    vladvildanov authored Oct 24, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    166ae7b View commit details
33 changes: 33 additions & 0 deletions .github/workflows/integration.yaml
Original file line number Diff line number Diff line change
@@ -90,6 +90,21 @@ jobs:
invoke ${{matrix.test-type}}-tests
ls -1
- name: Run tests against hiredis < 3.0.0
if: ${{ matrix.connection-type == 'hiredis' && matrix.python-version == '3.12'}}
run: |
pip uninstall hiredis -y
pip install -U setuptools wheel
pip install -r requirements.txt
pip install -r dev_requirements.txt
if [ "${{matrix.connection-type}}" == "hiredis" ]; then
pip install "hiredis<3.0.0"
fi
invoke devenv
sleep 10 # time to settle
invoke ${{matrix.test-type}}-tests
ls -1
- name: Upload test results and profiling data
uses: actions/upload-artifact@v4
with:
@@ -145,6 +160,24 @@ jobs:
invoke ${{matrix.test-type}}-tests --protocol=3
fi
- name: Run tests against hiredis < 3.0.0
if: ${{ matrix.connection-type == 'hiredis' && matrix.python-version == '3.12'}}
run: |
pip uninstall hiredis -y
pip install -U setuptools wheel
pip install -r requirements.txt
pip install -r dev_requirements.txt
if [ "${{matrix.connection-type}}" == "hiredis" ]; then
pip install "hiredis<3.0.0"
fi
invoke devenv
sleep 10 # time to settle
if [ "${{matrix.event-loop}}" == "uvloop" ]; then
invoke ${{matrix.test-type}}-tests --uvloop --protocol=3
else
invoke ${{matrix.test-type}}-tests --protocol=3
fi
- name: Upload test results and profiling data
uses: actions/upload-artifact@v4
with:
2 changes: 1 addition & 1 deletion dev_requirements.txt
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ packaging>=20.4
pytest
pytest-asyncio>=0.23.0,<0.24.0
pytest-cov
pytest-profiling
pytest-profiling==1.7.0
pytest-timeout
ujson>=4.2.0
uvloop
34 changes: 17 additions & 17 deletions doctests/dt_set.py
Original file line number Diff line number Diff line change
@@ -58,11 +58,11 @@
r.sadd("bikes:racing:usa", "bike:1", "bike:4")
# HIDE_END
res7 = r.sinter("bikes:racing:france", "bikes:racing:usa")
print(res7) # >>> ['bike:1']
print(res7) # >>> {'bike:1'}
# STEP_END

# REMOVE_START
assert res7 == ["bike:1"]
assert res7 == {"bike:1"}
# REMOVE_END

# STEP_START scard
@@ -83,12 +83,12 @@
print(res9) # >>> 3

res10 = r.smembers("bikes:racing:france")
print(res10) # >>> ['bike:1', 'bike:2', 'bike:3']
print(res10) # >>> {'bike:1', 'bike:2', 'bike:3'}
# STEP_END

# REMOVE_START
assert res9 == 3
assert res10 == ['bike:1', 'bike:2', 'bike:3']
assert res10 == {'bike:1', 'bike:2', 'bike:3'}
# REMOVE_END

# STEP_START smismember
@@ -109,11 +109,11 @@
r.sadd("bikes:racing:usa", "bike:1", "bike:4")

res13 = r.sdiff("bikes:racing:france", "bikes:racing:usa")
print(res13) # >>> ['bike:2', 'bike:3']
print(res13) # >>> {'bike:2', 'bike:3'}
# STEP_END

# REMOVE_START
assert res13 == ['bike:2', 'bike:3']
assert res13 == {'bike:2', 'bike:3'}
r.delete("bikes:racing:france")
r.delete("bikes:racing:usa")
# REMOVE_END
@@ -124,27 +124,27 @@
r.sadd("bikes:racing:italy", "bike:1", "bike:2", "bike:3", "bike:4")

res13 = r.sinter("bikes:racing:france", "bikes:racing:usa", "bikes:racing:italy")
print(res13) # >>> ['bike:1']
print(res13) # >>> {'bike:1'}

res14 = r.sunion("bikes:racing:france", "bikes:racing:usa", "bikes:racing:italy")
print(res14) # >>> ['bike:1', 'bike:2', 'bike:3', 'bike:4']
print(res14) # >>> {'bike:1', 'bike:2', 'bike:3', 'bike:4'}

res15 = r.sdiff("bikes:racing:france", "bikes:racing:usa", "bikes:racing:italy")
print(res15) # >>> []
print(res15) # >>> {}

res16 = r.sdiff("bikes:racing:usa", "bikes:racing:france")
print(res16) # >>> ['bike:4']
print(res16) # >>> {'bike:4'}

res17 = r.sdiff("bikes:racing:france", "bikes:racing:usa")
print(res17) # >>> ['bike:2', 'bike:3']
print(res17) # >>> {'bike:2', 'bike:3'}
# STEP_END

# REMOVE_START
assert res13 == ['bike:1']
assert res14 == ['bike:1', 'bike:2', 'bike:3', 'bike:4']
assert res15 == []
assert res16 == ['bike:4']
assert res17 == ['bike:2', 'bike:3']
assert res13 == {'bike:1'}
assert res14 == {'bike:1', 'bike:2', 'bike:3', 'bike:4'}
assert res15 == {}
assert res16 == {'bike:4'}
assert res17 == {'bike:2', 'bike:3'}
r.delete("bikes:racing:france")
r.delete("bikes:racing:usa")
r.delete("bikes:racing:italy")
@@ -160,7 +160,7 @@
print(res19) # >>> bike:3

res20 = r.smembers("bikes:racing:france")
print(res20) # >>> ['bike:2', 'bike:4', 'bike:5']
print(res20) # >>> {'bike:2', 'bike:4', 'bike:5'}

res21 = r.srandmember("bikes:racing:france")
print(res21) # >>> bike:4
103 changes: 103 additions & 0 deletions doctests/query_agg.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# EXAMPLE: query_agg
# HIDE_START
import json
import redis
from redis.commands.json.path import Path
from redis.commands.search import Search
from redis.commands.search.aggregation import AggregateRequest
from redis.commands.search.field import NumericField, TagField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType
import redis.commands.search.reducers as reducers

r = redis.Redis(decode_responses=True)

# create index
schema = (
TagField("$.condition", as_name="condition"),
NumericField("$.price", as_name="price"),
)

index = r.ft("idx:bicycle")
index.create_index(
schema,
definition=IndexDefinition(prefix=["bicycle:"], index_type=IndexType.JSON),
)

# load data
with open("data/query_em.json") as f:
bicycles = json.load(f)

pipeline = r.pipeline(transaction=False)
for bid, bicycle in enumerate(bicycles):
pipeline.json().set(f'bicycle:{bid}', Path.root_path(), bicycle)
pipeline.execute()
# HIDE_END

# STEP_START agg1
search = Search(r, index_name="idx:bicycle")
aggregate_request = AggregateRequest(query='@condition:{new}') \
.load('__key', 'price') \
.apply(discounted='@price - (@price * 0.1)')
res = search.aggregate(aggregate_request)
print(len(res.rows)) # >>> 5
print(res.rows) # >>> [['__key', 'bicycle:0', ...
#[['__key', 'bicycle:0', 'price', '270', 'discounted', '243'],
# ['__key', 'bicycle:5', 'price', '810', 'discounted', '729'],
# ['__key', 'bicycle:6', 'price', '2300', 'discounted', '2070'],
# ['__key', 'bicycle:7', 'price', '430', 'discounted', '387'],
# ['__key', 'bicycle:8', 'price', '1200', 'discounted', '1080']]
# REMOVE_START
assert len(res.rows) == 5
# REMOVE_END
# STEP_END

# STEP_START agg2
search = Search(r, index_name="idx:bicycle")
aggregate_request = AggregateRequest(query='*') \
.load('price') \
.apply(price_category='@price<1000') \
.group_by('@condition', reducers.sum('@price_category').alias('num_affordable'))
res = search.aggregate(aggregate_request)
print(len(res.rows)) # >>> 3
print(res.rows) # >>>
#[['condition', 'refurbished', 'num_affordable', '1'],
# ['condition', 'used', 'num_affordable', '1'],
# ['condition', 'new', 'num_affordable', '3']]
# REMOVE_START
assert len(res.rows) == 3
# REMOVE_END
# STEP_END

# STEP_START agg3
search = Search(r, index_name="idx:bicycle")
aggregate_request = AggregateRequest(query='*') \
.apply(type="'bicycle'") \
.group_by('@type', reducers.count().alias('num_total'))
res = search.aggregate(aggregate_request)
print(len(res.rows)) # >>> 1
print(res.rows) # >>> [['type', 'bicycle', 'num_total', '10']]
# REMOVE_START
assert len(res.rows) == 1
# REMOVE_END
# STEP_END

# STEP_START agg4
search = Search(r, index_name="idx:bicycle")
aggregate_request = AggregateRequest(query='*') \
.load('__key') \
.group_by('@condition', reducers.tolist('__key').alias('bicycles'))
res = search.aggregate(aggregate_request)
print(len(res.rows)) # >>> 3
print(res.rows) # >>>
#[['condition', 'refurbished', 'bicycles', ['bicycle:9']],
# ['condition', 'used', 'bicycles', ['bicycle:1', 'bicycle:2', 'bicycle:3', 'bicycle:4']],
# ['condition', 'new', 'bicycles', ['bicycle:5', 'bicycle:6', 'bicycle:7', 'bicycle:0', 'bicycle:8']]]
# REMOVE_START
assert len(res.rows) == 3
# REMOVE_END
# STEP_END

# REMOVE_START
# destroy index and data
r.ft("idx:bicycle").dropindex(delete_documents=True)
# REMOVE_END
124 changes: 124 additions & 0 deletions doctests/query_combined.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# EXAMPLE: query_combined
# HIDE_START
import json
import numpy as np
import redis
import warnings
from redis.commands.json.path import Path
from redis.commands.search.field import NumericField, TagField, TextField, VectorField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType
from redis.commands.search.query import Query
from sentence_transformers import SentenceTransformer


def embed_text(model, text):
return np.array(model.encode(text)).astype(np.float32).tobytes()

warnings.filterwarnings("ignore", category=FutureWarning, message=r".*clean_up_tokenization_spaces.*")
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
query = "Bike for small kids"
query_vector = embed_text(model, query)

r = redis.Redis(decode_responses=True)

# create index
schema = (
TextField("$.description", no_stem=True, as_name="model"),
TagField("$.condition", as_name="condition"),
NumericField("$.price", as_name="price"),
VectorField(
"$.description_embeddings",
"FLAT",
{
"TYPE": "FLOAT32",
"DIM": 384,
"DISTANCE_METRIC": "COSINE",
},
as_name="vector",
),
)

index = r.ft("idx:bicycle")
index.create_index(
schema,
definition=IndexDefinition(prefix=["bicycle:"], index_type=IndexType.JSON),
)

# load data
with open("data/query_vector.json") as f:
bicycles = json.load(f)

pipeline = r.pipeline(transaction=False)
for bid, bicycle in enumerate(bicycles):
pipeline.json().set(f'bicycle:{bid}', Path.root_path(), bicycle)
pipeline.execute()
# HIDE_END

# STEP_START combined1
q = Query("@price:[500 1000] @condition:{new}")
res = index.search(q)
print(res.total) # >>> 1
# REMOVE_START
assert res.total == 1
# REMOVE_END
# STEP_END

# STEP_START combined2
q = Query("kids @price:[500 1000] @condition:{used}")
res = index.search(q)
print(res.total) # >>> 1
# REMOVE_START
assert res.total == 1
# REMOVE_END
# STEP_END

# STEP_START combined3
q = Query("(kids | small) @condition:{used}")
res = index.search(q)
print(res.total) # >>> 2
# REMOVE_START
assert res.total == 2
# REMOVE_END
# STEP_END

# STEP_START combined4
q = Query("@description:(kids | small) @condition:{used}")
res = index.search(q)
print(res.total) # >>> 0
# REMOVE_START
assert res.total == 0
# REMOVE_END
# STEP_END

# STEP_START combined5
q = Query("@description:(kids | small) @condition:{new | used}")
res = index.search(q)
print(res.total) # >>> 0
# REMOVE_START
assert res.total == 0
# REMOVE_END
# STEP_END

# STEP_START combined6
q = Query("@price:[500 1000] -@condition:{new}")
res = index.search(q)
print(res.total) # >>> 2
# REMOVE_START
assert res.total == 2
# REMOVE_END
# STEP_END

# STEP_START combined7
q = Query("(@price:[500 1000] -@condition:{new})=>[KNN 3 @vector $query_vector]").dialect(2)
# put query string here
res = index.search(q,{ 'query_vector': query_vector })
print(res.total) # >>> 2
# REMOVE_START
assert res.total == 2
# REMOVE_END
# STEP_END

# REMOVE_START
# destroy index and data
r.ft("idx:bicycle").dropindex(delete_documents=True)
# REMOVE_END
6 changes: 6 additions & 0 deletions redis/_parsers/helpers.py
Original file line number Diff line number Diff line change
@@ -785,6 +785,9 @@ def string_keys_to_dict(key_string, callback):


_RedisCallbacksRESP2 = {
**string_keys_to_dict(
"SDIFF SINTER SMEMBERS SUNION", lambda r: r and set(r) or set()
),
**string_keys_to_dict(
"ZDIFF ZINTER ZPOPMAX ZPOPMIN ZRANGE ZRANGEBYSCORE ZRANK ZREVRANGE "
"ZREVRANGEBYSCORE ZREVRANK ZUNION",
@@ -829,6 +832,9 @@ def string_keys_to_dict(key_string, callback):


_RedisCallbacksRESP3 = {
**string_keys_to_dict(
"SDIFF SINTER SMEMBERS SUNION", lambda r: r and set(r) or set()
),
**string_keys_to_dict(
"ZRANGE ZINTER ZPOPMAX ZPOPMIN ZRANGEBYSCORE ZREVRANGE ZREVRANGEBYSCORE "
"ZUNION HGETALL XREADGROUP",
Loading