Skip to content

Commit 2a12023

Browse files
committed
Expanding docs regardin the storage access
New file: storage-access.mdx Modified python docs and transacctiona docs
1 parent bed635a commit 2a12023

File tree

4 files changed

+103
-6
lines changed

4 files changed

+103
-6
lines changed

pages/client-libraries/python.mdx

+15-2
Original file line numberDiff line numberDiff line change
@@ -435,7 +435,7 @@ for record in records:
435435
print(path.end_node)
436436
```
437437

438-
Path will contain [Nodes](#process-the-node-result) and [Relationships[#process-the-relationship-result], that can be accessed in the same way as in the previous examples.
438+
Path will contain [Nodes](#process-the-node-result) and [Relationships](#process-the-relationship-result), that can be accessed in the same way as in the previous examples.
439439

440440
### Transaction management
441441

@@ -452,6 +452,10 @@ In v2.10, Memgraph added the [multi-tenant support](/database-management/multi-t
452452

453453
The `execute_query()` procedure automatically creates a transaction that can include multiple Cypher statements as a single query. If the transaction fails, the procedure will automatically rerun it.
454454

455+
As of Memgraph version 3.2, queries are categorized as read or write and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
456+
While it is possible to define the read/write mode using `execute_query` it is recommended to use `execute_read` or `execute_write` instead.
457+
If `execute_query` must be used, it is possible to use the `routing_` variable to define the transaction's read/write type.
458+
455459
Bolt protocol specifies additional [metadata](/database-management/query-metadata) that can be sent along with the requested results. Metadata can be divided into two groups: query statistics and notifications.
456460
The query statistics metadata provides query counters that indicate the changes that the **write query** triggered on the server.
457461

@@ -545,6 +549,9 @@ With sessions, you can run:
545549
##### Managed transactions
546550

547551
To create a managed transaction, use `Session.execute_read()` procedure for read queries and `Session.execute_write()` procedure for write queries.
552+
As of Memgraph version 3.2, queries are categorized as read or write and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
553+
An exception will be thrown if the user tries to execute a write query inside a read transaction. See [transaction accessor misalignment](/fundamentals/transactions#transaction-accessor-misalignment) for more details.
554+
548555

549556
```python
550557
def match_user(tx, name):
@@ -581,6 +588,10 @@ To maintain multiple concurrent transactions, use [multiple concurrent sessions]
581588
With explicit transactions, you can get **complete control over transactions**. To begin a transaction, run `Session.begin_transaction()` procedure and to run a transaction, use `Transaction.run()` procedure.
582589
Explicit transactions offer the possibility of explicitly controlling the end of a transaction with `Transaction.commit()`, `Transaction.rollback()` or `Transaction.close()` methods.
583590

591+
As of Memgraph version 3.2, queries are categorized as read or write and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
592+
Explicit transactions can cover a number of individual queries, but storage access is given at the start. For best performance, the user needs to declare whether the transaction should use read or write access.
593+
This can be done by setting the session's `default_access_mode` to `"r"` or `"w"`. This will in turn set the access mode of a transaction created via the `begin_transaction` function. Note that `execute_read` and `execute_write` will override the session's default access.
594+
584595
Use explicit transaction if you need to **distribute Cypher execution across multiple functions for the same transaction** or if you need to **run multiple queries within a single transactions without automatic retries**.
585596

586597
The following example shows how to explicitly control the transaction of changing account balances based on a token transfer:
@@ -610,7 +621,7 @@ def create_users(client, sender, receiver):
610621

611622

612623
def transfer_tokens(client, sender_id, receiver_id, num_of_tokens):
613-
with client.session(database="memgraph") as session:
624+
with client.session(database="memgraph", default_access_mode="w") as session:
614625
tx = session.begin_transaction()
615626

616627
try:
@@ -675,6 +686,8 @@ In the above example, if John's account balance is changed to a number less than
675686

676687
Implicit or auto-commit transactions are the simplest way to run a Cypher query since they won't be automatically retried as with `execute_query()` procedure or managed transactions.
677688
With implicit transactions, you don't have the same control of transaction as with explicit transactions, so they are mostly used for quick prototyping.
689+
As of Memgraph version 3.2, queries are categorized as read or write and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
690+
Access mode is automatically determined when executing single queries through implicit transactions.
678691

679692
To run an implicit transaction, use the `Session.run()` method:
680693

pages/fundamentals/_meta.ts

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ export default {
33
"data-types": "Data types",
44
"data-durability": "Data durability",
55
"indexes": "Indexes",
6+
"storage-access" : "Storage access",
67
"storage-memory-usage": "Storage memory usage",
78
"telemetry": "Telemetry",
89
"transactions": "Transactions",

pages/fundamentals/storage-access.mdx

+59
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
title: Storage access
3+
description: Understand how Memgraph access the storage layer. A detailed resource to optimize multi-client throughput.
4+
---
5+
6+
import { Callout } from 'nextra/components'
7+
8+
# Storage access
9+
10+
The storage (or storage layer) refers to all data associated with the graph itself.
11+
This means vertices, edges, their properties, label and other data.
12+
13+
Queries that are reading or writing to the graph are said to be accessing the storage.
14+
These accesses are mediated through the `storage accessors`.
15+
The accessors guarantee a transactional view and concurrency safety.
16+
17+
## Storage accessors
18+
19+
There are 3 types of accessors:
20+
- **Shared access**: Allows multiple queries to run in parallel, marked as either read or write.
21+
- **Read-only access**: Permits multiple read queries to run in parallel but forbids any write operations or queries requiring unique access.
22+
- **Unique access**: Grants exclusive access to a single query, preventing any other type of access during its execution.
23+
24+
**Shared access** is the most common access granted. Any data oriented Cypher query will use it.
25+
26+
**Read-only access** is currently used only by `CREATE SNAPSHOT` when in ANALYTICAL mode. Using the read-only access guarantees that the snapshot is consistent, while also allowing for other (shared access) read queries to run in parallel.
27+
28+
**Unique access** queries is used by queries that require full control over the storage layer. These are:
29+
- Index queries
30+
- Constraint queries
31+
- TTL setup queries
32+
- Enum setup queries
33+
- `DROP GRAPH` query
34+
- `RECOVER SNAPSHOT` query
35+
36+
### Deducing the accessor type needed
37+
38+
The type is deduced at parsing time automatically.
39+
Read-only and unique accesses are given based on the query type (as described in the previous section).
40+
The shared access needs to additionally mark a query as read or write. This is also done automatically at parse-time.
41+
42+
The only instance where the user needs to explicitly specify the desired shared access type is when creating a managed (explicit) transactions.
43+
These transactions acquire and hold the storage accessor at the start of their execution.
44+
By default a write shared access is taken, but this can limit which queries can run in parallel. For the best performance, it is recommended to mark transactions with the desired access type.
45+
For more details, refer to [Transactions](/fundamentals/transactions).
46+
47+
## Queries that do not require storage access
48+
49+
Queries that do not read or modify any graph data do not need storage access.
50+
These queries are:
51+
- Auth queries
52+
- Multi-tenant queries
53+
- Replication queries
54+
- Show config queries
55+
- Setting queries
56+
- Version queries
57+
- Transaction queue queries
58+
59+
Please note that even if these queries do not access the storage, they still might be accessing a shared resource and could block or throw if called in parallel.

pages/help-center/errors/transactions.mdx

+28-4
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,28 @@ While some client drivers may handle serialization errors by retrying transactio
9999
developers should not rely solely on this mechanism. Always include comprehensive error handling
100100
in your application to address cases where the error persists beyond the retry logic.
101101

102+
## Transaction accessor misalignment
103+
104+
### Error message
105+
106+
1. **Accessor type {} and query type {} are misaligned!**
107+
108+
### Handling transaction timeout
109+
110+
Transactions in Memgraph must acquire the appropriate type of storage access at the start of their execution.
111+
112+
This access can be one of the following types:
113+
- **Shared access**: Allows multiple queries to run in parallel, marked as either read or write.
114+
- **Read-only access**: Permits multiple read queries to run in parallel but forbids any write operations or queries requiring unique access.
115+
- **Unique access**: Grants exclusive access to a single query, preventing any other type of access during its execution.
116+
117+
For more information regarding storage access, please refer to [Storage access](/fundamentals/storage-access).
118+
119+
While single queries can be parsed and the correct type of storage access can be determined automatically by Memgraph, this is not the case for explicit (managed) transactions.
120+
In managed transactions, the database cannot infer the required access type in advance because the transaction's operations are not know at the beginning.
121+
This can lead to storage access misalignment if the requested access type does not match the operations being performed.
122+
123+
See appropriate driver's documentation for more information on how to define transaction's type.
102124

103125
## Transaction timeout
104126

@@ -119,15 +141,17 @@ Here are the [instructions](/configuration/configuration-settings#using-flags-an
119141

120142
Here are the storage access error messages you might encounter:
121143

122-
1. **Cannot access storage, unique access query is running. Try again later.**
144+
1. **Cannot get shared access storage. Try stopping other queries that are running in parallel.**
123145
2. **Cannot get unique access to the storage. Try stopping other queries that are running in parallel.**
146+
3. **Cannot get read only access to the storage. Try stopping other queries that are running in parallel.**
124147

125148
### Understanding storage access timeout
126149

127-
Storage access timeouts occur during query preparation when the query execution engine cannot get the required type of access to the storage. There are two types of storage access:
150+
Storage access timeouts occur during query preparation when the query execution engine cannot get the required type of access to the storage. There are three types of storage access:
128151

129-
- **Shared access**: Multiple queries can have shared access at the same time, but shared access cannot be granted while a query with unique access is running.
130-
- **Unique access**: Only one query can have unique access at a time, and no other query can have any type of access during that period.
152+
- **Shared access**: Multiple queries can have shared access at the same time. These queries are marked with a read or write type, allowing Memgraph to efficiently execute multiple operations in parallel without conflicts.
153+
- **Unique access**: Only one query can have unique access at a time, and no other access type can be granted during that period.
154+
- **Read-only access**: Queries with read-only access allow other read queries to run in parallel but forbid any write operations or unique access queries.
131155

132156
These timeouts prevent worker starvation and database blocking that could occur if queries were to wait indefinitely for storage access.
133157

0 commit comments

Comments
 (0)