You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pages/advanced-algorithms/available-algorithms/community_detection.mdx
+1-1
Original file line number
Diff line number
Diff line change
@@ -78,7 +78,7 @@ the procedure.
78
78
gain in modularity goes below this threshold, a final iteration is performed using the
79
79
`community_alg_threshold` value.
80
80
Valid values are between 0 and 1 (exclusive). This parameter's value should be higher than `community_alg_threshold`.
81
-
-`num_of_threads: integer (default=Half of the system's maximum thread count)` ➡ Specifies the number of threads used for parallel execution in the algorithm's parallelized parts.
81
+
-`num_of_threads: integer (default=half of the system’s available hardware threads)` ➡ Specifies the number of threads used for parallel execution in the algorithm's parallelized parts.
82
82
**Note**: OpenMP (omp) is used for parallelization, so the actual thread usage may depend on system settings and OpenMP configurations.
Copy file name to clipboardExpand all lines: pages/advanced-algorithms/available-algorithms/migrate.mdx
+219
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,7 @@ description: Discover the migration capabilities of Memgraph for efficient trans
6
6
import { Cards } from'nextra/components'
7
7
importGitHubfrom'/components/icons/GitHub'
8
8
import { Steps } from'nextra/components'
9
+
import { Callout } from'nextra/components';
9
10
10
11
# migrate
11
12
@@ -35,6 +36,181 @@ filter, and convert relational data into a graph format.
35
36
36
37
## Procedures
37
38
39
+
### `arrow_flight()`
40
+
41
+
With the `arrow_flight()` procedure, users can access data sources which support the [Arrow Flight RPC protocol](https://arrow.apache.org/docs/format/Flight.html) for transfer
42
+
of large data records to achieve high performance. Underlying implementation is using the `pyarrow` Python library to stream rows to
43
+
Memgraph. [Dremio](https://www.dremio.com/) is a confirmed data source that works with the `arrow_flight()` procedure. Other sources may also be compatible, but Dremio is based on previous experience.
44
+
45
+
{<h4className="custom-header"> Input: </h4>}
46
+
47
+
-`query: str` ➡ Query used to query the data source.
48
+
-`config: mgp.Map` ➡ Connection parameters (as in `pyarrow.flight.connect`). Useful parameters for connecting are `host`, `port`, `username` and `password`.
49
+
-`config_path` ➡ Path to a JSON file containing configuration parameters.
50
+
51
+
{<h4className="custom-header"> Output: </h4>}
52
+
53
+
-`row: mgp.Map` ➡ The result table as a stream of rows.
54
+
55
+
#### Retrieve and inspect data
56
+
```cypher
57
+
CALL migrate.arrow_flight('SELECT * FROM users', {username: 'memgraph',
58
+
password: 'password',
59
+
host: 'localhost',
60
+
port: '12345'} )
61
+
YIELD row
62
+
RETURN row
63
+
LIMIT 5000;
64
+
```
65
+
66
+
#### Filter specific data
67
+
```cypher
68
+
CALL migrate.arrow_flight('SELECT * FROM users', {username: 'memgraph',
69
+
password: 'password',
70
+
host: 'localhost',
71
+
port: '12345'} )
72
+
YIELD row
73
+
WHERE row.age >= 30
74
+
RETURN row;
75
+
```
76
+
77
+
#### Create nodes from migrated data
78
+
```cypher
79
+
CALL migrate.arrow_flight('SELECT id, name, age FROM users', {username: 'memgraph',
CALL migrate.arrow_flight('SELECT user1_id, user2_id FROM friendships', {username: 'memgraph',
90
+
password: 'password',
91
+
host: 'localhost',
92
+
port: '12345'} )
93
+
YIELD row
94
+
MATCH (u1:User {id: row.user1_id}), (u2:User {id: row.user2_id})
95
+
CREATE (u1)-[:FRIENDS_WITH]->(u2);
96
+
```
97
+
98
+
### `duckdb()`
99
+
With the `migrate.duckdb()` procedure, users can connect to the ** DuckDB** database and query various data sources.
100
+
List of data sources that are supported by DuckDB can be found on their [official documentation page](https://duckdb.org/docs/stable/data/data_sources.html).
101
+
The underlying implementation streams results from DuckDB to Memgraph using the `duckdb` Python Library. DuckDB is started with the in-memory mode, without any
102
+
persistence and is used just to proxy to the underlying data sources.
103
+
104
+
{<h4className="custom-header"> Input: </h4>}
105
+
106
+
-`query: str` ➡ Table name or an SQL query.
107
+
-`setup_queries: mgp.Nullable[List[str]]` ➡ List of queries that will be executed prior to the query provided as the initial argument.
108
+
Used for setting up the connection to additional data sources.
109
+
110
+
{<h4className="custom-header"> Output: </h4>}
111
+
112
+
-`row: mgp.Map` ➡ The result table as a stream of rows.
113
+
114
+
{<h4className="custom-header"> Usage: </h4>}
115
+
116
+
#### Retrieve and inspect data
117
+
```cypher
118
+
CALL migrate.duckdb("SELECT * FROM 'test.parquet';")
119
+
YIELD row
120
+
RETURN row
121
+
LIMIT 5000;
122
+
```
123
+
124
+
#### Filter specific data
125
+
```cypher
126
+
CALL migrate.duckdb("SELECT * FROM 'test.parquet';")
127
+
YIELD row
128
+
WHERE row.age >= 30
129
+
RETURN row;
130
+
```
131
+
132
+
#### Create nodes from migrated data
133
+
```cypher
134
+
CALL migrate.duckdb("SELECT * FROM 'test.parquet';")
CALL migrate.duckdb("SELECT * FROM 'test.parquet';")
142
+
YIELD row
143
+
MATCH (u1:User {id: row.user1_id}), (u2:User {id: row.user2_id})
144
+
CREATE (u1)-[:FRIENDS_WITH]->(u2);
145
+
```
146
+
147
+
#### Setup connection to query additional data sources
148
+
```cypher
149
+
CALL migrate.duckdb("SELECT * FROM 's3://your_bucket/your_file.parquet';", ["CREATE SECRET secret1 (TYPE s3, KEY_ID 'key', SECRET 'secret', REGION 'region');"])
150
+
YIELD row
151
+
MATCH (u1:User {id: row.user1_id}), (u2:User {id: row.user2_id})
152
+
CREATE (u1)-[:FRIENDS_WITH]->(u2);
153
+
```
154
+
155
+
---
156
+
157
+
### `memgraph()`
158
+
159
+
With the `migrate.memgraph()` procedure, you can access another Memgraph instance and migrate your data to a new Memgraph instance.
160
+
The resulting nodes and edges are converted into a stream of rows which can include labels, properties, and primitives.
161
+
162
+
<Callouttype="info">
163
+
Streaming of raw node and relationship objects is not supported and users are advised to migrate all the necessary identifiers in order to recreate the same graph in Memgraph.
164
+
</Callout>
165
+
166
+
{<h4className="custom-header"> Input: </h4>}
167
+
168
+
-`label_or_rel_or_query: str` ➡ Label name (written in format `(:Label)`), relationship name (written in format `[:rel_type]`) or a plain cypher query.
169
+
-`config: mgp.Map` ➡ Connection parameters (as in `gqlalchemy.Memgraph`). Notable parameters are `host[String]`, and `port[Integer]`
170
+
-`config_path` ➡ Path to a JSON file containing configuration parameters.
-`row: mgp.Map` ➡ The result table as a stream of rows.
176
+
- when retrieving nodes using the `(:Label)` syntax, row will have the following keys: `labels`, and `properties`
177
+
- when retrieving relationships using the `[:REL_TYPE]` syntax, row will have the following keys: `from_labels`, `to_labels`, `from_properties`, `to_properties`, and `edge_properties`
178
+
- when retrieving results using a plain Cypher query, row will have keys identical to the returned column names from the Cypher query
179
+
180
+
{<h4className="custom-header"> Usage: </h4>}
181
+
182
+
#### Retrieve nodes of certain label and create them in a new Memgraph instance
With the `migrate.servicenow()` procedure, you can access [ServiceNow REST API](https://developer.servicenow.com/dev.do#!/reference/api/xanadu/rest/) and transfer your data to Memgraph.
519
+
The underlying implementation is using the [`requests` Python library] to migrate results to Memgraph. The REST API from
520
+
ServiceNow must provide results in the format `{results: []}` in order for Memgraph to stream it into result rows.
521
+
522
+
{<h4className="custom-header"> Input: </h4>}
523
+
524
+
-`endpoint: str` ➡ ServiceNow endpoint. Users can optionally include their own query parameters to filter results.
525
+
-`config: mgp.Map` ➡ Connection parameters. Notable connection parameters are `username` and `password`, per `requests.get()` method.
526
+
-`config_path: str` ➡ Path to a JSON file containing configuration parameters.
527
+
528
+
{<h4className="custom-header"> Output: </h4>}
529
+
530
+
-`row: mgp.Map` ➡ Each row from the CSV file as a structured dictionary.
531
+
532
+
{<h4className="custom-header"> Usage: </h4>}
533
+
534
+
#### Retrieve and inspect CSV data from ServiceNow
Copy file name to clipboardExpand all lines: pages/client-libraries/python.mdx
+18-2
Original file line number
Diff line number
Diff line change
@@ -435,7 +435,7 @@ for record in records:
435
435
print(path.end_node)
436
436
```
437
437
438
-
Path will contain [Nodes](#process-the-node-result) and [Relationships[#process-the-relationship-result], that can be accessed in the same way as in the previous examples.
438
+
Path will contain [Nodes](#process-the-node-result) and [Relationships](#process-the-relationship-result), that can be accessed in the same way as in the previous examples.
439
439
440
440
### Transaction management
441
441
@@ -546,6 +546,11 @@ With sessions, you can run:
546
546
547
547
To create a managed transaction, use `Session.execute_read()` procedure for read queries and `Session.execute_write()` procedure for write queries.
548
548
549
+
<Callouttype="info">
550
+
As of Memgraph version 3.2, queries are categorized as **read** or **write** and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
551
+
An exception will be thrown if the user tries to execute a write query inside a read transaction. For more details, see [transaction accessor misalignment](/fundamentals/transactions#transaction-accessor-misalignment).
552
+
</Callout>
553
+
549
554
```python
550
555
defmatch_user(tx, name):
551
556
result = tx.run(
@@ -581,6 +586,12 @@ To maintain multiple concurrent transactions, use [multiple concurrent sessions]
581
586
With explicit transactions, you can get **complete control over transactions**. To begin a transaction, run `Session.begin_transaction()` procedure and to run a transaction, use `Transaction.run()` procedure.
582
587
Explicit transactions offer the possibility of explicitly controlling the end of a transaction with `Transaction.commit()`, `Transaction.rollback()` or `Transaction.close()` methods.
583
588
589
+
<Callouttype="info">
590
+
As of Memgraph version 3.2, queries are categorized as **read** or **write** and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
591
+
Explicit transactions can cover a number of individual queries, but storage access is given at the start. For best performance, the user needs to declare whether the transaction should use read or write access.
592
+
This can be done by setting the session's `default_access_mode` to `"r"` or `"w"`. This will, in turn, set the access mode of a transaction created via the `begin_transaction` function. Note that `execute_read` and `execute_write` will override the session's default access.
593
+
</Callout>
594
+
584
595
Use explicit transaction if you need to **distribute Cypher execution across multiple functions for the same transaction** or if you need to **run multiple queries within a single transactions without automatic retries**.
585
596
586
597
The following example shows how to explicitly control the transaction of changing account balances based on a token transfer:
with client.session(database="memgraph") as session:
624
+
with client.session(database="memgraph", default_access_mode="w") as session:
614
625
tx = session.begin_transaction()
615
626
616
627
try:
@@ -676,6 +687,11 @@ In the above example, if John's account balance is changed to a number less than
676
687
Implicit or auto-commit transactions are the simplest way to run a Cypher query since they won't be automatically retried as with `execute_query()` procedure or managed transactions.
677
688
With implicit transactions, you don't have the same control of transaction as with explicit transactions, so they are mostly used for quick prototyping.
678
689
690
+
<Callouttype="info">
691
+
As of Memgraph version 3.2, queries are categorized as **read** or **write** and the corresponding storage access is taken. This allows for better query parallelization and higher throughput.
692
+
Access mode is automatically determined when executing single queries through implicit transactions.
693
+
</Callout>
694
+
679
695
To run an implicit transaction, use the `Session.run()` method:
0 commit comments