Merge pull request #312 from lzpap/async_tutorial

Tutorial 8: Async Send and Monitor

Author: lzpap

docs/tutorials.rst

@@ -716,11 +716,197 @@ Now you know how to use the Tangle for data storage while keeping privacy.
 When you need more granular access control on how and when one could read
 data from the Tangle, consider using `Masked Authenticated Messaging`_ (MAM).
 
+8. Send and Monitor Concurrently
+--------------------------------
+
+In this example, you will learn how to:
+
+- **Use the asynchronous PyOTA API.**
+- **Send transactions concurrently.**
+- **Monitor confirmation of transactions concurrently.**
+- **Execute arbitrary code concurrently while doing the former two.**
+
+.. warning::
+
+    If you are new to `coroutines`_  and asynchronous programming in Python, it
+    is strongly recommended that you check out this `article`_ and the official
+    `asyncio`_ documentation before proceeding.
+
+Code
+~~~~
+.. literalinclude:: ../examples/tutorials/08_async_send_monitor.py
+   :linenos:
+
+Discussion
+~~~~~~~~~~
+This example is divided into 4 logical parts:
+
+ 1. Imports and constant declarations
+ 2. `Coroutine`_ to send and monitor a list of transactions as a bundle.
+ 3. `Coroutine`_ to execute arbitrary code concurrently.
+ 4. A main `coroutine`_ to schedule the execution of our application.
+
+Let's start with the most simple one: **Imports and Constants**.
+
+.. literalinclude:: ../examples/tutorials/08_async_send_monitor.py
+   :lines: 1-17
+   :lineno-start: 1
+
+Notice, that we import the :py:class:`AsyncIota` api class, because we
+would like to use the asynchronous and concurrent features of PyOTA.
+:py:class:`List` from the :py:class:`typing` library is needed for correct
+type annotations, and we also import the `asyncio`_ library. This will come
+in handy when we want to schedule and run the coroutines.
+
+On line 6, we instantiate an asynchronous IOTA api. Functionally, it does the
+same operations as :py:class:`Iota`, but the api calls are defined as
+coroutines. For this tutorial, we connect to a devnet node, and explicitly tell
+this as well to the api on line 8.
+
+On line 12, we declare an IOTA address. We will send our zero value transactions
+to this address. Feel free to change it to your own address.
+
+Once we have sent the transactions, we start monitoring their confirmation by the
+network. Confirmation time depends on current network activity, the referenced
+tips, etc., therefore we set a ``timeout`` of 120 seconds on line 15. You might
+have to modify this value later to see the confirmation of your transactions.
+
+You can also fine-tune the example code by tinkering with ``polling_interval``.
+This is the interval between two subsequent confirmation checks.
+
+Let's move on to the next block, namely the **send and monitor coroutine**.
+
+.. literalinclude:: ../examples/tutorials/08_async_send_monitor.py
+   :lines: 20-62
+   :lineno-start: 20
+
+Notice, that coroutines are defined in python by the ``async def`` keywords.
+This makes them `awaitable`_.
+
+From the type annotations, we see that :py:meth:`send_and_monitor` accepts a
+list of :py:class:`ProposedTransaction` objects and return a ``bool``.
+
+On line 28, we send the transfers with the help of
+:py:meth:`AsyncIota.send_transfer`. Since this is not a regular method, but a
+coroutine, we have to ``await`` its result. :py:meth:`AsyncIota.send_transfer`
+takes care of building the bundle, doing proof-of-work and sending the
+transactions within the bundle to the network.
+
+Once we sent the transfer, we collect individual transaction hashes from the
+bundle, which we will use for confirmation checking.
+
+On line 39, the so-called confirmation checking starts. With the help of
+:py:meth:`AsyncIota.get_inclusion_states`, we determine if our transactions
+have been confirmed by the network.
+
+.. note::
+
+    You might wonder how your transactions get accepted by the network, that is,
+    how they become confirmed.
+
+        - Pre-`Coordicide`_ (current state), transactions are confirmed by
+          directly or indirectly being referenced by a `milestone`_.
+          A milestone is a special transaction issued by the `Coordinator`_.
+        - Post-`Coordicide`_ , confirmation is the result of nodes reaching
+          consensus by a `voting mechanism`_.
+
+The ``None`` value for the ``tips``
+parameter in the argument list basically means that we check against the latest
+milestone.
+
+On line 43, we iterate over our original ``sent_tx_hashes`` list of sent
+transaction hashes and ``gis_response['states']``, which is a list of ``bool``
+values, at the same time using the built-in `zip`_ method. We also employ
+`enumerate`_, because we need the index of the elements in each iteration.
+
+If a transaction is confirmed, we delete the corresponding elements from the
+lists. When all transactions are confirmed, ``sent_tx_hashes`` becomes empty,
+and the loop condition becomes ``False``.
+
+If however, not all transactions have been confirmed, we should continue
+checking for confirmation. Observe line 58, where we suspend the coroutine
+with :py:meth:`asyncio.sleep` for ``polling_interval`` seconds. Awaiting the
+result of :py:meth:`asyncio.sleep` will cause our coroutine to continue
+execution in ``polling_interval`` time. While our coroutine is sleeping,
+other coroutines can run concurrently, hence it is a non-blocking call.
+
+To do something in the meantime, we can **execute another coroutine concurrently**:
+
+.. literalinclude:: ../examples/tutorials/08_async_send_monitor.py
+   :lines: 65-71
+   :lineno-start: 65
+
+This is really just a dummy coroutine that prints something to the terminal and
+then goes to sleep periodically, but in a real application, you could do
+meaningful tasks here.
+
+Now let's look at how to **schedule the execution of our application with the
+main coroutine**:
+
+.. literalinclude:: ../examples/tutorials/08_async_send_monitor.py
+   :lines: 74-115
+   :lineno-start: 74
+
+First, we declare a list of :py:meth:`ProposedTransaction` objects, that will
+be the input for our :py:meth:`send_and_monitor` coroutine.
+
+The important stuff begins on line 101. We use :py:meth:`asyncio.gather` to
+submit our coroutines for execution, wait for their results and then return
+them in a list. `gather`_ takes our coroutines, transforms them into runnable
+`tasks`_, and runs them concurrently.
+
+Notice, that we listed :py:meth:`send_and_monitor` twice in
+:py:meth:`asyncio.gather` with the same list of :py:meth:`ProposedTransaction`
+objects. This is to showcase how you can send and monitor multiple transfers
+concurrently. In this example, two different bundles will be created from the
+same :py:meth:`ProposedTransaction` objects. The two bundles post zero value
+transactions to the same address, contain the same messages respectively,
+but are not dependent on each other in any way. That is why we can send them
+concurrently.
+
+As discussed previously, ``result`` will be a list of results of the coroutines
+submitted to :py:meth:`asyncio.gather`, preserving their order.
+``result[0]`` is the result from the first :py:meth:`send_and_monitor`, and
+``result[1]`` is the result from the second :py:meth:`send_and_monitor` from the
+argument list. If any of these are ``False``, confirmation did not happen
+before ``timeout``.
+
+When you see the message from line 109 in your terminal, try increasing
+``timeout``, or check the status of the network, maybe there is a temporary
+downtime on the devnet due to maintenance.
+
+Lastly, observe lines 113-115. If the current file (python module) is run
+from the terminal, we use :py:meth:`ayncio.run` to execute the main coroutine
+inside an `event loop`_.
+
+To run this example, navigate to ``examples/tutorial`` inside the cloned
+PyOTA repository, or download the source file of `Tutorial 8 from GitHub`_
+and run the following in a terminal:
+
+.. code-block:: sh
+
+    $ python 08_async_send_monitor.py
+
 .. _PyOTA Bug Tracker: https://github.com/iotaledger/iota.py/issues
 .. _bytestring: https://docs.python.org/3/library/stdtypes.html#bytes
 .. _tryte alphabet: https://docs.iota.org/docs/getting-started/0.1/introduction/ternary#tryte-encoding
 .. _Tangle Explorer: https://utils.iota.org
 .. _Account Module: https://docs.iota.org/docs/client-libraries/0.1/account-module/introduction/overview
 .. _spending twice from the same address: https://docs.iota.org/docs/getting-started/0.1/clients/addresses#spent-addresses
 .. _Base64: https://en.wikipedia.org/wiki/Base64
-.. _Masked Authenticated Messaging: https://docs.iota.org/docs/client-libraries/0.1/mam/introduction/overview?q=masked%20auth&highlights=author;authent
+.. _Masked Authenticated Messaging: https://docs.iota.org/docs/client-libraries/0.1/mam/introduction/overview?q=masked%20auth&highlights=author;authent
+.. _coroutine: https://docs.python.org/3/glossary.html#term-coroutine
+.. _coroutines: https://docs.python.org/3/glossary.html#term-coroutine
+.. _asyncio: https://docs.python.org/3/library/asyncio.html
+.. _article: https://realpython.com/async-io-python/
+.. _awaitable: https://docs.python.org/3/library/asyncio-task.html#awaitables
+.. _Coordicide: https://coordicide.iota.org/
+.. _milestone: https://docs.iota.org/docs/getting-started/0.1/network/the-coordinator#milestones
+.. _coordinator: https://docs.iota.org/docs/getting-started/0.1/network/the-coordinator
+.. _voting mechanism: https://coordicide.iota.org/module4.1
+.. _zip: https://docs.python.org/3.3/library/functions.html#zip
+.. _enumerate: https://docs.python.org/3.3/library/functions.html#enumerate
+.. _gather: https://docs.python.org/3/library/asyncio-task.html#running-tasks-concurrently
+.. _tasks: https://docs.python.org/3/library/asyncio-task.html#asyncio.Task
+.. _event loop: https://docs.python.org/3/library/asyncio-eventloop.html
+.. _Tutorial 8 from GitHub: https://github.com/iotaledger/iota.py/blob/master/examples/tutorials/08_async_send_monitor.py

examples/tutorials/08_async_send_monitor.py

@@ -0,0 +1,115 @@
+from iota import AsyncIota, ProposedTransaction, Address, TryteString
+from typing import List
+import asyncio
+
+# Asynchronous API instance.
+api = AsyncIota(
+        adapter='https://nodes.devnet.iota.org:443',
+        devnet=True,
+)
+
+# An arbitrary address to send zero-value transactions to.
+addy = Address('PZITJTHCIIANKQWEBWXUPHWPWVNBKW9GMNALMGGSIAUOYCKNWDLUUIGAVMJYCHZXHUBRIVPLFZHUVDLME')
+
+# Timeout after which confirmation monitoring stops (seconds).
+timeout = 120
+# How often should we poll for confirmation? (seconds)
+polling_interval = 5
+
+
+async def send_and_monitor(
+    transactions: List[ProposedTransaction]
+) -> bool:
+    """
+    Send a list of transactions as a bundle and monitor their confirmation
+    by the network.
+    """
+    print('Sending bundle...')
+    st_response = await api.send_transfer(transactions)
+
+    sent_tx_hashes = [tx.hash for tx in st_response['bundle']]
+
+    print('Sent bundle with transactions: ')
+    print(*sent_tx_hashes, sep='\n')
+
+    # Measure elapsed time
+    elapsed = 0
+
+    print('Checking confirmation...')
+    while len(sent_tx_hashes) > 0:
+        # Determine if transactions are confirmed
+        gis_response = await api.get_inclusion_states(sent_tx_hashes, None)
+
+        for i, (tx, is_confirmed) in enumerate(zip(sent_tx_hashes, gis_response['states'])):
+            if is_confirmed:
+                print('Transaction %s is confirmed.' % tx)
+                # No need to check for this any more
+                del sent_tx_hashes[i]
+                del gis_response['states'][i]
+
+        if len(sent_tx_hashes) > 0:
+            if timeout <= elapsed:
+                # timeout reached, terminate checking
+                return False
+            # Show some progress on the screen
+            print('.')
+            # Put on hold for polling_interval. Non-blocking, so you can
+            # do other stuff in the meantime.
+            await asyncio.sleep(polling_interval)
+            elapsed = elapsed + polling_interval
+
+    # All transactions in the bundle are confirmed
+    return True
+
+
+async def do_something() -> None:
+    """
+    While waiting for confirmation, you can execute arbitrary code here.
+    """
+    for _ in range(5):
+        print('Doing something in the meantime...')
+        await asyncio.sleep(2)
+
+
+async def main() -> None:
+    """
+    A simple application that sends zero-value transactions to the Tangle and
+    monitors the confirmation by the network. While waiting for the
+    confirmation, we schedule a task (`do_something()`) to be executed concurrently.
+    """
+    # Transactions to be sent.
+    transactions = [
+        ProposedTransaction(
+            address=addy,
+            value=0,
+            message=TryteString.from_unicode('First'),
+        ),
+        ProposedTransaction(
+            address=addy,
+            value=0,
+            message=TryteString.from_unicode('Second'),
+        ),
+        ProposedTransaction(
+            address=addy,
+            value=0,
+            message=TryteString.from_unicode('Third'),
+        ),
+    ]
+
+    # Schedule coroutines as tasks, wait for them to finish and gather their
+    # results.
+    result = await asyncio.gather(
+            send_and_monitor(transactions),
+            # Send the same content. Bundle will be different!
+            send_and_monitor(transactions),
+            do_something(),
+        )
+
+    if not (result[0] and result[1]):
+        print('Transactions did not confirm after %s seconds!' % timeout)
+    else:
+        print('All transactions are confirmed!')
+
+if __name__ == '__main__':
+    # Execute main() inside an event loop if the file is ran
+    asyncio.run(main())