Added support for Oracle Database 23ai statement pipelining.

Author: anthony-tuininga

doc/src/api_manual/async_connection.rst

@@ -166,6 +166,37 @@ AsyncConnection Methods
 
     Rolls back any pending transaction.
 
+.. method:: AsyncConnection.run_pipeline(pipeline, continue_on_error=False)
+
+    Runs all of the operations in the :ref:`pipeline <pipelineobj>` and returns
+    a list of :ref:`PipelineOpResult Objects <pipelineopresultobjs>`, each
+    entry corresponding to an operation executed in the pipeline.
+
+    The ``continue_on_error`` parameter determines whether operations should
+    continue to run after an error has occurred. If this parameter is set to
+    True, then the :attr:`PipelineOpResult.error` attribute will be populated
+    with an :ref:`_Error <exchandling>` instance which identifies the error
+    that occurred. If this parameter is set to False, then an exception will be
+    raised as soon as an error is detected and all subsequent operations will
+    be terminated. The default value is False.
+
+    See :ref:`pipelining` for more information.
+
+    .. note::
+
+        In this release, pipelining support is experimental and subject to
+        change.
+
+        True pipelining requires Oracle Database 23ai.
+
+        When you connect to an older database, operations are sequentially
+        executed by python-oracledb. Each operation concludes before the next
+        is sent to the database. There is no reduction in round-trips and no
+        performance benefit. This usage is only recommended for code
+        portability such as when preparing for a database upgrade.
+
+    .. versionadded:: 2.4.0
+
 .. method:: AsyncConnection.tpc_begin(xid, flags, timeout)
 
     Begins a Two-Phase Commit (TPC) on a global transaction using the specified

doc/src/api_manual/module.rst

@@ -950,6 +950,18 @@ Oracledb Methods
 
         The ``connection_id_prefix`` parameter was added.
 
+.. function:: create_pipeline()
+
+    Creates a :ref:`pipeline object <pipelineobjs>` which can be used to
+    process a set of operations against a database.
+
+    .. note::
+
+        In this release, pipelining support is experimental and subject to
+        change.
+
+    .. versionadded:: 2.4.0
+
 .. function:: create_pool(dsn=None, pool_class=oracledb.ConnectionPool, \
         params=None, min=1, max=2, increment=1, \
         connectiontype=oracledb.Connection, \
@@ -2662,6 +2674,85 @@ cx_Oracle 8.3. They are possible values for the ``mode`` parameter of the
         This constant deprecates the ``SYSRAC`` constant that was used in
         cx_Oracle 8.3.
 
+.. _pipeline-operation-types:
+
+Pipeline Operation Types
+------------------------
+
+These constants belong to the enumeration called ``PipelineOpType``. The
+pipelining constants listed below are used to identify the type of operation
+added. They are possible values for the :attr:`PipelineOp.op_type` attribute.
+
+.. note::
+
+    In this release, pipelining support is experimental and subject to change.
+
+.. versionadded:: 2.4.0
+
+.. data:: oracledb.PIPELINE_OP_TYPE_CALL_FUNC
+
+    This constant identifies the type of operation as the calling of a stored
+    function.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.CALL_FUNC``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_CALL_PROC
+
+    This constant identifies the type of operation as the calling of a stored
+    procedure.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.CALL_PROC``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_COMMIT
+
+    This constant identifies the type of operation as the performing of a
+    commit.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.COMMIT``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_EXECUTE
+
+    This constant identifies the type of operation as the executing of a
+    statement.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.EXECUTE``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_EXECUTE_MANY
+
+    This constant identifies the type of operations as the executing of a
+    statement multiple times.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.EXECUTE_MANY``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_FETCH_ALL
+
+    This constant identifies the type of operation as the executing of a
+    query and returning all of the rows from the result set.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.FETCH_ALL``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_FETCH_MANY
+
+    This constant identifies the type of operation as the executing of a
+    query and returning up to the specified number of rows from the result
+    set.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.FETCH_MANY``.
+
+.. data:: oracledb.PIPELINE_OP_TYPE_FETCH_ONE
+
+    This constant identifies the type of operation as the executing of a query
+    and returning the first row of the result set.
+
+    This enumerated value can also be identified by
+    ``oracledb.PipelineOpType.FETCH_ONE``.
 
 Database Shutdown Modes
 -----------------------

doc/src/api_manual/pipeline.rst

@@ -0,0 +1,211 @@
+.. _pipelineobj:
+
+*********************
+API: Pipeline Objects
+*********************
+
+.. note::
+
+    In this release, pipelining support is experimental and subject to change.
+
+See :ref:`pipelining` for more information about pipelining.
+
+.. note::
+
+    True pipelining is only available when connected to Oracle Database 23ai.
+
+.. versionadded:: 2.4.0
+
+.. _pipelineobjs:
+
+Pipeline Objects
+================
+
+Pipeline objects represent a pipeline used to execute multiple database
+operations.  A Pipeline object is created by calling
+:meth:`oracledb.create_pipeline()`.
+
+.. _pipelinemethods:
+
+Pipeline Methods
+----------------
+
+.. method:: Pipeline.add_callfunc(name, return_type, parameters=None, keyword_parameters=None)
+
+    Adds an operation that calls a stored PL/SQL function with the given
+    parameters and return type. When the Pipeline is executed, the
+    :ref:`PipelineOpResult object <pipelineopresultobjs>` that is returned for
+    this operation will have the :attr:`~PipelineOpResult.return_value`
+    attribute populated with the return value of the PL/SQL function if the
+    call completes successfully.
+
+.. method:: Pipeline.add_callproc(name, parameters=None, keyword_parameters=None)
+
+    Adds an operation that calls a stored procedure with the given parameters.
+
+.. method:: Pipeline.add_commit()
+
+    Adds an operation that performs a commit.
+
+.. method:: Pipeline.add_execute(statement, parameters=None)
+
+    Adds an operation that executes a statement with the given parameters.
+
+    Do not use this for queries that return rows.  Instead use
+    :meth:`Pipeline.add_fetchall()`, :meth:`Pipeline.add_fetchmany()`, or
+    :meth:`Pipeline.add_fetchone()`.
+
+.. method:: Pipeline.add_executemany(statement, parameters)
+
+    Adds an operation that executes a statement multiple times with the given
+    parameter mappings or sequences found in the sequence parameters.
+
+    If there are no parameters, the number of iterations can be specified as an
+    integer instead of needing to provide a list of empty mappings or
+    sequences.
+
+.. method:: Pipeline.add_fetchall(statement, parameters=None, arraysize=None, rowfactory=None)
+
+    Adds an operation that executes a query and returns all of the rows from
+    the result set. When the Pipeline is executed, the :ref:`PipelineOpResult
+    object <pipelineopresultobjs>` that is returned for this operation will
+    have the :attr:`~PipelineOpResult.rows` attribute populated with the list
+    of rows returned by the query.
+
+    The default value for ``arraysize`` is :attr:`defaults.arraysize`.
+
+    Internally, this operation's :attr:`Cursor.prefetchrows` size is set to the
+    value of the explicit or default ``arraysize`` parameter value.
+
+.. method:: Pipeline.add_fetchmany(statement, parameters=None, num_rows=None, rowfactory=None)
+
+    Adds an operation that executes a query and returns up to the specified
+    number of rows from the result set. When the Pipeline is executed, the
+    :ref:`PipelineOpResult object <pipelineopresultobjs>` that is returned for
+    this operation will have the :attr:`~PipelineOpResult.rows` attribute
+    populated with the list of rows returned by the query.
+
+    The default value for ``num_rows`` is the value of
+    :attr:`defaults.arraysize`.
+
+    Internally, this operation's :attr:`Cursor.prefetchrows` size is set to the
+    value of the explicit or default ``num_rows`` parameter, allowing all rows
+    to be fetched in one :ref:`round-trip <roundtrips>`
+
+    Since only one fetch is performed for a query operation, consider adding a
+    ``FETCH NEXT`` clause to the statement to prevent the database processing
+    rows that will never be fetched, see :ref:`rowlimit`.
+
+.. method:: Pipeline.add_fetchone(statement, parameters=None, rowfactory=None)
+
+    Adds an operation that executes a query and returns the first row of the
+    result set if one exists. When the Pipeline is executed, the
+    :ref:`PipelineOpResult object <pipelineopresultobjs>` that is returned for
+    this operation will have the :attr:`~PipelineOpResult.rows` attribute
+    populated with this row if the query is performed successfully.
+
+    Internally, this operation's :attr:`Cursor.prefetchrows` and
+    :attr:`Cursor.arraysize` sizes will be set to 1.
+
+    Since only one fetch is performed for a query operation, consider adding a
+    ``WHERE`` condition or using a ``FETCH NEXT`` clause in the statement to
+    prevent the database processing rows that will never be fetched, see
+    :ref:`rowlimit`.
+
+Pipeline Attributes
+-------------------
+
+.. attribute:: Pipeline.operations
+
+    This read-only attribute returns the list of operations associated with
+    the pipeline.
+
+.. _pipelineopobjs:
+
+PipelineOp Objects
+==================
+
+PipelineOp objects are created by calling the methods in the
+:ref:`Pipeline class <pipelineobjs>`.
+
+PipelineOp Attributes
+---------------------
+
+.. attribute:: PipelineOp.arraysize
+
+    This read-only attribute returns the :ref:`array size <tuningfetch>` that
+    will be used when fetching query rows with :meth:`Pipeline.add_fetchall()`.
+    For all other operations, the value returned is 0.
+
+.. attribute:: PipelineOp.keyword_parameters
+
+    This read-only attribute returns the keyword parameters to the stored
+    procedure or function being called by the operation, if applicable.
+
+.. attribute:: PipelineOp.name
+
+    This read-only attribute returns the name of the stored procedure or
+    function being called by the operation, if applicable.
+
+.. attribute:: PipelineOp.num_rows
+
+    This read-only attribute returns the number of rows to fetch when
+    performing a query of a specific number of rows. For all other operations,
+    the value returned is 0.
+
+.. attribute:: PipelineOp.op_type
+
+    This read-only attribute returns the type of operation that is taking
+    place. See :ref:`pipeline-operation-types` for types of operations.
+
+.. attribute:: PipelineOp.parameters
+
+    This read-only attribute returns the parameters to the stored procedure or
+    function or the parameters bound to the statement being executed by the
+    operation, if applicable.
+
+.. attribute:: PipelineOp.return_type
+
+    This read-only attribute returns the return type of the stored function
+    being called by the operation, if applicable.
+
+.. attribute:: PipelineOp.rowfactory
+
+    This read-only attribute returns the row factory callable function to be
+    used in a query executed by the operation, if applicable.
+
+.. attribute:: PipelineOp.statement
+
+    This read-only attribute returns the statement being executed by the
+    operation, if applicable.
+
+.. _pipelineopresultobjs:
+
+PipelineOpResult Objects
+========================
+
+PipelineOpResult objects are returned in list when calling
+:meth:`AsyncConnection.run_pipeline()`. These objects contain the results of
+the executed :ref:`PipelineOp objects <pipelineopobjs>` operations.
+
+PipelineOpResult Attributes
+---------------------------
+
+.. attribute:: PipelineOpResult.error
+
+    This read-only attribute returns the error that occurred when running this
+    operation. If no error occurred, then the value None is returned.
+
+.. attribute:: PipelineOpResult.operation
+
+    This read-only attribute returns the operation associated with the result.
+
+.. attribute:: PipelineOpResult.return_value
+
+    This read-only attribute returns the return value of the called PL/SQL
+    function, if a function was called for the operation.
+
+.. attribute:: PipelineOpResult.rows
+
+    This read-only attribute returns the rows that were fetched by the
+    operation, if a query was executed.

doc/src/index.rst

@@ -81,6 +81,7 @@ API Manual
     api_manual/async_connection_pool.rst
     api_manual/async_cursor.rst
     api_manual/async_lob.rst
+    api_manual/pipeline.rst
     api_manual/deprecations.rst
 
 .. toctree::

doc/src/release_notes.rst

@@ -17,6 +17,8 @@ oracledb 2.4.0 (TBD)
 Thin Mode Changes
 +++++++++++++++++
 
+#)  Added support for Oracle Database 23ai :ref:`statement pipelining
+    <pipelining>`.
 #)  Fixed bug resulting in a segfault when a closed cursor is bound as a REF
     CURSOR
     (`issue 368 <https://github.com/oracle/python-oracledb/issues/368>`__).

doc/src/user_guide/appendix_a.rst

@@ -294,10 +294,14 @@ see :ref:`driverdiff` and :ref:`compatibility`.
       - No
       - Yes
       - Yes
-    * - Concurrent programming with asyncio (see :ref:`asyncio`)
+    * - Concurrent programming with asyncio (see :ref:`concurrentprogramming`)
       - Yes
       - No
       - No
+    * - Oracle Database 23ai Pipelining (see :ref:`pipelining`)
+      - Yes - must use :ref:`asyncio <concurrentprogramming>`
+      - No
+      - No
     * - End-to-end monitoring and tracing attributes (see :ref:`tracingsql`)
       - Yes
       - Yes