Skip to content

Deflake OpenTelemetry assertion in system tests #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
larkee opened this issue Nov 11, 2020 · 0 comments
Closed

Deflake OpenTelemetry assertion in system tests #165

larkee opened this issue Nov 11, 2020 · 0 comments
Labels
api: spanner Issues related to the googleapis/python-spanner API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: cleanup An internal cleanup or hygiene concern.

Comments

@larkee
Copy link
Contributor

larkee commented Nov 11, 2020

The OpenTelemetry assertions in the system tests fail if the transaction is aborted and retried. This is extremely brittle so the assertions need to be updated to account for retrying aborted transactions.

Example of test failure:

________ TestSessionAPI.test_transaction_read_and_insert_then_rollback _________

self = <tests.system.test_system.TestSessionAPI testMethod=test_transaction_read_and_insert_then_rollback>

    @RetryErrors(exception=exceptions.ServerError)
    @RetryErrors(exception=exceptions.Aborted)
    def test_transaction_read_and_insert_then_rollback(self):
        retry = RetryInstanceState(_has_all_ddl)
        retry(self._db.reload)()

        session = self._db.session()
        session.create()
        self.to_delete.append(session)

        with self._db.batch() as batch:
            batch.delete(self.TABLE, self.ALL)

        transaction = session.transaction()
        transaction.begin()

        rows = list(transaction.read(self.TABLE, self.COLUMNS, self.ALL))
        self.assertEqual(rows, [])

        transaction.insert(self.TABLE, self.COLUMNS, self.ROW_DATA)

        # Inserted rows can't be read until after commit.
        rows = list(transaction.read(self.TABLE, self.COLUMNS, self.ALL))
        self.assertEqual(rows, [])
        transaction.rollback()

        rows = list(session.read(self.TABLE, self.COLUMNS, self.ALL))
        self.assertEqual(rows, [])

        if HAS_OPENTELEMETRY_INSTALLED:
            span_list = self.memory_exporter.get_finished_spans()
>           self.assertEqual(len(span_list), 8)
E           AssertionError: 14 != 8

tests/system/test_system.py:1026: AssertionError
----------------------------- Captured stdout call -----------------------------
409 Transaction was aborted., Trying again in 1 seconds...
------------------------------ Captured log call -------------------------------
WARNING  opentelemetry.trace:__init__.py:468 Overriding current TracerProvider
WARNING  opentelemetry.trace:__init__.py:468 Overriding current TracerProvider
@larkee larkee added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p2 Moderately-important priority. Fix may not be included in next release. labels Nov 11, 2020
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner API. label Nov 11, 2020
@yoshi-automation yoshi-automation added 🚨 This issue needs some love. and removed 🚨 This issue needs some love. labels Feb 9, 2021
@larkee larkee added priority: p3 Desirable enhancement or fix. May not be included in next release. and removed priority: p2 Moderately-important priority. Fix may not be included in next release. labels Feb 23, 2021
@larkee larkee added type: cleanup An internal cleanup or hygiene concern. and removed type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Mar 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: cleanup An internal cleanup or hygiene concern.
Projects
None yet
Development

No branches or pull requests

3 participants