Skip to content

Unable to call iterdump method for Sqlite3 connections #118221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kesry opened this issue Apr 24, 2024 · 8 comments
Closed

Unable to call iterdump method for Sqlite3 connections #118221

kesry opened this issue Apr 24, 2024 · 8 comments
Labels
3.11 only security fixes 3.12 only security fixes topic-sqlite3 type-bug An unexpected behavior, bug, or error

Comments

@kesry
Copy link

kesry commented Apr 24, 2024

Bug report

Bug description:

import sqlite3


def dict_factory(cursor, row):
    d = {}
    for idx, col in enumerate(cursor.description):
        d[col[0]] = row[idx]
    return d


def main():
    conn = sqlite3.connect(":memory:")
    cur = conn.cursor()
    # conn.row_factory = sqlite3.Row # is ok
    conn.row_factory = dict_factory # is fail
    cur.executescript("""    
        create table if not exists test(
            id integer primary key AUTOINCREMENT
        );
    """)
    cur.close()
    for line in conn.iterdump():
        print(line)


main()

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Linked PRs

@kesry kesry added the type-bug An unexpected behavior, bug, or error label Apr 24, 2024
@erlend-aasland
Copy link
Contributor

FTR, the traceback is:

Traceback (most recent call last):
  File "../iterdump.py", line 26, in <module>
    main()
  File "../iterdump.py", line 22, in main
    for line in conn.iterdump():
  File "/opt/homebrew/Cellar/[email protected]/3.8.19/Frameworks/Python.framework/Versions/3.8/lib/python3.8/sqlite3/dump.py", line 55, in _iterdump
    query_res = cu.execute(q)
sqlite3.OperationalError: no such table: name

Happens on Python versions from 3.8 through 3.13. Using the sqlite3.Row factory seems to work fine.

@kesry
Copy link
Author

kesry commented Apr 24, 2024

FTR, the traceback is:

Traceback (most recent call last):
  File "../iterdump.py", line 26, in <module>
    main()
  File "../iterdump.py", line 22, in main
    for line in conn.iterdump():
  File "/opt/homebrew/Cellar/[email protected]/3.8.19/Frameworks/Python.framework/Versions/3.8/lib/python3.8/sqlite3/dump.py", line 55, in _iterdump
    query_res = cu.execute(q)
sqlite3.OperationalError: no such table: name

Happens on Python versions from 3.8 through 3.13. Using the sqlite3.Row factory seems to work fine.

sqlite3. Row can't meet the needs of the production environment, is there any other way?

@erlend-aasland
Copy link
Contributor

The problem is that iterdump does not work with a custom row factory like this. Perhaps we should save the row factory before doing iterdump and restore it upon completion.

@kesry
Copy link
Author

kesry commented Apr 24, 2024

iterdump

Well, Thanks!

@kesry kesry closed this as completed Apr 24, 2024
@erlend-aasland
Copy link
Contributor

I still think we should fix this. Perhaps a mention in the docs is sufficient.

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Apr 24, 2024
…mp()

The iterdump() implementation depends on the row factory returning
resulting rows as tuples; it will fail with custom row factories like
for example a dict factory.

Fixed by overriding the row factory of the cursor used by iterdump().
FTR, this does not affect the row factory of the parent connection.
@erlend-aasland erlend-aasland added 3.11 only security fixes 3.12 only security fixes labels Apr 24, 2024
@erlend-aasland
Copy link
Contributor

The problem is that iterdump does not work with a custom row factory like this. Perhaps we should save the row factory before doing iterdump and restore it upon completion.

The solution is easier: overriding the row factory of a cursor does not impact the row factory of the parent connection, so all we need to do is to make sure the cursor used by iterdump() uses the default row factory by explicitly setting cursor.row_factory = None.

erlend-aasland added a commit that referenced this issue Apr 25, 2024
…118223)

sqlite3.iterdump() depends on the row factory returning resulting rows
as tuples; it will fail with custom row factories like for example a
dict factory.

With this commit, we explicitly reset the row factory of the cursor used
by iterdump(), so we always get predictable results. This does not
affect the row factory of the parent connection.

Co-authored-by: Mariusz Felisiak <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>
@erlend-aasland
Copy link
Contributor

erlend-aasland added a commit that referenced this issue Apr 25, 2024
…ump() (#118223) (#118270)

sqlite3.iterdump() depends on the row factory returning resulting rows
as tuples; it will fail with custom row factories like for example a
dict factory.

With this commit, we explicitly reset the row factory of the cursor used
by iterdump(), so we always get predictable results. This does not
affect the row factory of the parent connection.

Co-authored-by: Mariusz Felisiak <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>
@kesry
Copy link
Author

kesry commented Apr 25, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes 3.12 only security fixes topic-sqlite3 type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants