Skip to content

Fix psycopg2 register type #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 10, 2018

Conversation

jaysonsantos
Copy link
Contributor

This patch is intended to fix problems when you try to register extensions like uuid on psycopg2.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@haotianw465
Copy link
Contributor

The code looks good to me. Before merge may I ask what is the actual problem you are trying to fix? Even for wrapped cursor and connection only DBAPI like execute is being traced. Did you get any error when using wrapped cursor/connection to register extensions?

@jaysonsantos
Copy link
Contributor Author

Hey there @haotianw465 the problem is because I wanted to measure only psycopg2 but using it with sqlalchemy and sqlalchemy tries to register some extensions like UUID and it breaks because the connection is wrapped.

@haotianw465
Copy link
Contributor

@jaysonsantos do you mind providing the stack trace of the error you got so anyone has a similar use case can reference the actual problem?

@haotianw465
Copy link
Contributor

Hi @jaysonsantos , any update on this?

@jaysonsantos
Copy link
Contributor Author

Hi there, I could swear that I had already posted it here. I will try and find one on my Sentry account.

@jaysonsantos
Copy link
Contributor Author

Here is an example:

In [1]: db.session.execute('select 1')                                                                                                                                                                                                                               
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/p/eatfirst/cave/manage.py in <module>
----> 1 db.session.execute('select 1')

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py in do(self, *args, **kwargs)
    151 def instrument(name):
    152     def do(self, *args, **kwargs):
--> 153         return getattr(self.registry(), name)(*args, **kwargs)
    154     return do
    155 

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/orm/session.py in execute(self, clause, params, mapper, bind, **kw)
   1174 
   1175         return self._connection_for_bind(
-> 1176             bind, close_with_result=True).execute(clause, params or {})
   1177 
   1178     def scalar(self, clause, params=None, mapper=None, bind=None, **kw):

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/orm/session.py in _connection_for_bind(self, engine, execution_options, **kw)
   1038         if self.transaction is not None:
   1039             return self.transaction._connection_for_bind(
-> 1040                 engine, execution_options)
   1041         else:
   1042             conn = engine.contextual_connect(**kw)

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/orm/session.py in _connection_for_bind(self, bind, execution_options)
    407                         "given Connection's Engine")
    408             else:
--> 409                 conn = bind.contextual_connect()
    410 
    411         if execution_options:

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/engine/base.py in contextual_connect(self, close_with_result, **kwargs)
   2121         return self._connection_cls(
   2122             self,
-> 2123             self._wrap_pool_connect(self.pool.connect, None),
   2124             close_with_result=close_with_result,
   2125             **kwargs)

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/engine/base.py in _wrap_pool_connect(self, fn, connection)
   2156         dialect = self.dialect
   2157         try:
-> 2158             return fn()
   2159         except dialect.dbapi.Error as e:
   2160             if connection is None:

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/base.py in connect(self)
    353         """
    354         if not self._use_threadlocal:
--> 355             return _ConnectionFairy._checkout(self)
    356 
    357         try:

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/base.py in _checkout(cls, pool, threadconns, fairy)
    741     def _checkout(cls, pool, threadconns=None, fairy=None):
    742         if not fairy:
--> 743             fairy = _ConnectionRecord.checkout(pool)
    744 
    745             fairy._pool = pool

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/base.py in checkout(cls, pool)
    482     @classmethod
    483     def checkout(cls, pool):
--> 484         rec = pool._do_get()
    485         try:
    486             dbapi_connection = rec.get_connection()

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/impl.py in _do_get(self)
    126             except:
    127                 with util.safe_reraise():
--> 128                     self._dec_overflow()
    129         else:
    130             return self._do_get()

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py in __exit__(self, type_, value, traceback)
     64             self._exc_info = None   # remove potential circular references
     65             if not self.warn_only:
---> 66                 compat.reraise(exc_type, exc_value, exc_tb)
     67         else:
     68             if not compat.py3k and self._exc_info and self._exc_info[1]:

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/util/compat.py in reraise(tp, value, tb, cause)
    247         if value.__traceback__ is not tb:
    248             raise value.with_traceback(tb)
--> 249         raise value
    250 
    251 else:

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/impl.py in _do_get(self)
    123         if self._inc_overflow():
    124             try:
--> 125                 return self._create_connection()
    126             except:
    127                 with util.safe_reraise():

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/base.py in _create_connection(self)
    300         """Called by subclasses to create a new ConnectionRecord."""
    301 
--> 302         return _ConnectionRecord(self)
    303 
    304     def _invalidate(self, connection, exception=None, _checkin=True):

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/base.py in __init__(self, pool, connect)
    427         self.__pool = pool
    428         if connect:
--> 429             self.__connect(first_connect_check=True)
    430         self.finalize_callback = deque()
    431 

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/pool/base.py in __connect(self, first_connect_check)
    634                 pool.dispatch.first_connect.\
    635                     for_modify(pool.dispatch).\
--> 636                     exec_once(self.connection, self)
    637             if pool.dispatch.connect:
    638                 pool.dispatch.connect(self.connection, self)

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/event/attr.py in exec_once(self, *args, **kw)
    272                 if not self._exec_once:
    273                     try:
--> 274                         self(*args, **kw)
    275                     finally:
    276                         self._exec_once = True

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/event/attr.py in __call__(self, *args, **kw)
    282             fn(*args, **kw)
    283         for fn in self.listeners:
--> 284             fn(*args, **kw)
    285 
    286     def __len__(self):

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py in on_connect(dbapi_connection, connection_record)
    173                     if conn is None:
    174                         return
--> 175                     do_on_connect(conn)
    176 
    177                 event.listen(pool, 'first_connect', on_connect)

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py in on_connect(conn)
    671             def on_connect(conn):
    672                 for fn in fns:
--> 673                     fn(conn)
    674             return on_connect
    675         else:

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py in on_connect(conn)
    635         if self.dbapi and self.use_native_uuid:
    636             def on_connect(conn):
--> 637                 extras.register_uuid(None, conn)
    638             fns.append(on_connect)
    639 

~/.pyenv/versions/3.6.4/envs/cave/lib/python3.6/site-packages/psycopg2/extras.py in register_uuid(oids, conn_or_curs)
    666     _ext.UUIDARRAY = _ext.new_array_type((oid2,), "UUID[]", _ext.UUID)
    667 
--> 668     _ext.register_type(_ext.UUID, conn_or_curs)
    669     _ext.register_type(_ext.UUIDARRAY, conn_or_curs)
    670     _ext.register_adapter(uuid.UUID, UUID_adapter)

TypeError: argument 2 must be a connection, cursor or None

@jaysonsantos jaysonsantos force-pushed the fix-postgresql-register-type branch from 8996e8b to fe9ee2c Compare October 10, 2018 08:46
@jaysonsantos
Copy link
Contributor Author

I've also rebased the branch.

@haotianw465
Copy link
Contributor

@haotianw465 haotianw465 merged commit 5b35427 into aws:master Oct 10, 2018
@skunkwerk
Copy link

I'm still seeing this issue, has this not been released in the latest version on pip?

@haotianw465
Copy link
Contributor

@skunkwerk sorry for the delay. There are some in-progress changes for the next release and we will get this PR out as soon as those are done. Thank you for your patience.

@haotianw465
Copy link
Contributor

@skunkwerk this PR is released as part of 2.3.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants