Skip to content

int 类型 同步后变负数 #259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vovolie opened this issue Jun 6, 2018 · 10 comments
Closed

int 类型 同步后变负数 #259

vovolie opened this issue Jun 6, 2018 · 10 comments

Comments

@vovolie
Copy link

vovolie commented Jun 6, 2018

table 有个字段 int(11) 存的是数值化后的IP地址
dump时,在es内是正确的,replication 时,部份超出一定值后变成负数。

@siddontang
Copy link
Collaborator

do you use unsigned int for IP?

can you give me your table schema and test data?

@vovolie
Copy link
Author

vovolie commented Jun 7, 2018

+------------+--------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------------------------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| content | text | YES | | NULL | |
| article_id | int(11) | NO | MUL | NULL | |
| user_id | int(11) | NO | MUL | 0 | |
| created_at | datetime | NO | | NULL | |
| status | enum('publish','pending','private','spam') | NO | | pending | |
| ip | int(11) unsigned | NO | | 0 | |
| anonymous | tinyint(1) | NO | | 0 | |
| pos | int(11) | YES | | NULL | |
| neg | int(11) | YES | | NULL | |
| floor | int(11) | YES | | NULL | |
| score | int(11) | YES | | NULL | |
| parent_id | int(11) | YES | | NULL | |
| source | varchar(255) | YES | | NULL | |
+------------+--------------------------------------------+------+-----+---------+----------------+

test data:
{u'status': 'publish', u'user_id': 123456, u'floor': 1610, u'ip': 3748168280, u'created_at': datetime.datetime(2018, 6, 4, 16, 33, 47), u'pos': None, u'content': 'x88', u'parent_id': 123456, u'score': 123456, u'source': 'android', u'anonymous': 0, u'neg': None, u'article_id': 123456, u'id': 123456}

elasticsearch result:
# ip |   | -547,754,993

@vovolie
Copy link
Author

vovolie commented Jun 7, 2018

I found the problem.
row_event.go func decodeValue()
case MYSQL_TYPE_LONG: n = 4 v = ParseBinaryInt32(data)
maybe need ParseBinaryInt64 ? or tp as longlong?

@siddontang
Copy link
Collaborator

The problem is that we don't handle unsigned flag.
For binlog, we can't know the number is unsigned or not, but for go-mysql-elasticsearch, we can know it, we need to convert to the unsigned value if we find the column is number and unsigned.

@vovolie
Copy link
Author

vovolie commented Jun 7, 2018

Will you release fix version?

@siddontang
Copy link
Collaborator

no, have no time to fix it yet. maybe next week.

@vovolie
Copy link
Author

vovolie commented Jun 8, 2018

coolest!

@vovolie
Copy link
Author

vovolie commented Jun 15, 2018

has been fixed?

@siddontang
Copy link
Collaborator

no, too busy this week.

@siddontang
Copy link
Collaborator

@vovolie

fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants