Skip to content

incorrect handling of metadata url in some cases #697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cardanians opened this issue Jul 18, 2021 · 6 comments
Closed

incorrect handling of metadata url in some cases #697

cardanians opened this issue Jul 18, 2021 · 6 comments

Comments

@cardanians
Copy link
Contributor

cardanians commented Jul 18, 2021

poolId: pool1l92x52u37neuc2w38wrgeu3esu9qzddmzvkvyuf9vfk5gav8nxl

SELECT * FROM public.pool_offline_fetch_error
    WHERE fetch_error LIKE '%pool1l92x52u37neuc2w38wrgeu3esu9qzddmzvkvyuf9vfk5gav8nxl%'

if the metadata fails to download several times, it will stop downloading again.

This is fine in a way (but it might be worth repeating once a day anyway, wouldn't it?). however, they will not start re-fetching if the pool updates its poolcert.

obrazek

Look on screen.

  1. 13.7. some fetch tests with fail
  2. 17.7. reregistered pool
  3. then no other fetching...

obrazek

Second thing is, why is this new meta url not saved in DB? He registered first url with typo "BLOOIR" (in db). Then re-registered with fix, but in db is no record about this.

latest version (v10)

@erikd
Copy link
Contributor

erikd commented Jul 19, 2021

Please do not post images in bug reports. Lack of scaling on images can make the text tiny on high res screens (like mine) and i cannot cut and paste from an image.

Instead, please cut and paste the text.

@erikd
Copy link
Contributor

erikd commented Jul 19, 2021

There seem to be at least two issues here, and they do not seem to be well distinguished.

The first problem, is that if a PoolMetadataRef update for the same PoolId and same metadata hash, but a different url will not update the table. This is due to and incomplete uniqueness constraint.

The second issue seems to be a failure to parse some URLs.

@erikd
Copy link
Contributor

erikd commented Jul 20, 2021

It seems like the error with the uniqueness constraint is confounding and confusing the other problems, so I will fix that first.

@erikd
Copy link
Contributor

erikd commented Jul 20, 2021

Lets find the pool id:

select id, view from pool_hash where view = 'pool1l92x52u37neuc2w38wrgeu3esu9qzddmzvkvyuf9vfk5gav8nxl' ;
  id   |                           view                           
-------+----------------------------------------------------------
 18067 | pool1l92x52u37neuc2w38wrgeu3esu9qzddmzvkvyuf9vfk5gav8nxl
(1 row)

and the pool_metadata_ref:

select * from pool_metadata_ref where pool_id = 18067 ;
  id   | pool_id |             url             |                  hash                 | registered_tx_id 
-------+---------+-----------------------------+---------------------------------------+------------------
 18012 |   18067 | https://blooir.com/MTD.json | \x3f192a9740e983b...7028a25fcc99400dc |          8197291
 18015 |   18067 | https://booir.com/MTD.json  | \x3f192a9740e983b...7028a25fcc99400dc |          8198428
(2 rows)

Did not pick up any pool_offline_fetch_errors for this pool_id:

select * from pool_offline_fetch_error where pool_id = 18067 ;
 id | pool_id | fetch_time | pmr_id | fetch_error | retry_count 
----+---------+------------+--------+-------------+-------------
(0 rows)

Not sure if there is another error in here somewhere, but the main one is fixed.

erikd added a commit that referenced this issue Jul 20, 2021
Previously entries would be considered duplicates if the `pool_id` and
`hash` were the same, but ignrored any differences in the `url` field,
which was wrong. The `url` field *must* be part of the uniqueness
constraint.

Closes: #697
@erikd erikd closed this as completed in 012fa2d Jul 20, 2021
@cardanians
Copy link
Contributor Author

cardanians commented Jul 21, 2021

@erikd ok awesome! can you please push new release into world? with this + rewards, thanks.

@erikd
Copy link
Contributor

erikd commented Jul 21, 2021

There is one ore thing that needs fixing, before a new release, but that should be done this week.

tmcgilchrist pushed a commit that referenced this issue Aug 4, 2021
Previously entries would be considered duplicates if the `pool_id` and
`hash` were the same, but ignrored any differences in the `url` field,
which was wrong. The `url` field *must* be part of the uniqueness
constraint.

Closes: #697
jbgi added a commit that referenced this issue Feb 4, 2022
 before the pool hash was used in place of actual url, as show by
 reported errors, eg.:

  "URL parse error from for
  pool1zy4arks3chpruyt32p0mhwqu3qpdgyx7nejrw06nqu3uzfuv3aw resulted in :
  InvalidUrlException "pool1zy4arks3chpruyt32p0mhwqu3qpdgyx7nejrw06nqu3uzfuv3aw"

 Should complete the fix for #697.
jbgi added a commit that referenced this issue Feb 4, 2022
 before the pool hash was used in place of actual url, as show by
 reported errors, eg.:

  "URL parse error from for
  pool1zy4arks3chpruyt32p0mhwqu3qpdgyx7nejrw06nqu3uzfuv3aw resulted in :
  InvalidUrlException "pool1zy4arks3chpruyt32p0mhwqu3qpdgyx7nejrw06nqu3uzfuv3aw"

 Should complete the fix for #697.
erikd pushed a commit that referenced this issue Feb 7, 2022
 before the pool hash was used in place of actual url, as show by
 reported errors, eg.:

  "URL parse error from for
  pool1zy4arks3chpruyt32p0mhwqu3qpdgyx7nejrw06nqu3uzfuv3aw resulted in :
  InvalidUrlException "pool1zy4arks3chpruyt32p0mhwqu3qpdgyx7nejrw06nqu3uzfuv3aw"

 Should complete the fix for #697.
 Cherry-picked 00c96c2 from master.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants