Trouble while making authenticated cross wiki calls #21

maskaravivek · 2019-07-17T15:21:49Z

All of the authenticated calls(ie login, upload, nomination for deletion, thank, notifications etc) made to commons wiki is working perfectly. I am stuck with a cross-wiki call to Wikidata. I am trying to call Service:wbcreateclaim to create a claim on Wikidata but the call is failing. I have attached the relevant logs below.

@dbrant It would be great if you could take a look at the logs and suggest what might be wrong. Is it because of some issue with cookies because as far as I see, as expected I am sending the params in POST request body with application/x-www-form-urlencoded?

2019-07-07 03:59:27.528 30797-30831/fr.free.nrw.commons D/OkHttp: --> GET https://commons.wikimedia.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&meta=tokens&type=csrf
2019-07-07 03:59:27.528 30797-30831/fr.free.nrw.commons D/OkHttp: Cache-Control: no-cache
2019-07-07 03:59:27.528 30797-30831/fr.free.nrw.commons D/OkHttp: --> END GET
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: <-- 200 https://commons.wikimedia.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&meta=tokens&type=csrf (372ms)
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: date: Sat, 06 Jul 2019 22:29:28 GMT
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: content-type: application/json; charset=utf-8
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: server: mw1231.eqiad.wmnet
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: x-powered-by: HHVM/3.18.6-dev
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: x-content-type-options: nosniff
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: content-disposition: inline; filename=api-result.json
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: x-frame-options: DENY
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: cache-control: private, must-revalidate, max-age=0
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: backend-timing: D=49160 t=1562452168033060
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: vary: Accept-Encoding,X-Seven
2019-07-07 03:59:27.901 30797-30831/fr.free.nrw.commons D/OkHttp: x-varnish: 589036969, 270197130, 954072031, 197187785
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: via: 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: accept-ranges: bytes
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: age: 0
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: x-cache: cp1081 pass, cp2013 pass, cp5010 pass, cp5008 pass
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: x-cache-status: pass
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: server-timing: cache;desc="pass"
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: strict-transport-security: max-age=106384710; includeSubDomains; preload
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: x-analytics: ns=-1;special=Badtitle;loggedIn=1;WMF-Last-Access=06-Jul-2019;https=1
2019-07-07 03:59:27.902 30797-30831/fr.free.nrw.commons D/OkHttp: x-client-ip: 183.82.21.187
2019-07-07 03:59:27.906 30797-30831/fr.free.nrw.commons D/OkHttp: {"batchcomplete":true,"query":{"tokens":{"csrftoken":"7f2e657b7b3ed669908f9534d70c5d515d2120c8+\\"}}}
2019-07-07 03:59:27.906 30797-30831/fr.free.nrw.commons D/OkHttp: <-- END HTTP (101-byte body)
2019-07-07 03:59:27.925 30797-30831/fr.free.nrw.commons D/OkHttp: --> POST https://wikidata.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=wbcreateclaim&errorlang=uselang
2019-07-07 03:59:27.925 30797-30831/fr.free.nrw.commons D/OkHttp: Content-Type: application/x-www-form-urlencoded
2019-07-07 03:59:27.925 30797-30831/fr.free.nrw.commons D/OkHttp: Content-Length: 144
2019-07-07 03:59:27.926 30797-30831/fr.free.nrw.commons D/OkHttp: Cache-Control: no-cache
2019-07-07 03:59:27.926 30797-30831/fr.free.nrw.commons D/OkHttp: entity=Q6019809&snaktype=value&property=P18&value=Indian_Airlines_Flight_605.jpg&uselang=en&token=7f2e657b7b3ed669908f9534d70c5d515d2120c8%2B%5C
2019-07-07 03:59:27.926 30797-30831/fr.free.nrw.commons D/OkHttp: --> END POST (144-byte body)
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: <-- 200 https://www.wikidata.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=wbcreateclaim&errorlang=uselang (807ms)
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: date: Sat, 06 Jul 2019 22:29:29 GMT
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: content-type: application/json; charset=utf-8
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: server: mw1280.eqiad.wmnet
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: x-powered-by: HHVM/3.18.6-dev
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: mediawiki-api-error: noentity
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: p3p: CP="This is not a P3P policy! See https://www.wikidata.org/wiki/Special:CentralAutoLogin/P3P for more info."
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: cache-control: private, must-revalidate, max-age=0
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: content-disposition: inline; filename=api-result.json
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: x-content-type-options: nosniff
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: x-frame-options: DENY
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: backend-timing: D=44813 t=1562452168870923
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: vary: Accept-Encoding,X-Seven
2019-07-07 03:59:28.735 30797-30831/fr.free.nrw.commons D/OkHttp: x-varnish: 645642479, 8330244, 234735478, 199505191
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: via: 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: accept-ranges: bytes
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: age: 0
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: x-cache: cp1087 pass, cp2006 pass, cp5012 pass, cp5008 pass
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: x-cache-status: pass
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: server-timing: cache;desc="pass"
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: strict-transport-security: max-age=106384710; includeSubDomains; preload
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: x-analytics: WMF-Last-Access=06-Jul-2019;WMF-Last-Access-Global=06-Jul-2019;https=1
2019-07-07 03:59:28.736 30797-30831/fr.free.nrw.commons D/OkHttp: x-client-ip: 183.82.21.187
2019-07-07 03:59:28.740 30797-30831/fr.free.nrw.commons D/OkHttp: {"errors":[{"code":"noentity","text":"The \"entity\" parameter must be set.","module":"wbcreateclaim"}],"docref":"See https://www.wikidata.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.","servedby":"mw1280"}
2019-07-07 03:59:28.740 30797-30831/fr.free.nrw.commons D/OkHttp: <-- END HTTP (374-byte body)
2019-07-07 03:59:31.454 30797-30817/fr.free.nrw.commons I/zygote64: Explicit concurrent copying

The text was updated successfully, but these errors were encountered:

maskaravivek · 2019-07-24T06:42:27Z

@dbrant @sharvaniharan It would be great if someone could help us with this. :)

Authenticated cross-wiki calls are the only thing that we haven't been to figure out on our own. It would be great if someone could take a look and help us finish the network overhaul.

This is the branch that has all the changes:
https://github.com/commons-app/apps-android-commons/tree/backend-overhaul

maskaravivek · 2019-08-03T08:21:56Z

This is what I have been trying to use:

https://github.com/commons-app/apps-android-commons/blob/master/app/src/main/java/fr/free/nrw/commons/wikidata/WikidataClient.java

service.postCreateClaim(entityId, snaktype, property, value, "en", csrfTokenClient.getTokenBlocking())

vanshikaarora · 2019-08-03T09:08:19Z

Hey @dbrant I was working on Uploading captions and I also faced similar issue

D/OkHttp: <-- 200 https://www.wikidata.org/w/api.php?action=wbsetlabel&format=json&language=en&value=Testcaptions&&id=M80983832&token=8ff7c2311f05809103d2b74abda121a95d454621%2B%5C (1965ms)
    date: Sat, 03 Aug 2019 09:00:42 GMT
    content-type: application/json; charset=utf-8
    server: mw1314.eqiad.wmnet
    x-powered-by: HHVM/3.18.6-dev
    mediawiki-api-error: mustpostparams
    p3p: CP="This is not a P3P policy! See https://www.wikidata.org/wiki/Special:CentralAutoLogin/P3P for more info."
    cache-control: private, must-revalidate, max-age=0
    content-disposition: inline; filename=api-result.json
    x-content-type-options: nosniff
    x-frame-options: DENY
    backend-timing: D=56207 t=1564822841903216
    vary: Accept-Encoding,X-Seven
    x-varnish: 768625844, 844428123, 849965771, 438817744
    accept-ranges: bytes
    age: 0
    x-cache: cp1085 pass, cp2012 pass, cp5007 pass, cp5008 pass
    x-cache-status: pass
    server-timing: cache;desc="pass"
    strict-transport-security: max-age=106384710; includeSubDomains; preload
    x-analytics: WMF-Last-Access=03-Aug-2019;WMF-Last-Access-Global=03-Aug-2019;https=1
    x-client-ip: 118.185.164.5
D/OkHttp: {"error":{"code":"mustpostparams","info":"The following parameter was found in the query string, but must be in the POST body: token.","*":"See https://www.wikidata.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes."},"servedby":"mw1314"}
    <-- END HTTP (401-byte body)

Can you please look into the issue :-)

tgr · 2019-09-16T10:04:48Z

Both of those error messages seem straightforward: the first complains about the entity parameter not being set, the second about token being set as an (insecure) GET parameter instead of POST. How is this related to request authentication?

lucaswerkmeister · 2019-09-16T10:33:49Z

The entity is in the POST data, though:

D/OkHttp: --> POST https://wikidata.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=wbcreateclaim&errorlang=uselang
D/OkHttp: Content-Type: application/x-www-form-urlencoded
D/OkHttp: Content-Length: 144
D/OkHttp: Cache-Control: no-cache
D/OkHttp: entity=Q6019809&snaktype=value&property=P18&value=Indian_Airlines_Flight_605.jpg&uselang=en&token=_____
D/OkHttp: --> END POST (144-byte body)

So I’m not sure where the error from the Wikibase API is coming from.

tgr · 2019-09-16T12:09:05Z

Hm, yeah, that's strange.
Still can't imagine how it would be related to authentication, though.

bawolff · 2019-09-16T16:51:35Z

This does not seem to be your primary issue (based on error post params seem not recognized at all), but it also looks like you are using a commons csrf token at wikidata. I think you need to fetch the csrf token from wikidata that you use at wikidata

mdholloway · 2019-09-16T18:35:15Z

The user who filed https://phabricator.wikimedia.org/T109175 experienced similar problems. Unfortunately he didn't record the fixes for posterity, but as best I can tell, the issues were around headers and/or cookie handling.

mdholloway · 2019-09-16T18:36:24Z

This does not seem to be your primary issue (based on error post params seem not recognized at all), but it also looks like you are using a commons csrf token at wikidata. I think you need to fetch the csrf token from wikidata that you use at wikidata

Didn't see this until after posting. This sounds highly likely.

maskaravivek · 2019-10-31T07:28:45Z

Thank you, everyone, for your inputs. :)

Based on the discussion I fixed the CSRF token and am now getting the CSRF token from Wikidata itself. This is the new flow:

login to commons site
get central auth token from commons
using this central auth token request for wikidata's csrf token
use this csrf token + a new central auth token to make a call to the wbcreateclaim API.

I am still getting the same error as before.

the first complains about the entity parameter not being set, the second about token being set as an (insecure) GET parameter instead of POST.

I double-checked and all the params are set.

So I’m not sure where the error from the Wikibase API is coming from.

@lucaswerkmeister Yes, i think none of the parameters are being recognized by the API.

Moreover, none of the sandbox examples work for me either. For eg. https://www.wikidata.org/wiki/Special:ApiSandbox#action=wbcreateclaim&entity=Q42&property=P9001&snaktype=novalue doesn't work for me. I get the following error.

{
    "error": {
        "code": "invalid-snak",
        "info": "Invalid snak data.",
        "messages": [
            {
                "name": "wikibase-api-invalid-snak",
                "parameters": [],
                "html": {
                    "*": "Invalid snak data."
                }
            }
        ],
        "*": "See https://www.wikidata.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes."
    },
    "servedby": "mw1343"
}

It looks like the parameters are not being accepted by the API correctly.

maskaravivek · 2019-11-06T12:50:09Z

Here are the changes that I have tried. commons-app/apps-android-commons#3199

maskaravivek · 2019-11-23T10:00:29Z

This is fixed now. Thanks, everyone for your inputs. :)

misaochan mentioned this issue Aug 4, 2019

Items required for v2.12 release commons-app/apps-android-commons#3115

Closed

12 tasks

maskaravivek closed this as completed Nov 23, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Trouble while making authenticated cross wiki calls #21

Trouble while making authenticated cross wiki calls #21

maskaravivek commented Jul 17, 2019

maskaravivek commented Jul 24, 2019

Uh oh!

maskaravivek commented Aug 3, 2019

Uh oh!

vanshikaarora commented Aug 3, 2019

Uh oh!

tgr commented Sep 16, 2019

Uh oh!

lucaswerkmeister commented Sep 16, 2019

Uh oh!

tgr commented Sep 16, 2019

Uh oh!

bawolff commented Sep 16, 2019

Uh oh!

mdholloway commented Sep 16, 2019

Uh oh!

mdholloway commented Sep 16, 2019

Uh oh!

maskaravivek commented Oct 31, 2019

Uh oh!

maskaravivek commented Nov 6, 2019

Uh oh!

maskaravivek commented Nov 23, 2019

Uh oh!

Trouble while making authenticated cross wiki calls #21

Trouble while making authenticated cross wiki calls #21

Comments

maskaravivek commented Jul 17, 2019

maskaravivek commented Jul 24, 2019

Uh oh!

maskaravivek commented Aug 3, 2019

Uh oh!

vanshikaarora commented Aug 3, 2019

Uh oh!

tgr commented Sep 16, 2019

Uh oh!

lucaswerkmeister commented Sep 16, 2019

Uh oh!

tgr commented Sep 16, 2019

Uh oh!

bawolff commented Sep 16, 2019

Uh oh!

mdholloway commented Sep 16, 2019

Uh oh!

mdholloway commented Sep 16, 2019

Uh oh!

maskaravivek commented Oct 31, 2019

Uh oh!

maskaravivek commented Nov 6, 2019

Uh oh!

maskaravivek commented Nov 23, 2019

Uh oh!