Skip to content

JSON formatting for application/vnd.api+json does not work #382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mtift opened this issue May 16, 2024 · 12 comments · Fixed by #501
Closed

JSON formatting for application/vnd.api+json does not work #382

mtift opened this issue May 16, 2024 · 12 comments · Fixed by #501

Comments

@mtift
Copy link

mtift commented May 16, 2024

If I include the following in my .http file, I get formatted results:

GET https://jsonplaceholder.typicode.com/posts/3

This suggests to me that I have everything installed correctly, including jq for formatting. But if I include the following config (based on #143 (comment)):

      local ok, rest = pcall(require, "rest-nvim")
      if not ok then
        return
      end

      rest.setup {
        result = {
          behavior = {
            formatters = {
              json = "jq",
              vnd = "jq",
            },
          },
        },
      }

And query a JSON:API (from a Drupal site), then I get the following warning before the results come back unformatted:

[rest.nvim] INFO: Could not find a formatter for the body type vnd.api+json returned in the request, the results will not be formatted

Any idea what I'm doing wrong?

@boltlessengineer
Copy link
Contributor

rest.nvim can’t understand non-standard mimetypes like application/vnd.api+json.
You can use [“vnd.api+json”] = “jq” instead to format with jq from v2.
v3 currently doesn’t support custom mimetypes

@mawkler
Copy link

mawkler commented Oct 12, 2024

Is this still supposed to work? I get this error on startup:

[rest.nvim] Unrecognized configs found in setup: { "result" }

I've tried a bunch of variants like changing result to response, but I get the same behaviour. Looking at the code it seems that the there's only response.hooks.decode and response.hooks.format.

I'm on commit 97cc922.

@boltlessengineer
Copy link
Contributor

boltlessengineer commented Oct 12, 2024

@mawkler setup above is using legacy config. So yes, there's no result field in config now.
Since v3 release, you don't need to configure formatter specific for rest.nvim. rest.nvim can use the formatters you configured for json filetype using 'formatprg' or 'formatexpr' Neovim options.
See this comment as a reference.

btw for application/vnd.api+json mimetype, you can use this config as a workaround.

vim.api.nvim_create_autocmd("FileType", {
    pattern = "vnd.api+json",
    callback = function (ev)
        vim.bo[ev.buf].filetype = "json"
    end
})

@mawkler
Copy link

mawkler commented Oct 12, 2024

Thank you for the response! However, I get the following error message:

Can't set filetype to 'vnd.api+json' (E474: Invalid argument). Formatting is canceled

Here's the request file that I'm using:

GET https://httpbin.org/response-headers?content-type=application/vnd.api%2Bjson

It seems to be the + that's an invalid filetype character.

Here's the minimal config that I'm using:

Click to expand
local lazypath = vim.fn.stdpath('data') .. '/lazy/lazy.nvim'
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    'git',
    'clone',
    '--filter=blob:none',
    'https://github.com/folke/lazy.nvim.git',
    '--branch=stable', -- latest stable release
    lazypath,
  })
end
vim.opt.rtp:prepend(lazypath)

vim.keymap.set('n', '<c-q>', vim.cmd.quitall)

vim.api.nvim_create_autocmd('FileType', {
  pattern = 'vnd.api+json',
  group = augroup,
  callback = function(ev)
    vim.bo[ev.buf].filetype = 'json'
  end
})

require('lazy').setup({
  {
    "rest-nvim/rest.nvim",
    dependencies = 'nvim-treesitter/nvim-treesitter',
  },
})

@boltlessengineer
Copy link
Contributor

This has been fixed now.
Be sure to disable auto url encoding in config: request.hooks.encode_url = false

example config:

vim.g.rest_nvim = {
    request = {
        hooks = {
            encode_url = false,
        },
    },
}

@mawkler
Copy link

mawkler commented Dec 30, 2024

@boltlessengineer Thank you for fixing this! I no longer get the error message.

Is it possible to also set the filetype of the response body to JSON in the case of application/vnd.api+json, just like for regular application/json responses?

@boltlessengineer
Copy link
Contributor

Is it possible to also set the filetype of the response body to JSON in the case of application/vnd.api+json, just like for regular application/json responses?

Current version will set filetype to json for application/vnd.api+json content-type.

@mawkler
Copy link

mawkler commented Dec 30, 2024

@boltlessengineer Hmm you're right, I tested against httpbin like so, and it does work:

GET https://httpbin.org/response-headers?content-type=application/vnd.api%2bjson

However, it does not work for responses from the real server that I'm sending requests to at work. I tried imitating the full response via httpbin.org/response-headers, but when checking :InspectTree rest-nvim still doesn't interpret the response body of the real server as JSON (even though it is valid JSON, I checked), which is very odd to me.

The only difference that I can see while comparing the responses is that the real server does not set a content-length header, while httpbin does. Do you think that could be the reason why rest-nvim doesn't format the response body of the real server as JSON?

@boltlessengineer
Copy link
Contributor

@mawkler I don't think the absence of content-length header can be an issue. I made a simple web server with golang that responds with exactly this output but can't reproduce the issue.

HTTP/1.1 200 OK
Content-Type: application/json

{"message":"Test response"}

It's quite hard to help you without actually testing with the response you are dealing with. It would be helpful if you share the actual response but I understand if you can't because of security reasons. If you don't mind sharing the response from the server you are working with, please share the output of following command:

curl -sL -v https://example.com -X GET -H "User-Agent: rest.nvim v3.9.1"

replace https://example.com to actual server uri and add some payloads if you need any.

Alternatively, can you share your log output when running the request?
You can open the log file with :Rest logs command and you can enable debug mode with following config:

vim.g.rest_nvim = {
    _log_level = vim.log.levels.DEBUG,
}

@mawkler
Copy link

mawkler commented Jan 5, 2025

Here's the output of curl -sL -v curl https://httpbin.org/uuid -X GET -H "User-Agent: rest.nvim v3.9.1" for me:

Click to expand
❯ curl -sL -v curl https://httpbin.org/uuid -X GET -H "User-Agent: rest.nvim v3.9.1"
* Could not resolve host: curl
* shutting down connection #0
* Host httpbin.org:443 was resolved.
* IPv6: (none)
* IPv4: 34.200.57.114, 50.19.58.113, 34.197.122.172, 3.210.94.60
*   Trying 34.200.57.114:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 / secp256r1 / rsaEncryption
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=httpbin.org
*  start date: Aug 20 00:00:00 2024 GMT
*  expire date: Sep 17 23:59:59 2025 GMT
*  subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Connected to httpbin.org (34.200.57.114) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://httpbin.org/uuid
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: httpbin.org]
* [HTTP/2] [1] [:path: /uuid]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [user-agent: rest.nvim v3.9.1]
> GET /uuid HTTP/2
> Host: httpbin.org
> Accept: */*
> User-Agent: rest.nvim v3.9.1
>
* Request completely sent off
< HTTP/2 200
< date: Sun, 05 Jan 2025 13:51:53 GMT
< content-type: application/json
< content-length: 53
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
<
{
"uuid": "f4cab9ac-42dd-41cf-b2f4-bfe7f2ba9505"
}

And here are my logs:

Click to expand
[START][2025-01-05 14:57:20] rest.nvim logging initiated
DEBUG | 2025-01-05 14:57:20 | ...local/share/nvim/lazy/rest.nvim/lua/rest-nvim/dotenv.lua:80 | searching for requests.env file
DEBUG | 2025-01-05 14:57:20 | ...local/share/nvim/lazy/rest.nvim/lua/rest-nvim/dotenv.lua:93 | searching for .env file
DEBUG | 2025-01-05 14:57:20 | ...local/share/nvim/lazy/rest.nvim/lua/rest-nvim/dotenv.lua:104 | found .env file: .env
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:61 | load file `.env` to context
DEBUG | 2025-01-05 14:57:22 | ...hare/nvim/lazy/rest.nvim/lua/rest-nvim/parser/dotenv.lua:67 | { "COOKIE_NAME=*********", 'COOKIE_VALUE="*********"', "BASE_URL=https://*********.com", "COMPANY=*******", "JOB_NAME=*******" }
DEBUG | 2025-01-05 14:57:22 | ...hare/nvim/lazy/rest.nvim/lua/rest-nvim/parser/dotenv.lua:72 | set COOKIE_NAME=*********
DEBUG | 2025-01-05 14:57:22 | ...hare/nvim/lazy/rest.nvim/lua/rest-nvim/parser/dotenv.lua:72 | set COOKIE_VALUE=*********
DEBUG | 2025-01-05 14:57:22 | ...hare/nvim/lazy/rest.nvim/lua/rest-nvim/parser/dotenv.lua:72 | set BASE_URL=https://*********.com
DEBUG | 2025-01-05 14:57:22 | ...hare/nvim/lazy/rest.nvim/lua/rest-nvim/parser/dotenv.lua:72 | set COMPANY=*********
DEBUG | 2025-01-05 14:57:22 | ...hare/nvim/lazy/rest.nvim/lua/rest-nvim/parser/dotenv.lua:72 | set JOB_NAME=*******
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: BASE_URL
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: COMPANY
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: COOKIE_NAME
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: COOKIE_VALUE
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: JOB_NAME
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: JOB_NUMBER
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: base_url
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: company
DEBUG | 2025-01-05 14:57:22 | .../share/nvim/lazy/rest.nvim/lua/rest-nvim/parser/init.lua:480 | find request node child: res_handler_script
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: cookie_name
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: cookie_name
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/context.lua:119 | resolving variable: cookie_value
DEBUG | 2025-01-05 14:57:22 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:204 | loading cookies for request:https://*********.com/*********
DEBUG | 2025-01-05 14:57:22 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:180 | cookie AWSALB with domain .*********.com and path / matched to url: https://*********.com/*********
DEBUG | 2025-01-05 14:57:22 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:180 | cookie AWSALBCORS with domain .*********.com and path / matched to url: https://*********.com/*********
DEBUG | 2025-01-05 14:57:22 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:180 | cookie ******* with domain .*********.com and path / matched to url: https://*********.com/*********
DEBUG | 2025-01-05 14:57:22 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:180 | cookie ********* with domain .*********.com and path / matched to url: https://*********.com/*********
DEBUG | 2025-01-05 14:57:22 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/request.lua:46 | running request:requests#1
INFO | 2025-01-05 14:57:22 | ...re/nvim/lazy/rest.nvim/lua/rest-nvim/client/curl/cli.lua:33 | { "curl", "-sL", "-v", "https://*********.com/*******", "-X", "POST", "-H", "Cookie: *********=*********", "-H", "Authorization: X-Cookie ********", "-H", "Content-Type: application/*******+json", "-H", "User-Agent: rest.nvim v3.9.1", "-b", "AWSALB=*******", "-b", "AWSALBCORS=*******", "-b", '*******="*******"', "-b", '*******="*********"', "--data-raw", '{ "panes": { "card": { "fields": [] }, "table": { "fields": [] } } }', "-w", "%{stderr}? time_total:%{time_total}\n? size_download:%{size_download}\n" }
ERROR | 2025-01-05 14:57:23 | ...re/nvim/lazy/rest.nvim/lua/rest-nvim/client/curl/cli.lua:81 | Error while parsing verbose curl output:

DEBUG | 2025-01-05 14:57:23 | ...re/nvim/lazy/rest.nvim/lua/rest-nvim/client/curl/cli.lua:112 | transforming stat pair as time: time_total 0.406690
DEBUG | 2025-01-05 14:57:23 | ...re/nvim/lazy/rest.nvim/lua/rest-nvim/client/curl/cli.lua:109 | transforming stat pair as size: size_download 1621
INFO | 2025-01-05 14:57:23 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/request.lua:73 | request success
DEBUG | 2025-01-05 14:57:23 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/request.lua:76 | run 1 handers
INFO | 2025-01-05 14:57:23 | ...ocal/share/nvim/lazy/rest.nvim/lua/rest-nvim/request.lua:80 | handler done
DEBUG | 2025-01-05 14:57:23 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:68 | parsing set-cookie: AWSALB *******
DEBUG | 2025-01-05 14:57:23 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:102 | cookie parsed from Set-Cookie Header: {
  domain = ".*********.com",
  expires = 1736690242,
  name = "AWSALB",
  path = "/",
  value = "****"
}
DEBUG | 2025-01-05 14:57:23 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:68 | parsing set-cookie: AWSALBCORS ****
DEBUG | 2025-01-05 14:57:23 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:102 | cookie parsed from Set-Cookie Header: {
  domain = ".*********.com",
  expires = 1736690242,
  name = "AWSALBCORS",
  path = "/",
  samesite = "None",
  secure = true,
  value = "****"
}
DEBUG | 2025-01-05 14:57:23 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:68 | parsing set-cookie: ********* "*********"
DEBUG | 2025-01-05 14:57:23 | ...l/share/nvim/lazy/rest.nvim/lua/rest-nvim/cookie_jar.lua:102 | cookie parsed from Set-Cookie Header: {
  domain = ".*********.com",
  expires = -1,
  name = "*********",
  path = "/",
  value = '"****"'
}
DEBUG | 2025-01-05 14:57:23 | ...al/share/nvim/lazy/rest.nvim/lua/rest-nvim/ui/result.lua:67 | {
  code = 200,
  text = "",
  version = "HTTP/2"
}
DEBUG | 2025-01-05 14:57:23 | ....local/share/nvim/lazy/rest.nvim/lua/rest-nvim/utils.lua:304 | formatting with `gq`
DEBUG | 2025-01-05 14:57:23 | ....local/share/nvim/lazy/rest.nvim/lua/rest-nvim/utils.lua:337 | can't find formatexpr or formatprg for json filetype. Formatting is canceled
DEBUG | 2025-01-05 14:57:23 | ...al/share/nvim/lazy/rest.nvim/lua/rest-nvim/ui/result.lua:120 | {
  ["cache-control"] = { "no-cache,no-store" },
  ["content-language"] = { "en-US-x-lvariant-W" },
  ["content-type"] = { "application/*******+json; charset=utf-8; version=6.0" },
  date = { "Sun, 05 Jan 2025 13:57:22 GMT" },
  expires = { "Thu, 01 Jan 1970 00:00:00 GMT" },
  ["*****-concurrency-control"] = { "*******" },
  ["*****-cookie"] = { "*********" },
  ["*****-requestid"] = { "*******" },
  server = { "Jetty(-)" },
  ["set-cookie"] = { "AWSALB=*********; Expires=Sun, 12 Jan 2025 13:57:22 GMT; Path=/", "AWSALBCORS=****; Expires=Sun, 12 Jan 2025 13:57:22 GMT; Path=/; SameSite=None; Secure", '*********="*********";Path=/;Secure;HttpOnly' },
  vary = { "Accept,Accept-Language" }
}
DEBUG | 2025-01-05 14:57:23 | ...al/share/nvim/lazy/rest.nvim/lua/rest-nvim/ui/result.lua:125 | { { "cache-control", { "no-cache,no-store" },
    <metatable> = <1>{}
  }, { "content-language", { "en-US-x-lvariant-W" },
    <metatable> = <table 1>
  }, { "content-type", { "application/*******+json; charset=utf-8; version=6.0" },
    <metatable> = <table 1>
  }, { "date", { "Sun, 05 Jan 2025 13:57:22 GMT" },
    <metatable> = <table 1>
  }, { "expires", { "Thu, 01 Jan 1970 00:00:00 GMT" },
    <metatable> = <table 1>
  }, { "*******-concurrency-control", { "*******" },
    <metatable> = <table 1>
  }, { "*******-cookie", { "*********" },
    <metatable> = <table 1>
  }, { "*******-requestid", { "*******" },
    <metatable> = <table 1>
  }, { "server", { "Jetty(-)" },
    <metatable> = <table 1>
  }, { "set-cookie", { "AWSALB=*******; Expires=Sun, 12 Jan 2025 13:57:22 GMT; Path=/", "AWSALBCORS=*****; Expires=Sun, 12 Jan 2025 13:57:22 GMT; Path=/; SameSite=None; Secure", '*********="*********";Path=/;Secure;HttpOnly' },
    <metatable> = <table 1>
  }, { "vary", { "Accept,Accept-Language" },
    <metatable> = <table 1>
  } }

I noticed that it says can't find formatexpr or formatprg for json filetype. Formatting is canceled. Could that be the smoking gun?

@boltlessengineer
Copy link
Contributor

Yes! Thank you for providing those.

So rest.nvim does know the response body type is json but it isn't formatting because you don't have any formatters attached to json filetype. See #414 (comment) to set jq as json formatter (you need jq or other json formatter installed in your machine.)

but when checking :InspectTree rest-nvim still doesn't interpret the response body of the real server as JSON

Doesn't your response body start with { or [ ? (whitespace matters).
tree-sitter-http parser only parse response body matching /^[{\[]\s+/ to json. So that can be the issue.

I think setting json formatter will fix this issue because it will format json body to start with correct pattern.

@mawkler
Copy link

mawkler commented Jan 7, 2025

That solved the issue for me, thank you! After setting vim.bo.formatexpr and vim.bo.formatprg :InspectTree also shows JSON as the language of the response body.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants