`os_version` invalid error when trying to test iOS 13 #226

mgol · 2020-05-07T21:46:38Z

Expected Behavior

I'm trying to test iOS 11, 12 & 13; they should all spawn.

Actual Behavior

iOS 13 doesn't spawn.

Steps to Reproduce the Problem

Provide the following browsers field in the config:

  "browsers": [
  	{
  		"os": "iOS",
  		"os_version": "11",
  		"device": "iPhone X"
  	},
  	{
  		"os": "iOS",
  		"os_version": "12",
  		"device": "iPhone XS"
  	},
  	{
  		"os": "iOS",
  		"os_version": "13",
  		"device": "iPhone 11"
  	}
  ]

Run browserstack-runner.

browserstack.json

Platform details

browserstack-runner version: 0.9.1
node version: v12.16.3
os type and version: macOS 10.15.4 (19E287)

Details

With the above config I'm getting an error:

Error from BrowserStack:  Error: Validation Failed - `os_version` invalid

iOS 11 & 12 run then tests correctly, iOS 13 doesn't. I'm providing correct os_version & device according to the v4 API.

The text was updated successfully, but these errors were encountered:

iOS 13 is skipped for now due to a bug in browserstack-runner, see browserstack/browserstack-runner#226. IE 9 && iOS 10 are skipped because I can't get them to show any output, they just times out.

iOS 13 is skipped for now due to a bug in browserstack-runner, see browserstack/browserstack-runner#226. IE 9, iOS 10 & safari_previous are skipped because I can't get them to show any output, they just times out.

mgol · 2020-05-08T08:33:53Z

I have many more issues, actually. This one at least shows an error, although it'd be much easier to follow up if it showed the browser entry that's problematic. However, entries for iOS 10 & IE 9 just don't fire up for me at all. These ones:

browsers: [
		"ie_9",
		{
			"os": "iOS",
			"os_version": "10.3",
			"device": "iPhone 7",
 			"real_mobile": false
 		}
]

I also tried a different iOS 10, same result:

browsers: [
		{
			"os": "iOS",
			"os_version": "10",
			"device": "iPhone 7"
 		}
]

For these browsers, I don't see any error when they get spawned, all I see is that they don't show up in the console and at the end I see a grouped error message; example:

Using config: /Users/mgolebiowski/projects/public/jquery/jquery-color/build/browserstack-legacy.json
Launching 9 worker(s) for 9 run(s).
[Windows 7, Firefox 75.0] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 571ms
[OS X Mojave, Safari 12.1] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 349ms
[Windows 8.1, Edge 81.0] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 420ms
[Windows 8, Chrome 81.0] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 345ms
[iOS 8.3, iPhone 6] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 372ms
[iOS 9.3, iPhone 6S] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 317ms
[Windows 7, Internet Explorer 10.0] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 372ms
[iOS 7.0, iPhone 5S] Passed: 56 tests, 56 passed, 0 failed, 0 skipped; ran for 335ms
All tests done, failures: 1.

You can see that in the jquery-color repo. Relevant files are in the build directory, you can also run it via npm run browserstack.

Krinkle · 2021-04-11T22:15:13Z

I'm running into the same issue as well. It seems iOS 11, 13 and 14 are not considered valid, despite being listed at https://www.browserstack.com/automate/browsers.json and https://www.browserstack.com/list-of-browsers-and-platforms/js_testing.

Launching 8 worker(s) for 8 run(s).
Error from BrowserStack:  Error: Validation Failed - `os_version` invalid
    at IncomingMessage.<anonymous> (/qunit/node_modules/browserstack/lib/client.js:73:9)
    at IncomingMessage.emit (events.js:194:15)
    at endReadableNT (_stream_readable.js:1103:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
Error from BrowserStack:  Error: Validation Failed - `device` invalid
    at IncomingMessage.<anonymous> (/qunit/node_modules/browserstack/lib/client.js:73:9)
    at IncomingMessage.emit (events.js:194:15)
    at endReadableNT (_stream_readable.js:1103:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
Error from BrowserStack:  Error: Validation Failed - `os_version` invalid
    at IncomingMessage.<anonymous> (/qunit/node_modules/browserstack/lib/client.js:73:9)
    at IncomingMessage.emit (events.js:194:15)
    at endReadableNT (_stream_readable.js:1103:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
[ios 12] Passed: 225 tests, 220 passed, 0 failed, 5 skipped; ran for 1739ms
[ios 8] Passed: 219 tests, 211 passed, 0 failed, 8 skipped; ran for 1227ms
[ios 7] Passed: 215 tests, 207 passed, 0 failed, 8 skipped; ran for 10025ms
[ios 10] Passed: 223 tests, 218 passed, 0 failed, 5 skipped; ran for 2012ms
[ios 9] Passed: 220 tests, 215 passed, 0 failed, 5 skipped; ran for 4378ms
All tests done, failures: 0.

Config:


  "browsers": [
    {
      "device": null,
      "os": "ios",
      "os_version": "14"
    },
    {
      "device": null,
      "os": "ios",
      "os_version": "13"
    },
    {
      "device": null,
      "os": "ios",
      "os_version": "12"
    },
    {
      "device": "iPhone SE",
      "os": "ios",
      "os_version": "11"
    },
    {
      "device": null,
      "os": "ios",
      "os_version": "10"
    },
    {
      "device": null,
      "os": "ios",
      "os_version": "9"
    },
    {
      "device": null,
      "os": "ios",
      "os_version": "8"
    },
    {
      "device": null,
      "os": "ios",
      "os_version": "7"
    }
  ]

Krinkle · 2021-04-11T22:37:55Z

Most of the Android versions don't seem to work either. I've tried many variations (with and without .0), but only 4.3 and 5.0 seem to work. versions 6, 7, 8, 9, 10, and 11 all yield:

Error from BrowserStack:  Error: Validation Failed - `os_version` invalid

  "browsers": [
    {
      "device": null,
      "os": "android",
      "os_version": "11.0"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "10.0"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "9.0"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "8.0"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "7"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "6"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "5"
    },
    {
      "device": null,
      "os": "android",
      "os_version": "4.3"
    }
  ]

In the builds overview I see:

Android 4.3 (Amazon Kindle Fire HDX 7)
Android 5.0 (Null)

And the others never show up.

Namrata-m13 · 2021-04-12T03:36:54Z

@Krinkle I believe os_version needs to be specified as '11.0' or '12.0' instead of '11' or '12' both on iOS as well as Android devices.
However, to avoid these errors, you can also configure your tests exactly as per the configuration mentioned here, https://github.com/karma-runner/karma-browserstack-launcher#configuration

mgol · 2021-06-06T22:09:45Z

#244 should have fixed the issue (though I haven't tested it personally). Closing.

mgol closed this as completed Jun 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`os_version` invalid error when trying to test iOS 13 #226

`os_version` invalid error when trying to test iOS 13 #226

mgol commented May 7, 2020

mgol commented May 8, 2020

Krinkle commented Apr 11, 2021

Krinkle commented Apr 11, 2021

Namrata-m13 commented Apr 12, 2021

mgol commented Jun 6, 2021

os_version invalid error when trying to test iOS 13 #226

os_version invalid error when trying to test iOS 13 #226

Comments

mgol commented May 7, 2020

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

browserstack.json

Platform details

Details

mgol commented May 8, 2020

Krinkle commented Apr 11, 2021

Krinkle commented Apr 11, 2021

Namrata-m13 commented Apr 12, 2021

mgol commented Jun 6, 2021

`os_version` invalid error when trying to test iOS 13 #226

`os_version` invalid error when trying to test iOS 13 #226