USB Detection problem with kernel 4.4.20 (4.4.15+ works fine)

[mypiandrew]

`This concerns a custom compute module board with various USB devices connected through an integrated GL850 hub chip.

I get strange usb errors on a cold boot that result in an FTDI USB-2-UART device often not getting detected correctly. This only appears to happen when using kernel 4.4.20 (maybe 1 in 4 cold boots in a row), but these don't appear on kernel 4.4.15 regardless of how often I cold boot the board.

This doesn't appear to affect any other device on the bus, and the FTDI device is the only device affected by this

root@raspberrypi:~# lsusb -t
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=dwc_otg/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/3p, 480M
            |__ Port 1: Dev 5, If 0, Class=Vendor Specific Class, Driver=smsc95xx, 480M
        |__ Port 2: Dev 4, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 12M
        |__ Port 3: Dev 6, If 0, Class=Mass Storage, Driver=usb-storage, 480M
        |__ Port 4: Dev 7, If 0, Class=Vendor Specific Class, Driver=option, 480M
        |__ Port 4: Dev 7, If 1, Class=Vendor Specific Class, Driver=option, 480M
        |__ Port 4: Dev 7, If 2, Class=Vendor Specific Class, Driver=option, 480M
        |__ Port 4: Dev 7, If 3, Class=Vendor Specific Class, Driver=option, 480M
        |__ Port 4: Dev 7, If 4, Class=Vendor Specific Class, Driver=qmi_wwan, 480M

dmesg diagnostics :

Good :

Linux raspberrypi 4.4.15+ #897 Tue Jul 12 18:38:58 BST 2016 armv6l GNU/Linux

[    4.193382] usb 1-1.2: new full-speed USB device number 4 using dwc_otg
[    4.352995] usb 1-1.2: New USB device found, idVendor=0403, idProduct=6015
[    4.368671] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    4.384556] usb 1-1.2: Product: FT230X Basic UART
[    4.397745] usb 1-1.2: Manufacturer: FTDI
[    4.410167] usb 1-1.2: SerialNumber: DN01VO6I
[    4.423024] usb 1-1.1.1: new high-speed USB device number 5 using dwc_otg
...
...
[   11.639605] usbcore: registered new interface driver usbserial
[   11.639783] usbcore: registered new interface driver usbserial_generic
[   11.639940] usbserial: USB Serial support registered for generic
[   11.803008] usbcore: registered new interface driver ftdi_sio
[   11.803181] usbserial: USB Serial support registered for FTDI USB Serial Device
[   11.896966] usbcore: registered new interface driver cdc_wdm
[   12.006102] ftdi_sio 1-1.2:1.0: FTDI USB Serial Device converter detected
[   12.006502] usb 1-1.2: Detected FT-X

Bad :

Linux raspberrypi 4.4.20+ #908 Wed Sep 7 14:35:04 BST 2016 armv6l GNU/Linux

[    4.216782] usb 1-1.2: unable to read config index 0 descriptor/start: -32
[    4.232586] usb 1-1.2: chopping to 0 config(s)
[    4.248247] systemd[1]: Inserted module 'ipv6'
[    4.266318] systemd[1]: Set hostname to <raspberrypi>.
[    4.281041] usb 1-1.2: string descriptor 0 read error: -32
[    4.295902] usb 1-1.2: New USB device found, idVendor=0403, idProduct=6015
[    4.311878] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    4.328049] usb 1-1.1.1: new high-speed USB device number 5 using dwc_otg
[    4.357016] usb 1-1.2: no configuration chosen from 0 choices
[    4.371839] usb 1-1.2: No support over 500mA

[popcornmix]

Can you identify the exact update which caused this. See:
https://github.com/Hexxeh/rpi-firmware/commits/master

If you click on each commit the end of the url contains a git hash. Run
sudo rpi-update <hash>
to revert back to that version. Report the first version with this error.

[mypiandrew]

Will do :-)

Give me some time to work on it this week and I'll get back to you asap

[mypiandrew]

4.4.16+ has the USB problem (first one I tried!)

root@raspberrypi:~# rpi-update 3dec77a63fbe71aea8feaa8ca600e7c723bb3a7b
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Your firmware is already up to date
root@raspberrypi:~# uname -a
Linux raspberrypi 4.4.15+ #897 Tue Jul 12 18:38:58 BST 2016 armv6l GNU/Linux
root@raspberrypi:~# rpi-update e7178c06ac5b527085ceb380aa56f1ed52558ea0
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
This update bumps to rpi-4.4.y linux tree
Be aware there could be compatibility issues with some drivers
Discussion here:
https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=144087
##############################################################
 *** Downloading specific firmware revision (this will take a few minutes)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168    0   168    0     0    314      0 --:--:-- --:--:-- --:--:--   315
100 51.1M    0 51.1M    0     0   977k      0 --:--:--  0:00:53 --:--:-- 1085k
 *** Updating firmware
 *** Updating kernel modules
 *** depmod 4.4.16-v7+
 *** depmod 4.4.16+
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to e7178c06ac5b527085ceb380aa56f1ed52558ea0
 *** A reboot is needed to activate the new firmware
root@raspberrypi:~# reboot

[    4.466767] usb 1-1.2: new full-speed USB device number 4 using dwc_otg
[    4.824881] usb 1-1.2: unable to read config index 0 descriptor/all
[    4.839972] usb 1-1.2: can't read configurations, error -32
[    4.905373] usb 1-1.1.1: new high-speed USB device number 5 using dwc_otg
[    5.036048] usb 1-1.1.1: New USB device found, idVendor=0424, idProduct=ec00
[    5.052029] usb 1-1.1.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    5.084720] smsc95xx v1.0.4
[    5.121934] systemd[1]: Cannot add dependency job for unit display-manager.service, ignoring: Unit display-manager.service failed to load: No such file or directory.
[    5.155502] usb 1-1.2: new full-speed USB device number 6 using dwc_otg
[    5.178146] systemd[1]: Starting Forward Password Requests to Wall Directory Watch.
[    5.200823] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[    5.222063] systemd[1]: Expecting device dev-ttyAMA0.device...
[    5.240075] smsc95xx 1-1.1.1:1.0 eth0: register 'smsc95xx' at usb-20980000.usb-1.1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:8a:d3:39
[    5.272920] systemd[1]: Starting Remote File Systems (Pre).
[    5.300264] systemd[1]: Reached target Remote File Systems (Pre).
[    5.317233] usb 1-1.2: device descriptor read/all, error -32

Full dmesg :
http://pastebin.com/0b48HzXK

[mypiandrew]

Going back to 4.4.15 fixes it again:

root@raspberrypi:~# rpiupdate 50f4bc47fc16c05c657c6f6f15719e57917f9b81
-bash: rpiupdate: command not found
root@raspberrypi:~# rpi-update 50f4bc47fc16c05c657c6f6f15719e57917f9b81
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
This update bumps to rpi-4.4.y linux tree
Be aware there could be compatibility issues with some drivers
Discussion here:
https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=144087
##############################################################
 *** Downloading specific firmware revision (this will take a few minutes)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168    0   168    0     0    313      0 --:--:-- --:--:-- --:--:--   314
100 51.0M    0 51.0M    0     0   946k      0 --:--:--  0:00:55 --:--:-- 1046k
 *** Updating firmware
 *** Updating kernel modules
 *** depmod 4.4.15-v7+
 *** depmod 4.4.15+
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to 50f4bc47fc16c05c657c6f6f15719e57917f9b81
 *** A reboot is needed to activate the new firmware
root@raspberrypi:~# reboot

[popcornmix]

Can you clarify that 3dec77a63fbe71aea8feaa8ca600e7c723bb3a7b is the first firmware with the bug and 50f4bc47fc16c05c657c6f6f15719e57917f9b81 works correctly?

[mypiandrew]

Correct, the problem re-occurred on the next re-boot after the reboot command issued at the end of the update when moving to 4.4.16 and vanished immediately when moving back to 4.4.15 as above

4.4.17 (82d8b8040558ffcfda5a0714400549e819054c28) also has the USB problem FWIW

[popcornmix]

Unfortunately that makes no sense 3dec77a63fbe71aea8feaa8ca600e7c723bb3a7b only contained a trivial fix for deinterlace in the firmware with no kernel change.

Can you double check your results?

[mypiandrew]

OK, I'll reboot the board 10 times in a row on 4.4.15 and 4.4.16 and report back

Will be tomorrow now as I have to switch task

[mypiandrew]

Just for interest I've just done on 4.4.15+ 10 reboot (5 soft 5 power cycles) and all work AOK

Soft reboot (reboot by command line)
1 OK
2 OK
3 OK
4 OK
5 OK

Hard reboot (power off/on)
1 OK
2 OK
3 OK
4 OK
5 OK

Will be a bit more ordered tomorrow

[popcornmix]

3dec77a63fbe71aea8feaa8ca600e7c723bb3a7b is 4.4.15
50f4bc47fc16c05c657c6f6f15719e57917f9b81is 4.4.15
4.4.16 didn't come until two commits later.
Can you make sure you are testing the two commits you have referenced.
uname -a will tell you the new kernel version after reboot. Don't run it after rpi-update without rebooting.

[mypiandrew]

Does this help?

http://pastebin.com/NNg1WSxJ

I power cycled the between the first two logins' then upgraded and did a command line reboot, then downgraded and again did a command line reboot.

It's very strange...

[popcornmix]

But that log updates to e7178c06ac5b527085ceb380aa56f1ed52558ea0 which isn't the revision you said the problem started with.
I want two consecutive firmware hashes, the older without no problem and the newer with the problem so we can investigate exactly what changed.

I suspect the answer will be
6ab8282964b77388e12cc7f5de7c3d7b72cdcf4c last firmware that worked
e7178c06ac5b527085ceb380aa56f1ed52558ea0 first firmware with problem

but we really need to get that confirmed.

[mypiandrew]

I'm sorry for the confusing time here. I'm clearly not the best with Git :(

Thanks for explicitly stating what you want me to do, appreciate it

[mypiandrew]

You are correct

6ab8282964b77388e12cc7f5de7c3d7b72cdcf4c <= Works
e7178c06ac5b527085ceb380aa56f1ed52558ea0 <= Doesn't work

If there are any "in between" then let me know the numbers and I'll check (I'm clearly hopeless at that part!)

For checking here's the log

http://pastebin.com/CXNHUb7n

Thanks for your help, hope you can fix it

[DougieLawson]

I'm seeing a similar problem with a couple of my USB WiFi dongles and with my B+ that has the filesystem on a USB hard drive.

On my A+ the WiFi keeps breaking (usually around 1:00am until 8:00am when I fix it).

Sep 15 08:10:07 aplus kernel: [35066.553872] ieee80211 phy0: rt2800usb_tx_sta_fi
fo_read_completed: Warning - TX status read failed -71
Sep 15 08:10:07 aplus kernel: [35066.553976] ieee80211 phy0: rt2800usb_tx_sta_fi
fo_read_completed: Warning - TX status read failed -108

On one RPi I ended up having to wire it to my network as a I couldnt get any WiFi dongle to run. Both

Bus 001 Device 005: ID 148f:5370 Ralink Technology, Corp. RT5370 Wireless Adapter

and

Bus 001 Device 005: ID 0bda:8179 Realtek Semiconductor Corp.

failed to work.

They both work OK with BRANCH=next rpi-update (4.4.17) kernel (which is now running on five out of my eleven raspberries).

On my B+ after I'd tried the experimental bootcode.bin to see if I could boot from the hard drive (with just bootcode.bin on the SDCard) I couldn't get it running when I reverted to 4.4.20 #908 so I reverted to 4.4.17 and that's what it's running today.

[mypiandrew]

Hey Dougie

I found 4.4.15 was the last that worked for me, in fact I had to back up to 4.1.21 otherwise I hit a problem with concurrent ttyS0 and SPI access.

Would be interested to see the result of lsusb -t on your system as on mine only the 12M device appeared to be affected and not any of the 480M devices.

[DougieLawson]

@mypiandrew at the moment I'm not going to do any additional testing. As far as I'm concerned 4.4.20 #908 is poisonous and I won't be going near it until a later version. My B+ runs my webserver and a bunch of other stuff so I'd like it to be a little bit stable.

[popcornmix]

@DougieLawson did your problem start with 4.4.16 (e7178c06ac5b527085ceb380aa56f1ed52558ea0) ? If not it is probably unrelated to the original poster's problem and a new issue would be less confusing.

There is a kernel bump to 4.4.21 available with rpi-update.
While I have no evidence that fixes the problem, it may be worth trying.

[mypiandrew]

As far as I can tell, this has been fixed as of

Linux raspberrypi 4.4.43-v7+ #948 SMP Sun Jan 15 22:20:07 GMT 2017 armv7l GNU/Linux

I can't vouch for @DougieLawson but I've been running my CM3 with the same setup using the above rpi update and I've not seen a occurrence.