Maybe is due to usb powersave? I turned off it on that port ( echo "-1" > /sys/bus/usb/devices/usb1/power/autosuspend ) to check it.