USB: further fix for usb-serial

this is a fix for the outstanding usb-serial issues.

Signed-off-by: Oliver Neukum <oneukum@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>