usb-serial: fix shutdown / device_unregister order

Ensure that the ->port_remove() callbacks get called before the
->shutdown() callback which makeing the order symmetric with
->attach() being called before ->port_probe().

Signed-off-by: Jim Radford <radford@blackbean.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>