Matching a driver to a device has both generic parts, and parts which
are specific to PHY devices. Move the PHY specific parts into
phy_device.

Signed-off-by: Andrew Lunn <andrew@lunn.ch>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>