ASoC: SOF: simplify snd_sof_device_remove()

The commit "ASoC: SOF: core: Unregister machine driver before IPC and
debugfs" moved the call to unregister the machine driver to be done
before freeing the IPC. With this change, we can safely move the call to
snd_sof_remove() inside the same if() block just above.

Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Reviewed-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Reviewed-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Link: https://lore.kernel.org/r/20220307181111.49392-3-ranjani.sridharan@linux.intel.com


Signed-off-by: Mark Brown <broonie@kernel.org>