ASoC: samsung: spdif: remove unnecessary check of mem_res

The resource is checked in probe function, so there is
no need do this check in remove function.

Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Link: https://lore.kernel.org/r/20220510124749.2663874-1-yangyingliang@huawei.com


Signed-off-by: Mark Brown <broonie@kernel.org>