drm/ttm: stop destroying pinned ghost object

Daniel added a warning for this, but we were abusing that behavior here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Fixes: 57fcd550 ("drm/ttm: Warn on pinning without holding a reference")
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/series/84456/