drm/armada: remove obsolete fb unreferencing kfifo and workqueue

Remove the obsolete fb unreferencing system that is no longer used
since we've transitioned to atomic modeset.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>