This is a more proper isolation of the code. Done in preparation of making
PRM an individual driver.

Signed-off-by: Tero Kristo <t-kristo@ti.com>