Generally you want to use rem for setting the font-size of the elements and to controll overall spacing.

em is often used for the padding of elements. For example buttons that exist in different sizes.