Make sure to link librt if we need to.