如何知道纱线公平份额调度程序是否发生抢占?

Question

有没有办法确定抢占机制是否已在 YARN 上触发？

也许在 YARN 资源管理器或日志中？

Answer 1

如果日志级别设置为info则应在 YARN 资源管理器日志中看到此信息。

// Warn application about containers to be killed
for (RMContainer container : containers) {
FSAppAttempt app = scheduler.getSchedulerApp(
container.getApplicationAttemptId());
LOG.info("Preempting container " + container +
" from queue " + app.getQueueName());
app.trackContainerForPreemption(container);
}

在 https://github.com/apache/hadoop/.../scheduler/fair/FSPreemptionThread.java#L143。

如果您有要debug的日志级别，您还将看到以下内容：

if (LOG.isDebugEnabled()) {
LOG.debug(
"allocate: post-update" + " applicationAttemptId=" + appAttemptId
+ " #ask=" + ask.size() + " reservation= " + application
.getCurrentReservation());
LOG.debug("Preempting " + preemptionContainerIds.size()
+ " container(s)");
}

在 https://github.com/apache/hadoop/.../scheduler/fair/FairScheduler.java#937