无法停止Kerberos调试日志记录 [英] Unable to stop Kerberos debug logging

问题描述

我正在使用启用Kerberos的Spark集群来运行我们的Spark应用程序. Kerberos已由组织的其他成员预先设置，我不知道它是如何工作的.最初，我们使用Kerberos调试日志来了解无法从用户获取密码"异常，该异常是由于jre security的cacerts文件夹中缺少JCE证书而引发的.但是，我们不再需要日志，因此使用了-Dsun.security.krb5.debug = false参数来禁用日志记录.但是，这没有任何效果.还有其他参数可以解决问题吗?请帮助我.

I'm using a Kerberos enabled Spark cluster for running our Spark applications. The Kerberos has been setup previously by other members of the organization, and I have no idea how it works. In the initial days, we had used the Kerberos debug logs to understand the exception "Unable to obtain password from user" which was being raised due to absence of a JCE certificate in the cacerts folder of jre security. However, we no longer require the logs and thus, used the -Dsun.security.krb5.debug=false parameter to disable the logging. However, this did not have any effect. Is there any other parameter that could do the trick? Please help me.

推荐答案

GitBook摘录 "Hadoop和Kerberos:门外的疯狂" ，作者:史蒂夫·拉夫兰(Steve Loughran)， 低级秘密

Excerpt from the GitBook "Hadoop and Kerberos: The Madness Beyond the Gate" by Steve Loughran, chapter Low-Level Secrets

JVM库日志记录

您可以在
-Dsun.security.krb5.debug=true

这不是通过Log4J或java.util日志记录出来的；它来了 在控制台上.这有点不方便-但请记住 他们正在系统的非常低级的部分进行日志记录.它确实 至少日志.如果您发现自己处于这个水平，那您就处于 麻烦.记住这一点.

This doesn't come out via Log4J, or java.util logging; it just comes out on the console. Which is somewhat inconvenient —but bear in mind they are logging at a very low level part of the system. And it does at least log. If you find yourself down at this level you are in trouble. Bear that in mind.

如果要调试SPNEGO中发生的情况，请使用其他系统 属性可让您启用此功能:
-Dsun.security.spnego.debug=true

If you want to debug what is happening in SPNEGO, another system property lets you enable this:
-Dsun.security.spnego.debug=true

您可以在HADOOP_OPTS环境变量
中要求这两个条件 export HADOOP_OPTS="-Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true"

You can ask for both of these in the HADOOP_OPTS environment variable
export HADOOP_OPTS="-Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true"

将环境变量HADOOP_JAAS_DEBUG设置为true，然后UGI将设置 它创建的任何JAAS文件上的调试"标志.

Set the env variable HADOOP_JAAS_DEBUG to true and UGI will set the "debug" flag on any JAAS files it creates.

您可以在发出hadoop，hdfs或yarn之前在客户端上执行此操作 命令，并将其在YARN服务的环境脚本中设置为 打开那里.
export HADOOP_JAAS_DEBUG=true

You can do this on the client, before issuing a hadoop, hdfs or yarn command, and set it in the environment script of a YARN service to turn it on there.
export HADOOP_JAAS_DEBUG=true

在下一个Hadoop命令上，您会看到类似(.........)

On the next Hadoop command, you'll see a trace like (.........)

注意事项:以sun.security.开头的Java属性适用于Sun/Oracle Java运行时以及OpenJDK运行时及其变体.但不适用于IBM Java等.

Caveat: the Java properties starting with sun.security. apply to Sun/Oracle Java run-time, and also OpenJDK run-time and its variants. But not to IBM Java, etc.

---

摘录自 Java 8文档" rel ="nofollow noreferrer">安全性疑难解答

---

Excerpt from the Java 8 documentation under Troubleshooting Security

如果要监视安全性访问，可以设置 java.security.debug系统属性.
(.......)用逗号分隔多个选项.

If you want to monitor security access, you can set the java.security.debug System property.
(.......) Separate multiple options with a comma.

在专门对Kerberos进行故障排除时，我个人使用该组合:
-Djava.security.debug=gssloginconfig,configfile,configparser,logincontext

When troubleshooting Kerberos specifically, I personally use that combination:
-Djava.security.debug=gssloginconfig,configfile,configparser,logincontext

---

摘录自Oracle JDK 9发行说明 部分工具/启动器

---

Excerpt from the Oracle JDK 9 Release Notes section tools/launcher

JDK 9支持新的环境变量JDK_JAVA_OPTIONS 将选项添加到命令行上指定的选项之前.新的 环境变量相对于 旧版/不受支持的_JAVA_OPTIONS环境变量，包括 能够包含Java启动器选项(...)

JDK 9 supports a new environment variable JDK_JAVA_OPTIONS to prepend options to those specified on the command line. The new environment variable has several advantages over the legacy/unsupported _JAVA_OPTIONS environment variable including the ability to include java launcher options (...)

这两个env变量是一种非常肮脏的方法(并且非常难以检测)，用于在不将它们出现在命令行中的情况下注入Java系统属性.

These two env variables are a very dirty (and utterly difficult to detect) way to inject Java system properties without them appearing on the command line.

---

这对您意味着什么?好吧，您必须搜索可能已设置的多个Java系统属性和环境变量

---

What does that mean for you? Well, you have to search for multiple Java system props and environment variables, which might be set

• 用于env变量:全局(参见/etc/profile.d/*.sh)，或帐户级别(参见~/.bashrc和朋友)，或者在Hadoop包含文件"内部，或者直接在运行Spark作业的Shell脚本内部
• 对于系统道具:在以后在shell脚本(...)或Java在启动时选择的任何env var或YARN配置文件中开发的任何shell-or-env变量中(使用(Yarn上的Spark)，或直接在Java命令行上

• for env variables: globally (cf. /etc/profile.d/*.sh), or at account level (cf. ~/.bashrc and friends), or inside Hadoop "include files", or directly inside a shell script that runs your Spark job
• for system props: in any shell-or-env variable that is later developed in a shell script (...) or any env var picked up by Java on startup, or in YARN configuration files (when using Spark-on-YARN), or directly on a Java command-line

祝你好运.
我个人将运行一个虚拟的Spark作业，该作业只会转储所有env变量和Java系统道具；然后检查转储以查找要搜索的内容；然后在Linux文件系统(根据需要重复)上运行蛮力find ... -exec grep ....

Good luck.
I personally would run a dummy Spark job that just dumps all env variables and Java system props; then inspect the dump to detect what to search for; then run a brute-force find ... -exec grep ... on the Linux filesystem (repeat as needed).

这篇关于无法停止Kerberos调试日志记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！