启用kerberos后，historyserver无法读取日志 [英] historyserver not able to read log after enabling kerberos
问题描述
我在群集上启用了Kerberos，它工作正常.但是由于某些问题，被映射的用户无法通过JobHistory服务器读取和显示日志.我检查了作业历史记录服务器的日志，并给出了如下访问错误:
I enable the Kerberos on the cluster and it is working fine. But due to some issue mapred user is not able to read and display log over JobHistory server. I check the logs of job history server and it giving access error as:
org.apache.hadoop.security.AccessControlException: Permission denied:user=mapred, access=READ_EXECUTE, inode="/user/history/done_intermediate/prakul":prakul:hadoop:drwxrwx---
我们可以看到该目录有权访问hadoop组，而mapred在hadoop组中，即使这样它也无法读取日志.由于/tmp/logs/文件夹出现类似错误，因此资源管理器UI上没有显示日志.
as we can see the directory have access to hadoop group and mapred is in hadoop group, even then it is not able to read the logs. Similar error it is giving for /tmp/logs/ folder due to which no log was displayed on resource manager UI.
我在所有计算机上验证hadoop组包含所有计算机上的映射用户:
I verify over all machine that hadoop group contains mapred user on all machine:
[cloudera]# id mapred uid=491(mapred) gid=489(mapred) groups=489(mapred),496(hadoop)
我还初始化被映射的用户，并尝试手动访问这些目录，但是即使具有770权限的文件夹也无法访问被映射的用户:
I also kinit the mapred user and try to access manually to these directory, but mapred not able to access even when folder having 770 permission:
[root@mn0 cloudera]# hdfs dfs -ls /tmp/logs/prakul
ls: Permission denied: user=mapred, access=READ_EXECUTE, inode="/tmp/logs/prakul":prakul:hadoop:drwxrwx---
[root@mn0 cloudera]# hdfs dfs -ls /tmp/logs/
Found 8 items
drwxrwx--- - xyz hadoop 0 2016-06-14 19:19 /tmp/logs/xyz
drwxrwx--- - abc hadoop 0 2016-06-13 06:06 /tmp/logs/abc
drwxrwx--- - prakul hadoop 0 2016-06-10 04:47 /tmp/logs/prakul
[root@mn0 cloudera]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: mapred/mn0.eastus.cloudapp.azure.com@AD.COM
Valid starting Expires Service principal
06/27/16 01:07:32 06/27/16 11:07:32 krbtgt/AD.COM@AD.COM
renew until 07/04/16 01:07:32
如果我授予该目录777的权限，则mapred能够通过UI和CLI读取和显示日志.
If i give 777 permission to the directory then mapred is able to read and show log over UI as well as CLI.
任何人都可以知道这是Cloudera错误还是由于配置问题而导致，即使在组级别具有完全权限，被映射者也无法访问日志?
Can any one know whether it is some Cloudera bug or there is some configuration issue due to which mapred not able to access the log even having full permission at group level?
我正在使用启用了Kerberos的Cloudera 5.7.
I am using Cloudera 5.7 with Kerberos enabled.
预先感谢
推荐答案
我遵循以下cloudera/tmp/logs权限
I have followed below permissions to cloudera /tmp/logs
/tmp/logs --————----
/tmp/logs —————----
-
假定user1是有效的本地OS用户.以下文件夹结构包含适当的JobHistory功能:
drwxrwxrwt-hdfs supergroup 0 2014-09-15 17:01/tmp
drwxrwxrwt-mapred hadoop 0 2014-09-18 12:02/tmp/logs
drwxrwx----user1 hadoop 0 2014-09-18 12:03/tmp/logs/user1
drwxrwx----user1 hadoop 0 2014-09-18 12:03/tmp/logs/user1/logs
Assume user1 is a valid local OS user. The following folder structure comprises of a proper JobHistory functionality:
drwxrwxrwt - hdfs supergroup 0 2014-09-15 17:01 /tmp
drwxrwxrwt - mapred hadoop 0 2014-09-18 12:02 /tmp/logs
drwxrwx--- - user1 hadoop 0 2014-09-18 12:03 /tmp/logs/user1
drwxrwx--- - user1 hadoop 0 2014-09-18 12:03 /tmp/logs/user1/logs
以下是权限错误的示例条目:
drwxrwx---hive supergroup 0 2014-09-18 12:00/tmp/logs/user1/logs/
Here is an example entry for incorrect permissions:
drwxrwx--- - hive supergroup 0 2014-09-18 12:00 /tmp/logs/user1/logs/
递归调整/tmp/logs/文件夹，以反映与上述类似的所有权和权限:
Adjust the /tmp/logs/ folders recursively to reflect the ownership and permissions similar to the above:
用于更新客户在HDFS中的权限的示例命令:
Example commands to update the customer's permissions in HDFS:
sudo -u hdfs hadoop fs -chown mapred:hadoop/tmp/logs
须藤-u hdfs hadoop fs -chown -R:hadoop/tmp/logs/*
sudo -u hdfs hadoop fs -chown mapred:hadoop /tmp/logs
sudo -u hdfs hadoop fs -chown -R :hadoop /tmp/logs/*
这篇关于启用kerberos后，historyserver无法读取日志的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
