VM cloudera-用户cloudera和权限? [英] VM cloudera - user cloudera and permissions?

问题描述

我下载并安装了VM Cloudera 4.4以与Hadoop一起玩.我已经在工作平台上建立了集群，因此我对hadoop的工作原理有所了解.因此，我认为我的问题来自对Linux及其用户和团体的误解.

I downloaded and installed the VM Cloudera 4.4 to play with Hadoop. I have already a cluster on a platform for my job, so I know a little how works hadoop. So I think my problem comes from my misunderstanding of linux and his users and group.

带有Hive:

我尝试用外壳创建一个配置单元表，并且该表有效.我在/user/hive/warehouse/test女巫中有一张桌子，它属于组cloudera的用户cloudera.

I try to create a hive table with the shell, and it works. I have a table in /user/hive/warehouse/test witch belongs to user cloudera of group cloudera.

我在hdfs中有一些数据文件(.txt):/user/cloudera(user:cloudera，组:hive)，我在hive表中加载的是:

I have some data files (.txt) in hdfs : /user/cloudera ( user:cloudera and group: hive) that I load in my hive table with :

LOAD DATA INPATH '/user/cloudera/*.txt' INTO TABLE test;

这是我获得的:

hive> LOAD DATA INPATH '/user/cloudera/jeuDeTest/*.txt' INTO TABLE test;
Loading data to table default.test
chgrp: changing ownership of '/user/hive/warehouse/test/_log24310.txt': User does not belong to hive
chgrp: changing ownership of '/user/hive/warehouse/test/_log24311.txt': User does not belong to hive
Table default.test stats: [num_partitions: 0, num_files: 2, num_rows: 0, total_size: 10161843, raw_data_size: 0]
OK
Time taken: 2.472 seconds

我从没有这种错误消息，但是文件已移动.如果我尝试SELECT *，则没有结果.

I never had this kind of error message but the files are moved. If I try a SELECT *, there is no result.

使用HBase:

我在使用HBase时也遇到了一些困难.我可以创建一个表，但是当我使用importTSV时:

I have also some difficulties with HBase. I can create a table but when I use importTSV :

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv 
-Dimporttsv.columns=HBASE_ROW_KEY,cf:nl,ch:nt,cf:ti,cf:ip,cf:cr,cf:am,cf:op,cf:mr,cf:ct 
'-Dimporttsv.separator=|' testhbase -Dimporttsv.skip.bad.lines=false  
/user/cloudera/jeuDeTest/*.txt

我遇到此错误:

ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) 
cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: 
hdfs://localhost.localdomain:8020/user/cloudera/jeuDeTest/_logGeneral_C_24310_SO.txt
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:     
hdfs://localhost.localdomain:8020/user/cloudera/jeuDeTest/_logGeneral_C_24310_SO.txt

---

我认为这个问题是由于权限引起的，但是我不知道该怎么做才有权执行请求，那是更好的方法呢? (在我正在工作的平台上，我已经扎根，并且我没有所有这些困难，但是我不知道它是如何工作的)

---

I think this problems are due to permissions but I don't know how to do have the right to execute request, what is the better way to do that. (On the platform I have at work, I am root, and I don't have all this difficulties, but I don't understand how it works)

谢谢你读我.

天使

我尝试将我的cloudera用户添加到组配置单元中.加载过程中没有错误，但是选择始终没有结果.

I try to add my cloudera user to the group hive. I don't have the error during the load but I have always no result on a select.

hive> LOAD DATA INPATH '/user/cloudera/jeuDeTest/*.txt' INTO TABLE test;                     
Loading data to table default.test
Table default.test stats: [num_partitions: 0, num_files: 10, num_rows: 0, total_size: 10161843,   raw_data_size: 0]
OK
Time taken: 0.486 seconds
hive> select * from test limit 20;
OK
Time taken: 0.303 seconds

推荐答案

我在权限方面存在相同的问题-> chgrp:更改了'/user/hive/warehouse/test/_log24310.txt'的所有权:用户不属于蜂巢.

I had same issue with permissions -> chgrp: changing ownership of '/user/hive/warehouse/test/_log24310.txt': User does not belong to hive.

1. 使用以下命令将名为cloudera的现有用户添加到名为hive的现有组中: usermod -a -G配置单元cloudera
2. 重新启动系统
3. 使用了加载命令"，然后从table_name中进行了选择*->未显示任何数据.
4. 从table_name中执行选择计数(*)，并开始执行MapReduce作业.
5. 从表中执行select *，现在结果已正确返回.
6. 使用impala-shell命令打开一个impala shell.
7. 从table_name中执行了select *，没有返回任何结果.
8. 执行的命令会使impala外壳中的元数据无效
9. 已执行的命令刷新table_name
10. 已执行的命令显示表
11. 执行的命令从table_name中选择*，现在结果将同时显示在impala-shell和hive shell中.

1. Added the existing user named cloudera to existing group named hive with command: usermod -a -G hive cloudera
2. Restarted the system
3. Used Load Command and after that did a select * from table_name -> No data was getting displayed.
4. Executed select count(*) from table_name and a MapReduce job got started.
5. Executed select * from table and now results was returned correctly.
6. Opened a impala shell using impala-shell command.
7. Executed a select * from table_name and no results was getting returned.
8. Executed command invalidate metadata in the impala-shell
9. Executed command refresh table_name
10. Executed command show tables
11. Executed command select * from table_name and now results are getting displayed both in the impala-shell and hive shell.

这篇关于VM cloudera-用户cloudera和权限?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！