Hadoop迷你集群模拟(MiniDFSCluster) [英] Hadoop Mini Cluster Mock (MiniDFSCluster)

问题描述

我需要您有关 hadoop-minicluster

我正在使用scala(带有sbt)，并且尝试模拟HDFS的调用.我播下hadoop-minicluster来部署一个小的集群并对其进行测试.

I'm working with scala (with sbt) and I try to Mock calls of HDFS. I sow hadoop-minicluster for deploying a little cluster and test on it.

但是，当我添加sbt依赖项时:

However, when I add the sbt dependency :

libraryDependencies + ="org.apache.hadoop"％"hadoop-minicluster"％"3.1.0"％测试

未添加源，我无法导入软件包 org.apache.hadoop.hdfs.MiniDFSCluster

The sources are not added and I can't import the package org.apache.hadoop.hdfs.MiniDFSCluster

你知道我怎么解决这个问题吗?

Do you know how I can solve the problem ?

感谢您的回答

推荐答案

令人惊讶的是，它不在hadoop-minicluster中. 尝试 libraryDependencies + ="org.apache.hadoop"％"hadoop-hdfs"％HADOOP_VERSION分类程序测试"

Surprisingly, it's not in hadoop-minicluster. Try libraryDependencies += "org.apache.hadoop" % "hadoop-hdfs" % HADOOP_VERSION classifier "tests"

您可能还必须排除某些组件，例如 "org.apache.hadoop"％"hadoop-hdfs"％HADOOP_VERSION分类器"tests"排除("javax.servlet"，"servlet-api")

You also may have to exclude some components, such as "org.apache.hadoop" % "hadoop-hdfs" % HADOOP_VERSION classifier "tests" exclude ("javax.servlet", "servlet-api")

这篇关于Hadoop迷你集群模拟(MiniDFSCluster)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！