在Windows中Hadoop构建失败：native.sln中缺少zconf.h？ [英] Hadoop build Failing in Windows: zconf.h is missing from native.sln?

问题描述

我试图在Windows 10家庭版的开发计算机上构建以下hadoop版本

hadoop-2.7.3-src

以下是有关本地开发环境的详细信息：

-Windows 10 Home Edition

-Intel Core i5-6200U CPU @ 2.30GHz

-RAM 16 GB

-64- bit操作系统，基于x64的处理器

微软Visual Studio社区2015版本14.0.25431.01更新3

- 还将MSBUILD位置添加到Windows系统环境变量路径中：C：\程序文件（x86）\ MSBuild \ 14.0 \ Bin \ amd64

- 。 NET Framework 4.6.01586

-cmake version 3.7.2

-CYGWIN_NT-10.0 LTPBCV82DUG 2.7.0（ 0.306 / 5/3）2017-02-12 13:18 x86_64 Cygwin

-java版本1.8.0_121

-Java（TM）SE运行环境（build 1.8.0_121-b13）

-Java HotSpot（TM）64-Bit Server VM（构建25.121-b13，混合模式）

-Apache Maven 3.3.9（bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11：41：47-05：00）

-Google协议缓冲区protoc --version libprotoc 2.5.0

另外，我创建了一个名为Platform的系统环境变量并将其设置为x64

我打开了Visual Studio 2015的开发人员命令提示符（VS2015 ）

c：\hadoop\hadoop-2.7.3-src> mvn包-Pdist，native-win -DskipTests -Dtar -X

不幸的是，我收到以下错误：

  [C：\hadoop \hadoop-2.7.3-src\hadoop -common-project\hadoop-common\src\main\\\
ative\\\
ative.vcxproj] 
 ZlibDecompressor.c 
c：\ hadoop\hadoop-2.7.3-src\hadoop-共project\hadoop-common\src\main\\\
ative\src\org\apache\hadoop\io\compress\\ \\ zlib \org_apache_hadoop_io_compress_zlib.h（36）：致命错误C1083：无法打开包含文件：'zlib.h'：没有这样的文件或目录[C：\hadoop \ haddoop-2.7.3-src\hadoop -common-project\hadoop-common\src\main\\\
ative\\\
ative.vcxproj] 
完成构建项目C：\ haddoop \\ \\ hadoop-2.7.3-src\hadoop -common-project\hadoop-common\src\main\\\
ative\\\
ative.vcxproj（默认目标） - 失败。 
 Done Building ProjectC：\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\\\
ative\\\
ative.sln（默认目标） - 失败。 
 
构建失败。 
 
C：\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\\\
ative\\\
ative.sln （默认目标）（1） - > 
C：\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\\\
ative\\\
ative.vcxproj（默认目标） （2） - > 
（ClCompile target） - > 
C：\hadoop\hadoop-2.7.3-src\hadoop-共project\hadoop-common\src\main\\\
ative\src\org\apache\hadoop \io\compress\zlib\org_apache_hadoop_io_compress_zlib.h（36）：致命错误C1083：无法打开包含文件：'zlib.h'：没有这样的文件或目录[C：\hadoop\hadoop-2.7。 3-src\hadoop-common-project\hadoop-common\src\main\\\
ative\\\
ative.vcxproj] 
c：\hadoop\hadoop-2.7.3-src\hadoop -common-project \hadoop-common\src\main\\\
ative\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h（36）：致命错误C1083：无法打开包含文件：'zlib.h'：没有这样的文件或目录[C：\ haddoop \ haddoop-2.7.3-src\hadoop-common-project\hadoop-common\src\ main''native''native.vcxproj] 
 
 0警告
 2错误
 
已用时间00 ：00：02.49 
  

上述错误与zlib工具有关。

经过在线研究，有人说Visual Studio解决方案文件需要在Visual Studio中成功构建：

.... \hadoop-2.7.3-src\hadoop -common-project\hadoop-common\src\main\\\
ative\\\
ative.sln

在Administrator模式下使用Visual Studio 2015，我打开native.sln文件，并立即看到一个错误：

接下来，我打开了Dos命令提示符，并检查了我的Windows操作系统的确切版本：

ver

Microsoft Windows [版本10.0.14393] p>

另外，我打开了libwinutils项目的属性窗口，并确保在以下快照中标记的属性具有适当的值：

另外，我对winutils项目的属性采取了相同的步骤：

（对不起，stackoverflow不允许我放置其他图片快照，但是所有你基本上必须做的就是确保winutils项目的属性设置正确）

我下载了zlib 1.2.11源代码。使用VS2015开发者命令提示符（Visual Studio 2015）我使用cmake从zlib 1.2.11源代码构建zlib

  c： \ zlib \ zlib-1.2.11> cmake -GVisual Studio 14 2015-A x64 c：\ zlib \ zlib-1.2.11 \ 
  -  C编译器标识为MSVC 19.0 .24215.1 
  - 检查工作的C编译器：C：/ Program Files（x86）/ Microsoft Visual Studio 14.0 / VC / bin / x86_amd64 / cl.exe 
  - 检查工作的C编译器：C ：/ Program Files（x86）/ Microsoft Visual Studio 14.0 / VC / bin / x86_amd64 / cl.exe  -  works 
  - 检测C编译器ABI信息
  - 检测C编译器ABI信息 -  done 
  - 寻找sys / types.h 
  - 寻找sys / types.h  - 找到
  - 寻找stdint.h 
  - 寻找stdint.h  - 找到
  - 寻找stddef.h 
  - 寻找stddef.h  - 找到
  - 检查大小off64_t 
  - 检查大小off64_t  - 失败
  - 寻找fseeko 
  - 寻找fseeko  - 找不到
  - 寻找佛r unistd.h 
  - 寻找unistd.h  - 未找到
  - 配置完成
  - 生成完成
  - 生成文件已写入：C：/ zlib / zlib-1.2.11 
  

最后，用cmake运行build

  c：\ zlib \ zlib-1.2.11> cmake --build。 
  

在Windows系统变量中，我定义了以下变量：

ZLIB_HOME设置为C：\ zlib \ zlib-1.2.11

I'm trying to build the following hadoop version on development computer with Windows 10 Home Edition

hadoop-2.7.3-src

Here are the details about my local development environment:

-Windows 10 Home Edition

-Intel Core i5-6200U CPU @2.30GHz

-RAM 16 GB

-64-bit Operating System, x64-based processor

-Microsoft Visual Studio Community 2015 Version 14.0.25431.01 Update 3

-Also added MSBUILD location as C:\Program Files (x86)\MSBuild\14.0\Bin\amd64 to Windows System Environment Variable Path

-.NET Framework 4.6.01586

-cmake version 3.7.2

-CYGWIN_NT-10.0 LTPBCV82DUG 2.7.0(0.306/5/3) 2017-02-12 13:18 x86_64 Cygwin

-java version "1.8.0_121"

-Java(TM) SE Runtime Environment (build 1.8.0_121-b13)

-Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

-Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00)

-Google Protocol Buffers protoc --version libprotoc 2.5.0

Also, I've created and system environment variable called Platform and set it to x64

I opened up Developer Command Prompt for Visual Studio 2015 (VS2015)

c:\hadoop\hadoop-2.7.3-src> mvn package -Pdist,native-win -DskipTests -Dtar -X

Unfortunately, I'm getting the following error:

 [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
  ZlibDecompressor.c
c:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
Done Building Project "C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (default targets) -- FAILED.
Done Building Project "C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default targets) -- FAILED.

Build FAILED.

"C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default target) (1) ->
"C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (default target) (2) ->
(ClCompile target) ->
  c:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
  c:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]

    0 Warning(s)
    2 Error(s)

Time Elapsed 00:00:02.49

The aforementioned error has to do with zlib tool.

After researching online, someone said that the following Visual Studio solution file needs to be built successfully in Visual Studio:

....\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.sln

Using Visual Studio 2015 in Administrator mode, I opened up the native.sln file, and immediately saw an error:

enter image description here

Could someone please tell me what steps I have to take to resolve said error?

解决方案

So there were quite a few steps I had to take in order to resolve the problems.

Within the ....\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\winutils directory, I opened up the following solution in Visual Studio 2015:

winutils.sln

Within .....\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.c , I commented out the following line of code, and made a modified copy of it as shown below:

//const WCHAR* wsceConfigRelativePath = WIDEN_STRING(STRINGIFY(WSCE_CONFIG_DIR)) L"\\" WIDEN_STRING(STRINGIFY(WSCE_CONFIG_FILE));

const WCHAR* wsceConfigRelativePath = WIDEN_STRING("../etc/hadoop") L"\\" WIDEN_STRING("wsce-site.xml");

Also, In the winutils solution's property window, I had to set the platform value to x64 as the screenshot below shows:

Next, I opened Dos command prompt, and checked the exact version of my Windows OS:

ver

Microsoft Windows [Version 10.0.14393]

Also, I opened up the property window of the libwinutils project, and ensured that properties that are marked in the following snapshot had the proper values:

Also, I took the same steps for the properties of the winutils project:

(Sorry, stackoverflow would not allow me to place another picture snapshot, but all you basically have to do is make sure the the winutils project's property are set properly )

I downloaded zlib version 1.2.11 source code. Using Developer Command Prompt for VS2015 ( Visual Studio 2015 ) I built zlib from zlib version 1.2.11 source code using cmake

c:\zlib\zlib-1.2.11>cmake  -G "Visual Studio 14 2015" -A x64 c:\zlib\zlib-1.2.11\
-- The C compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - failed
-- Looking for fseeko
-- Looking for fseeko - not found
-- Looking for unistd.h
-- Looking for unistd.h - not found
-- Configuring done
-- Generating done
-- Build files have been written to: C:/zlib/zlib-1.2.11

Finally, run the build with cmake

c:\zlib\zlib-1.2.11>cmake --build .

In Windows System Variables I have the following variable defined:

ZLIB_HOME is set to C:\zlib\zlib-1.2.11

这篇关于在Windows中Hadoop构建失败：native.sln中缺少zconf.h？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！