OpenSSL FIPS_mode_set在Python加密库中不起作用 [英] OpenSSL FIPS_mode_set not working in Python cryptography library
问题描述
根据Python密码库的文档[1]，可以使用静态链接的OpenSSL构建自定义密码轮.我尝试通过使用FIPS对象模块构建的OpenSSL安装来执行此操作，并且能够成功构建转盘，但发现它不具有FIPS功能(无法设置FIPS_mode_set = 1).
我创建了一个可以重现相同结果的Dockerfile.末尾的Python代码应该显示为"1"和"OpenSSL 1.0.2t-fips 2019年9月10日"，而应显示为"0"和"OpenSSL 1.0.2t 10 Sep 2019"(无 -fips 名称).
让我感到困惑的是，当我调用构建的 openssl版本 CLI时，它会正确显示带有 -fips 后缀的版本.因为，因此，我猜我在构建密码学时某个地方出了错.
在这里感谢任何帮助！
FROM centos#安装构建依赖项运行yum groupinstall -y开发工具"&&\百胜安装-y python-devel libffi-devel#安装Python依赖项运行curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py&&\python get-pip.py&&\pip install virtualenv setuptools轮pip#构建Fips对象模块运行curl -O https://www.openssl.org/source/openssl-fips-2.0.16.tar.gz&&\tar xvf openssl-fips-2.0.16.tar.gz&&\cd openssl-fips-2.0.16&&\./config&&\制作&&\进行安装#构建OpenSSL运行curl -O https://www.openssl.org/source/openssl-1.0.2t.tar.gz&&\tar xvf openssl-1.0.2t.tar.gz&&\cd/openssl-1.0.2t&&\./config fips不共享-fPIC --prefix =/openssl-1.0.2t/openssl&&\使依赖\制作&&\使install_sw#建立加密RUN CFLAGS =-I/openssl-1.0.2t/openssl/include" LDFLAGS =-L/openssl-1.0.2t/openssl/lib" pip wheel --no-cache --no-binary:all:加密和加密;&\点安装加密* .whl#测试是否启用了fips运行python -c"\从cryptography.hazmat.backends.openssl.backend导入后端; \打印后端._lib.FIPS_mode_set(1); \打印''.join([backend._lib.OPENSSL_VERSION_TEXT [i] for i in range(30)]) [1]
我记得前一段时间也有类似情况:除了2个标准模块之外，还使用了 M2Crypto .在这种情况下(我们完全交付了 Python )，这2种特定的(支持 FIPS ) OpenSSL (动态)<还提供了em> lib ，所有与它们链接的 Python 模块.它适用于各种(桌面)环境(其中有许多奇异"环境):
- CPU 架构( LE / BE ): x86 ， AMD64 ， IA64 ， SPARC ， PPC ， zSeries
- OS es(具有多个版本): Win ， Lnx ( RH ， CentOS ， OEL ， SuSE ， Xen ， Ubuntu )， Solaris ， AIX ， HP-UX (并且作为个人锻炼，我添加了 OSX )
[OpenSSL]:UserGuide-2.0.pdf-OpenSSL用户指南FIPS对象模块v2.0 (引用自 [OpenSSL]:FIPS-140 以防 URL 发生更改)包含所有所需的详细信息.
在进一步介绍之前，我将在整个帖子中使用以下术语:
- FOM - FIPS 对象模块( fipscanister.o )
- FOME - FOM的可执行文件( ELF ( Win 上的 PE ))em>已链接.请记住，它既可以是可执行文件本身，也可以是 .so ( .dll ).另外，如果它包含在静态库( .a )中，则不会链接(只是存档).附带说明一下，当共享构建 OpenSSL 时， FOME 是 libcrypto.so.* ，而静态构建时(如本例)，它是与 libcrypto.a 链接的可执行文件(例如 openssl 可执行文件)
FOM 位于 OpenSSL 之上(可能还有其他类似的密码提供程序，例如 LibreSSL ， WolfSSL ，)，并通过限制某些本来可以使用的功能来增强安全性(根据 NIST 标准).这样的功能示例之一是 md5 哈希的用法，该哈希被认为是弱函数(我很确定 sha1 也会遵循该规则)下一个版本即将发布.
这是(运行时)发生的事情(非常简化)版本:
- FIPS 模式:
- 检查是否满足其他约束:
- 是:继续(使用默认功能)
- 否:错误返回
- 检查是否满足其他约束:
- FIPS 模式关闭:
- 恢复到默认功能
#1.1. 的一部分是自检.其中包括:
- 计算 FOME 签名
- 将其与一个值进行比较(该值也存储在 FOME 中)
这是为了确保(或大幅度减少机会)没有人篡改(手动修改，拆卸等) FOME .为了更好地了解签名机制，让我们深入了解 FOME 构建过程:
- 所有 FOME 源+ FOM 被编译(到目标文件中)
- 它们链接在一起(成为 FOME ).这是正常构建结束的时间
- 正在计算 FOME 签名
- 来自#1的项目. + fips_premain.o 被链接到一个(实际的，不是 .dll )可执行文件(FPD ) 针对 FOME (#1.)调用
- FPD .它读取 FOME 的 .rodata 部分，并计算其 sha1 哈希.请注意，它会忽略位于特定地址的 41 个字节区域(打孔)
重复 - #3.1.，但是这次重新编译了 fips_premain.o ，其中还包括了上一步中的哈希值.现在可以清楚地看到上一步中的打孔了，这是签名所在的位置:( sha 哈希的长度( 40 )+ nul ).这是最终的 FOME
注意:在 Win 上，事情发生的地方略有不同.
我设法重现了这个问题.我将从测试脚本开始.
code00.py :
<代码>#！/usr/bin/env python3导入系统从cryptography.hazmat.backends.openssl.backend导入后端导入cffidef main():ffi = cffi.FFI()lib =后端._libfmt ="OpenSSL版本:{0:s} \ nFIPS_mode():{1:d} \ nFIPS_mode_set(1):{2:d} \ nFIPS_mode():{3:d}"打印(fmt.format(ffi.string(lib.OPENSSL_VERSION_TEXT).decode()，lib.FIPS_mode()，lib.FIPS_mode_set(1)，lib.FIPS_mode()))错误= lib.ERR_get_error()如果错误:err_fmt =错误:[{{0:d}]:[{1:s}]:[{2:s}]:[{3:s}]"打印(err_fmt.format(呃，ffi.string(lib.ERR_lib_error_string(err)).decode()，ffi.string(lib.ERR_func_error_string(err)).decode()，ffi.string(lib.ERR_reason_error_string(err)).decode()))别的:打印(成功！！！")如果__name__ =="__main__":print("Python {0:s} {1:d}在{2:s} \ n上的位" .format(" .join(item.strip()表示sys.version.split("; \ n))，如果sys.maxsize> 0x100000000，则为64，否则为sys.platform)32)主要的()打印("\ nDone".) 2.设置
我已经构建了 FOM 和一个支持 OpenSSL 的 FIPS (类似于您的，但是我自定义了它们的路径).在构建 FOM 和 OpenSSL 时都使用了 $ {FIPSDIR} 变量.
<代码> [cfati @ cfati-ubtu16x64-0:〜/Work/Dev/StackOverflow/q058228435]>〜/sopr.sh***设置较短的提示以使其更好地适合在StackOverflow(或其他)页面中粘贴时***[064bit-prompt]>优名Linux cfati-ubtu16x64-0 4.15.0-65-generic#74〜16.04.1-Ubuntu SMP Wed Sep 18 09:51:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux[064bit-prompt]>猫/etc/lsb-release |grep DESCRDISTRIB_DESCRIPTION ="Ubuntu 16.04.6 LTS"；[064bit-prompt]>gcc --version |grep gccgcc(Ubuntu 5.4.0-6ubuntu1〜16.04.11)5.4.0 20160609[064bit-prompt]>[064bit-prompt]>回声$ {FIPSDIR}/home/cfati/Work/Dev/Tools/zzz_Build/OpenSSL/int/openssl-fips-2.0.16[064bit-prompt]>树$ {FIPSDIR}/home/cfati/Work/Dev/Tools/zzz_Build/OpenSSL/int/openssl-fips-2.0.16├──箱│├──fipsld│└──fips_standalone_sha1├──包括│└──openssl│├──aes.h│├──bn.h│├──buffer.h│├──cmac.h│├──crypto.h│├──des.h│├──des_old.h│├──dh.h│├──dsa.h│├──ebcdic.h│├──ecdh.h│├──ecdsa.h│├──ec.h│├──e_os2.h│├──evp.h│├──fips.h│├──fips_rand.h│├──fipssyms.h│├──hmac.h│├──modes.h│├──opensslconf.h│├──opensslv.h│├──ossl_typ.h│├──rsa.h│├──sha.h│└──symhacks.hlib──lib├──fipscanister.o├──fipscanister.o.sha1├──fips_premain.c└──fips_premain.c.sha14个目录，32个文件[064bit-prompt]>[064bit-prompt]>回声$ {OPENSSL_DIR}/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static[064bit-prompt]>树$ {OPENSSL_DIR}/bin $ {OPENSSL_DIR}/lib/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static/bin├──c_rehash└──openssl/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static/lib├──发动机├──libcrypto.a├──libssl.a└──pkgconfig├──libcrypto.pc├──libssl.pc└──openssl.pc2个目录，7个文件[064bit-prompt]>[064bit-prompt]>ldd $ {OPENSSL_DIR}/bin/openssllinux-vdso.so.1 =>(0x00007ffeec045000)libdl.so.2 =>/lib/x86_64-linux-gnu/libdl.so.2(0x00007f12c19c2000)libc.so.6 =>/lib/x86_64-linux-gnu/libc.so.6(0x00007f12c15f8000)/lib64/ld-linux-x86-64.so.2(0x00007f12c1bc6000)[064bit-prompt]>[064bit-prompt]>$ {OPENSSL_DIR}/bin/openssl版本OpenSSL 1.0.2t-fips 2019年9月10日[064bit-prompt]>$ {OPENSSL_DIR}/bin/openssl sha1 ./code00.pySHA1(./code00.py)= ff122260b025103dbc03316e3d3e26cd683e7a12[064bit-prompt]>$ {OPENSSL_DIR}/bin/openssl md5 ./code00.pyMD5(./code00.py)= eac85e46734260c1bfcceb89d6a3bd32[064bit-prompt]>OPENSSL_FIPS = 1 $ {OPENSSL_DIR}/bin/openssl sha1 ./code00.pySHA1(./code00.py)= ff122260b025103dbc03316e3d3e26cd683e7a12[064bit-prompt]>OPENSSL_FIPS = 1 $ {OPENSSL_DIR}/bin/openssl md5 ./code00.py设置摘要MD5时出错140584610875032:错误:060A80A3:数字信封例程:FIPS_DIGESTINIT:禁用了fips:fips_md.c:180: 3.密码学模块
[064bit-prompt]>lscode00.py cryptography-2.7.tar.gz[064bit-prompt]>mkdir构建[064bit-prompt]>光盘制作[064bit-prompt]>[064bit-prompt]>CFLAGS =-I $ {OPENSSL_DIR}/include -DOPENSSL_FIPS = 1"LDFLAGS =-L $ {OPENSSL_DIR}/lib"python3 -m pip wheel --no-cache --no-binary:all:../cryptography-2.7.tar.gz处理/home/cfati/Work/Dev/StackOverflow/q058228435/cryptography-2.7.tar.gz安装构建依赖项...完成完成构建车轮的要求...完成准备砂轮元数据...完成收集asn1crypto> = 0.21.0(来自密码术== 2.7)正在下载https://files.pythonhosted.org/packages/d1/e2/c518f2bc5805668803ebf0659628b0e9d77ca981308c7e9e5564b30b8337/asn1crypto-1.0.1.tar.gz(115kB)|█████████████████████████████████|122kB 801kB/秒收集cffi！= 1.11.3，> = 1.8(来自密码术== 2.7)正在下载https://files.pythonhosted.org/packages/93/1a/ab8c62b5838722f29f3daffcc8d4bd61844aa9b5f437341cc890ceee483b/cffi-1.12.3.tar.gz(456kB)|████████████████████████████████||460kB 1.8MB/秒收集六个> = 1.4.1(来自密码学== 2.7)正在下载https://files.pythonhosted.org/packages/dd/bf/4138e7bfb757de47d1f4b6994648ec67a51efe58fa907c1e11e350cddfca/six-1.12.0.tar.gz收集pycparser(来自cffi！= 1.11.3，> = 1.8-> cryptography == 2.7)下载https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz(158kB)|████████████████████████████████||163kB 4.5MB/秒收集软件包的构建轮:加密，asn1crypto，cffi，六个，pycparser密码学的构建轮(PEP 517)...已完成存储在以下目录中:/home/cfati/Work/Dev/StackOverflow/q058228435/buildasn1crypto的构建轮(setup.py)...已完成存储在以下目录中:/home/cfati/Work/Dev/StackOverflow/q058228435/buildcffi的构建轮(setup.py)...已完成存储在以下目录中:/home/cfati/Work/Dev/StackOverflow/q058228435/build六个安装车轮(setup.py)...完成存储在以下目录中:/home/cfati/Work/Dev/StackOverflow/q058228435/buildpycparser的构建轮(setup.py)...已完成存储在以下目录中:/home/cfati/Work/Dev/StackOverflow/q058228435/build成功建立了密码学asn1crypto cffi六个pycparser警告:您正在使用pip版本19.1.1，但是可以使用版本19.2.3.您应该考虑通过"pip install --upgrade pip"命令进行升级.[064bit-prompt]>[064bit-prompt]>lsasn1crypto-1.0.1-py3-none-any.whl密码学-2.7-cp35-cp35m-linux_x86_64.whl六-1.12.0-py2.py3-none-any.whlcffi-1.12.3-cp35-cp35m-linux_x86_64.whl pycparser-2.19-py2.py3-none-any.whl[064bit-prompt]>[064bit-prompt]>为$(ls * .whl)中的f;解压缩$ {f}>/dev/null;完毕[064bit-prompt]>lsasn1crypto cffi-1.12.3-cp35-cp35m-linux_x86_64.whl密码学-2.7-cp35-cp35m-linux_x86_64.whl pycparser-2.19-py2.py3-none-any.whlasn1crypto-1.0.1.dist-info cffi-1.12.3.dist-info cryptography-2.7.dist-info六-1.12.0.dist-infoasn1crypto-1.0.1-py3-none-any.whl _cffi_backend.cpython-35m-x86_64-linux-gnu.so pycparser六-1.12.0-py2.py3-none-any.whlcffi密码学pycparser-2.19.dist-info six.py[064bit-prompt]>PYTHONPATH =.:$ {PYTHONPATH} python3 ../code00.pyPython 3.5.2(默认，2019年7月10日，11:58:48)[GCC 5.4.0 20160609] Linux上的64位OpenSSL版本:OpenSSL 1.0.2t-fips 2019年9月10日FIPS_mode():0FIPS_mode_set(1):0FIPS_mode():0错误:[755413103]:[FIPS例程]:[FIPS_check_incore_fingerprint]:[指纹不匹配]完毕.
如图所示，我几乎在哪里.
4.更深的潜水
经过长时间(有些人可能会觉得很痛苦)的调试，试验……，我得出了一个结论.考虑到:
- 篡改 FOM 中的任何内容( $ {FIPSDIR} 的内容)，将没有任何人可以通过 FIPS 验证强>.坦率地说，这也不是，因为有特定的说明，在构建 FOM 时，只有 sys管理员才应将工件复制到安全的位置....，bla，bla，bla.对我来说这似乎有些偏执，但这是事实.值得一提的是，早在2013年，当我们1 st 与 FIPS 接触时(可能是为了防止任何可能的攻击(例如 MITM ))， FOM 来源 CD 从 USA 运送到 ROU :)))
- 默认的 Python 版本是静态构建的(再次:))，这意味着 $ {PYTHONCORE} ( Python 解释器)位于 python 可执行文件，而不是可链接到的 .so ( libpython * .so * )，并且 python可执行文件(如果是共享版本)
- 密码学的 _openssl 扩展模块( _openssl.abi * .so )需要来自 $ {PYTHONCORE}的符号(例如 PyLong_FromLong )，但这是 OK ，因为此刻它将被加载到( Python )进程中(从上述方法启动)可执行文件)，它将找到它们(这是 Nix 的常见做法)
- 构建步骤#3.2. :可执行文件( FPD -必须运行)找不到符号，因此失败
这是一个僵局(任何一个约束离开的房间，被其他约束关闭的房间)，因此不能简单地完成！！！ (至少在目前是这样).句点！ X(
5.替代
我建议将此作为一种优雅的替代方法(包括 OpenSSL .so s(与任何( .so )客户端一起使用)在 .whl 中将 rpath 设置为"there")，在链接到它们的 _openssl.abi3.so 旁边)，但是显然是唯一的方法(至少我已经找到了).
第一个 st 步骤是构建共享的 OpenSSL 版本( FOME 将为 libcrypto.so.* ).
[064bit-prompt]>lscode00.py cryptography-2.7.tar.gz[064bit-prompt]>导出OPENSSL_DIR =/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16[064bit-prompt]>ldd $ {OPENSSL_DIR}/bin/openssllinux-vdso.so.1 =>(0x00007ffe62faf000)libssl.so.1.0.0 =>/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16/lib/libssl.so.1.0.0(0x00007fe33c06f000)libcrypto.so.1.0.0 =>/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16/lib/libcrypto.so.1.0.0(0x00007fe33bb92000)libc.so.6 =>/lib/x86_64-linux-gnu/libc.so.6(0x00007fe33b7c8000)libdl.so.2 =>/lib/x86_64-linux-gnu/libdl.so.2(0x00007fe33b5c4000)/lib64/ld-linux-x86-64.so.2(0x00007fe33c2e3000)[064bit-prompt]>[064bit-prompt]>$ {OPENSSL_DIR}/bin/openssl版本OpenSSL 1.0.2t-fips 2019年9月10日[064bit-prompt]>$ {OPENSSL_DIR}/bin/openssl md5 ./code00.pyMD5(./code00.py)= eac85e46734260c1bfcceb89d6a3bd32[064bit-prompt]>OPENSSL_FIPS = 1 $ {OPENSSL_DIR}/bin/openssl md5 ./code00.py设置摘要MD5时出错139796140275352:错误:060A80A3:数字信封例程:FIPS_DIGESTINIT:禁用了fips:fips_md.c:180
在进行了另一次深度潜水(许多尝试失败)之后，我得以使其工作.但是，采取了很多措施:
- 手动干预
- 乱七八糟
- (相同)解决方法( gainarii )
恐怕如果我将所有内容都放到这里，它将大大超出 30K char 的限制(.whl .rel ="nofollow noreferrer"> [GitHub]:CristiFati/Prebuilt-Binaries-(主)Prebuilt-Binaries/Cryptography/v2.7 .到目前为止，它仅适用于 Python 3.5 ( 64bit )，因为它是 Ubuntu 16 上的默认版本.如果您使用另一个(较新的)版本，请让我知道，我会得到它(也许是我自己构建)，然后也为该版本构建 .whl (我将这样做)无论如何).
将原始的 .whl 替换为我自己构建的文件后:
[cfati @ cfati-ubtu16x64-0:〜/Work/Dev/StackOverflow/q058228435/build]>二总计2936drwxrwxr-x 2 cfati cfati 4096 10月9日21:40.drwxrwxr-x 4 cfati cfati 4096 10月9日21:28 ..-rw-rw-r-- 1 cfati cfati 108067 Oct 9 08:43 asn1crypto-1.0.1-py3-none-any.whl-rw-rw-r-- 1 cfati cfati 318045 10月9日08:43 cffi-1.12.3-cp35-cp35m-linux_x86_64.whl-rw-rw-r-- 1 cfati cfati 2438739 10月9日21:40 cryptography-2.7-cp35-cp35m-linux_x86_64.whl-rw-rw-r-- 1 cfati cfati 112066 Oct 9 08:43 pycparser-2.19-py2.py3-none-any.whl-rw-rw-r-- 1 cfati cfati 12099 Oct 9 08:43 six-1.12.0-py2.py3-none-any.whl[cfati @ cfati-ubtu16x64-0:〜/Work/Dev/StackOverflow/q058228435/build]>为$(ls * .whl)中的f;解压缩$ {f}>/dev/null;完毕[cfati @ cfati-ubtu16x64-0:〜/Work/Dev/StackOverflow/q058228435/build]>lsasn1crypto cffi-1.12.3-cp35-cp35m-linux_x86_64.whl密码学-2.7-cp35-cp35m-linux_x86_64.whl pycparser-2.19-py2.py3-none-any.whlasn1crypto-1.0.1.dist-info cffi-1.12.3.dist-info cryptography-2.7.dist-info六-1.12.0.dist-infoasn1crypto-1.0.1-py3-none-any.whl _cffi_backend.cpython-35m-x86_64-linux-gnu.so pycparser六-1.12.0-py2.py3-none-any.whlcffi密码学pycparser-2.19.dist-info six.py[cfati @ cfati-ubtu16x64-0:〜/Work/Dev/StackOverflow/q058228435/build]>PYTHONPATH =.:$ {PYTHONPATH} python3 ../code00.pyPython 3.5.2(默认，2019年7月10日，11:58:48)[GCC 5.4.0 20160609] Linux上的64位OpenSSL版本:OpenSSL 1.0.2t-fips 2019年9月10日FIPS_mode():0FIPS_mode_set(1):1FIPS_mode():1成功 ！！！完毕.
相关(或多或少)帖子:
- [SO]:无法构建在HP-UX上运行FIPS的OpenSSL功能
- [[SO]:如何为Python封装的libcrypto和libssl启用FIPS模式?(@CristiFati的答案)
- -fips designation).
The thing that boggles my mind is that when I invoke
openssl versionCLI that I built, it correctly shows the version with the-fipssuffix. Because, of this I'm guessing that I went wrong somewhere in building cryptography.Appreciate any help here!
FROM centos # Install build dependencies RUN yum groupinstall -y "Development Tools" && \ yum install -y python-devel libffi-devel # Install Python dependencies RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \ python get-pip.py && \ pip install virtualenv setuptools wheel pip # Build Fips object module RUN curl -O https://www.openssl.org/source/openssl-fips-2.0.16.tar.gz && \ tar xvf openssl-fips-2.0.16.tar.gz && \ cd openssl-fips-2.0.16 && \ ./config && \ make && \ make install # Build OpenSSL RUN curl -O https://www.openssl.org/source/openssl-1.0.2t.tar.gz && \ tar xvf openssl-1.0.2t.tar.gz && \ cd /openssl-1.0.2t && \ ./config fips no-shared -fPIC --prefix=/openssl-1.0.2t/openssl && \ make depend && \ make && \ make install_sw # Build cryptography RUN CFLAGS="-I/openssl-1.0.2t/openssl/include" LDFLAGS="-L/openssl-1.0.2t/openssl/lib" pip wheel --no-cache --no-binary :all: cryptography && \ pip install cryptography*.whl # Test if fips is enabled RUN python -c "\ from cryptography.hazmat.backends.openssl.backend import backend;\ print backend._lib.FIPS_mode_set(1);\ print ''.join([backend._lib.OPENSSL_VERSION_TEXT[i] for i in range(30)])"[1] https://cryptography.io/en/latest/installation/#static-wheels
EDIT: By adding
-DOPENSSL_FIPSto the cryptography build, I was able to make output ofOPENSSL_VERSION_TEXTbecomeOpenSSL 1.0.2t-fips 10 Sep 20but the output of FIPS_mode_set(1) is still 0.EDIT 2: Using
ERR_get_error()shows the following:>>> print backend._lib.FIPS_mode_set(1) 0 >>> print backend._lib.ERR_get_error() 755413103When I pop this into
openssl errstrI get:openssl errstr 755413103 error:755413103:lib(85):func(1043):reason(259)According to some Google searches, this indicates that fingerprint doesn't match (
FIPS_R_FINGERPRINT_DOES_NOT_MATCH). Not sure where to go from here though.解决方案1. General
First of all, I want to mention that although I understand the reasons, I don't fully agree with Cryptography's vision ([Cryptography]: Installation - Static Wheels). Shared libraries exist for decades and have proven their superiority. Not to mention that Python ships 2 (standard) modules (_ssl and _hashlib) that dynamically link to OpenSSL (whatever it's on the system). As a side note, on Win, the 2 Python modules also used to link statically to OpenSSL, but starting with v3.7, they no longer do. Back to Nix: 2 OpenSSL versions are loaded into the same (Python) process. It doesn't seem to harm, but it looks funny. And as things are today (191009), there are a bunch of .whls for v2.7 and v3.4, but none for a fairly decent (Python) environment:
I remember a similar situation a while ago: besides the 2 standard modules, M2Crypto was also used. In that situation (we shipped Python entirely), the 2 specific (FIPS capable) OpenSSL (dynamic) libs were shipped too, and all the Python modules linked to them. It worked on a variety of (desktop) environments (out of which many "exotic" ones):
- CPU architectures (LE / BE): x86, AMD64, IA64, SPARC, PPC, zSeries
- OSes (with multiple versions): Win, Lnx (RH, CentOS, OEL, SuSE, Xen, Ubuntu), Solaris, AIX, HP-UX (and as personal exercise, I added OSX)
[OpenSSL]: UserGuide-2.0.pdf - User Guide for the OpenSSL FIPS Object Module v2.0 (referenced from [OpenSSL]: FIPS-140 in case the URL changes) contains all the details needed.
Before going further, here are some terms that I'm going to use throughout the post:
- FOM - FIPS object module (fipscanister.o)
- FOME - The executable (ELF (PE on Win)) that FOM was linked in. Bear in mind, that it can be either an executable per se, either an .so (.dll). Also, if it's included in a static lib (.a), it's not linked (just archived). As a side note, when OpenSSL is built shared, FOME is libcrypto.so.*, while when built statically (like in this case), it's the executable that links with libcrypto.a (e.g. openssl executable)
FOM comes on top of OpenSSL (and probably other such cryptography providers, like LibreSSL, WolfSSL, ), and it's meant to strengthen security, (according to NIST standards), by restricting some features that otherwise would be available. One such feature example is the usage of md5 hash, which is considered weak (I'm pretty sure that sha1 will follow too, in the next version about to be released).
Here's a (very simplified) version of what happens (at runtime):- FIPS mode on:
- Check whether the additional constraints are met:
- Yes: Proceed (with default functionality)
- No: Return with error
- Check whether the additional constraints are met:
- FIPS mode off:
- Fall back to default functionality
Part of #1.1. is the selftest. That consists of:
- Computing the FOME signature
- Comparing it to a value (that was also stored in FOME)
This happens to make sure (or drastically reduce the chances) that no one tampered (manually modifying, disassembling, ...) with FOME. To have a better understanding of the signature mechanism, let's dive into FOME build process:
- All the FOME sources + FOM are compiled (into object files)
- They are linked together (into FOME). This is when a normal build ends
- The FOME signature is being computed
- Items from #1. + fips_premain.o are being linked into a (real, not .dll) executable (FPD)
- FPD is invoked against FOME (#1.). It reads FOME's .rodata section and computes its sha1 hash. Note that it ignores a 41 bytes zone (punches a hole) located at a specific address
- #3.1. is repeated, but this time fips_premain.o was recompiled to also include the hash from previous step. Now it becomes clear the punched hole from previous step, it's the place where the signature goes: (length of sha hash (40) + nul). This is the final FOME
Note: On Win, things happen just a bit differently.
I've managed to reproduce the problem. I'm going to start with the test script.
code00.py:
#!/usr/bin/env python3 import sys from cryptography.hazmat.backends.openssl.backend import backend import cffi def main(): ffi = cffi.FFI() lib = backend._lib fmt = "OpenSSL version: {0:s}\nFIPS_mode(): {1:d}\nFIPS_mode_set(1): {2:d}\nFIPS_mode(): {3:d}" print(fmt.format(ffi.string(lib.OPENSSL_VERSION_TEXT).decode(), lib.FIPS_mode(), lib.FIPS_mode_set(1), lib.FIPS_mode())) err = lib.ERR_get_error() if err: err_fmt = "error:[{0:d}]:[{1:s}]:[{2:s}]:[{3:s}]" print(err_fmt.format( err, ffi.string(lib.ERR_lib_error_string(err)).decode(), ffi.string(lib.ERR_func_error_string(err)).decode(), ffi.string(lib.ERR_reason_error_string(err)).decode() )) else: print("Success !!!") if __name__ == "__main__": print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform)) main() print("\nDone.")2. Setup
I've already built FOM and a FIPS capable OpenSSL (similar to yours, but I customized their paths). The ${FIPSDIR} variable was used when building both FOM and OpenSSL.
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435]> ~/sopr.sh *** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages *** [064bit-prompt]> uname -a Linux cfati-ubtu16x64-0 4.15.0-65-generic #74~16.04.1-Ubuntu SMP Wed Sep 18 09:51:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux [064bit-prompt]> cat /etc/lsb-release | grep DESCR DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS" [064bit-prompt]> gcc --version | grep gcc gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609 [064bit-prompt]> [064bit-prompt]> echo ${FIPSDIR} /home/cfati/Work/Dev/Tools/zzz_Build/OpenSSL/int/openssl-fips-2.0.16 [064bit-prompt]> tree ${FIPSDIR} /home/cfati/Work/Dev/Tools/zzz_Build/OpenSSL/int/openssl-fips-2.0.16 ├── bin │ ├── fipsld │ └── fips_standalone_sha1 ├── include │ └── openssl │ ├── aes.h │ ├── bn.h │ ├── buffer.h │ ├── cmac.h │ ├── crypto.h │ ├── des.h │ ├── des_old.h │ ├── dh.h │ ├── dsa.h │ ├── ebcdic.h │ ├── ecdh.h │ ├── ecdsa.h │ ├── ec.h │ ├── e_os2.h │ ├── evp.h │ ├── fips.h │ ├── fips_rand.h │ ├── fipssyms.h │ ├── hmac.h │ ├── modes.h │ ├── opensslconf.h │ ├── opensslv.h │ ├── ossl_typ.h │ ├── rsa.h │ ├── sha.h │ └── symhacks.h └── lib ├── fipscanister.o ├── fipscanister.o.sha1 ├── fips_premain.c └── fips_premain.c.sha1 4 directories, 32 files [064bit-prompt]> [064bit-prompt]> echo ${OPENSSL_DIR} /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static [064bit-prompt]> tree ${OPENSSL_DIR}/bin ${OPENSSL_DIR}/lib /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static/bin ├── c_rehash └── openssl /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static/lib ├── engines ├── libcrypto.a ├── libssl.a └── pkgconfig ├── libcrypto.pc ├── libssl.pc └── openssl.pc 2 directories, 7 files [064bit-prompt]> [064bit-prompt]> ldd ${OPENSSL_DIR}/bin/openssl linux-vdso.so.1 => (0x00007ffeec045000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f12c19c2000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f12c15f8000) /lib64/ld-linux-x86-64.so.2 (0x00007f12c1bc6000) [064bit-prompt]> [064bit-prompt]> ${OPENSSL_DIR}/bin/openssl version OpenSSL 1.0.2t-fips 10 Sep 2019 [064bit-prompt]> ${OPENSSL_DIR}/bin/openssl sha1 ./code00.py SHA1(./code00.py)= ff122260b025103dbc03316e3d3e26cd683e7a12 [064bit-prompt]> ${OPENSSL_DIR}/bin/openssl md5 ./code00.py MD5(./code00.py)= eac85e46734260c1bfcceb89d6a3bd32 [064bit-prompt]> OPENSSL_FIPS=1 ${OPENSSL_DIR}/bin/openssl sha1 ./code00.py SHA1(./code00.py)= ff122260b025103dbc03316e3d3e26cd683e7a12 [064bit-prompt]> OPENSSL_FIPS=1 ${OPENSSL_DIR}/bin/openssl md5 ./code00.py Error setting digest md5 140584610875032:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180:3. Cryptography module
[064bit-prompt]> ls code00.py cryptography-2.7.tar.gz [064bit-prompt]> mkdir build [064bit-prompt]> cd build [064bit-prompt]> [064bit-prompt]> CFLAGS="-I${OPENSSL_DIR}/include -DOPENSSL_FIPS=1" LDFLAGS="-L${OPENSSL_DIR}/lib" python3 -m pip wheel --no-cache --no-binary :all: ../cryptography-2.7.tar.gz Processing /home/cfati/Work/Dev/StackOverflow/q058228435/cryptography-2.7.tar.gz Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Collecting asn1crypto>=0.21.0 (from cryptography==2.7) Downloading https://files.pythonhosted.org/packages/d1/e2/c518f2bc5805668803ebf0659628b0e9d77ca981308c7e9e5564b30b8337/asn1crypto-1.0.1.tar.gz (115kB) |████████████████████████████████| 122kB 801kB/s Collecting cffi!=1.11.3,>=1.8 (from cryptography==2.7) Downloading https://files.pythonhosted.org/packages/93/1a/ab8c62b5838722f29f3daffcc8d4bd61844aa9b5f437341cc890ceee483b/cffi-1.12.3.tar.gz (456kB) |████████████████████████████████| 460kB 1.8MB/s Collecting six>=1.4.1 (from cryptography==2.7) Downloading https://files.pythonhosted.org/packages/dd/bf/4138e7bfb757de47d1f4b6994648ec67a51efe58fa907c1e11e350cddfca/six-1.12.0.tar.gz Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography==2.7) Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB) |████████████████████████████████| 163kB 4.5MB/s Building wheels for collected packages: cryptography, asn1crypto, cffi, six, pycparser Building wheel for cryptography (PEP 517) ... done Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build Building wheel for asn1crypto (setup.py) ... done Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build Building wheel for cffi (setup.py) ... done Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build Building wheel for six (setup.py) ... done Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build Building wheel for pycparser (setup.py) ... done Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build Successfully built cryptography asn1crypto cffi six pycparser WARNING: You are using pip version 19.1.1, however version 19.2.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. [064bit-prompt]> [064bit-prompt]> ls asn1crypto-1.0.1-py3-none-any.whl cryptography-2.7-cp35-cp35m-linux_x86_64.whl six-1.12.0-py2.py3-none-any.whl cffi-1.12.3-cp35-cp35m-linux_x86_64.whl pycparser-2.19-py2.py3-none-any.whl [064bit-prompt]> [064bit-prompt]> for f in $(ls *.whl); do unzip ${f} > /dev/null; done [064bit-prompt]> ls asn1crypto cffi-1.12.3-cp35-cp35m-linux_x86_64.whl cryptography-2.7-cp35-cp35m-linux_x86_64.whl pycparser-2.19-py2.py3-none-any.whl asn1crypto-1.0.1.dist-info cffi-1.12.3.dist-info cryptography-2.7.dist-info six-1.12.0.dist-info asn1crypto-1.0.1-py3-none-any.whl _cffi_backend.cpython-35m-x86_64-linux-gnu.so pycparser six-1.12.0-py2.py3-none-any.whl cffi cryptography pycparser-2.19.dist-info six.py [064bit-prompt]> PYTHONPATH=.:${PYTHONPATH} python3 ../code00.py Python 3.5.2 (default, Jul 10 2019, 11:58:48) [GCC 5.4.0 20160609] 64bit on linux OpenSSL version: OpenSSL 1.0.2t-fips 10 Sep 2019 FIPS_mode(): 0 FIPS_mode_set(1): 0 FIPS_mode(): 0 error:[755413103]:[FIPS routines]:[FIPS_check_incore_fingerprint]:[fingerprint does not match] Done.As seen, I'm pretty much where you are.
4. Deeper dive
After long (and some might consider painful) hours of debugging, trials, ..., I reached to a conclusion. Considering that:
- Tampering with anything from FOM (contents of ${FIPSDIR}), will no loner qualify as FIPS validated. To be frank, neither does this, as there are specific instructions that when building FOM, only a sys admin should copy the artifacts in a secure location ...., bla, bla, bla. This seems paranoid to me, but these are the facts. As a remark, back in 2013, when we 1st came in contact with FIPS (probably to prevent any possible attack (e.g. MITM)), the FOM sources CD was shipped from USA to ROU :)))
- Default Python version is statically (again :) ) built, meaning that ${PYTHONCORE} (the Python interpreter) resides in the python executable, rather than in a .so (libpython*.so*) that can be linked to (and that the python executable does in case of shared builds)
- Cryptography's _openssl extension module (_openssl.abi*.so) needs symbols from ${PYTHONCORE} (e.g. PyLong_FromLong), but that's OK since at the moment it will be loaded into the (Python) process (launched from the aforementioned executable), it will find them (this is a common practice on Nix)
- Build step #3.2.: the executable (FPD - which must run) doesn't find the symbols, so it fails
This is a deadlock (whatever room one constraint leaves, is closed by others), so IT SIMPLY CAN'T BE DONE !!! (at least, at this time). Period!!! X(
5. Alternative
I was going to suggest this as an elegant alternative (including the OpenSSL .sos (with any (.so) client having rpath set to "there") in the .whl, next to _openssl.abi3.so which links to them), but apparently this is the only way (that I've found, at least).
The 1st step, is to build a shared OpenSSL version (FOME will be libcrypto.so.*).
[064bit-prompt]> ls code00.py cryptography-2.7.tar.gz [064bit-prompt]> export OPENSSL_DIR=/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16 [064bit-prompt]> ldd ${OPENSSL_DIR}/bin/openssl linux-vdso.so.1 => (0x00007ffe62faf000) libssl.so.1.0.0 => /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16/lib/libssl.so.1.0.0 (0x00007fe33c06f000) libcrypto.so.1.0.0 => /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16/lib/libcrypto.so.1.0.0 (0x00007fe33bb92000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe33b7c8000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe33b5c4000) /lib64/ld-linux-x86-64.so.2 (0x00007fe33c2e3000) [064bit-prompt]> [064bit-prompt]> ${OPENSSL_DIR}/bin/openssl version OpenSSL 1.0.2t-fips 10 Sep 2019 [064bit-prompt]> ${OPENSSL_DIR}/bin/openssl md5 ./code00.py MD5(./code00.py)= eac85e46734260c1bfcceb89d6a3bd32 [064bit-prompt]> OPENSSL_FIPS=1 ${OPENSSL_DIR}/bin/openssl md5 ./code00.py Error setting digest md5 139796140275352:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180After another session of deep diving (lots of failed attempts), I was able to get it working. However, took a lot of actions:
- Manual interventions
- Hacks
- (Lame) workarounds (gainarii)
I'm afraid that if I'd put everything here it would well exceed the 30K chars limit ([SE.Meta]: Knowing Your Limits: What is the maximum length of a question title, post, image and links used?).
However, I published the .whl at [GitHub]: CristiFati/Prebuilt-Binaries - (master) Prebuilt-Binaries/Cryptography/v2.7. So far it's for Python 3.5 (64bit) only, as it's the default version that comes on Ubuntu 16. If you use another (newer) version, just let me know, and I'll get it (maybe build it myself), and build the .whl for that version too (I am going to do it anyway).
After replacing the original .whl with the one built by me:
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> ll total 2936 drwxrwxr-x 2 cfati cfati 4096 Oct 9 21:40 . drwxrwxr-x 4 cfati cfati 4096 Oct 9 21:28 .. -rw-rw-r-- 1 cfati cfati 108067 Oct 9 08:43 asn1crypto-1.0.1-py3-none-any.whl -rw-rw-r-- 1 cfati cfati 318045 Oct 9 08:43 cffi-1.12.3-cp35-cp35m-linux_x86_64.whl -rw-rw-r-- 1 cfati cfati 2438739 Oct 9 21:40 cryptography-2.7-cp35-cp35m-linux_x86_64.whl -rw-rw-r-- 1 cfati cfati 112066 Oct 9 08:43 pycparser-2.19-py2.py3-none-any.whl -rw-rw-r-- 1 cfati cfati 12099 Oct 9 08:43 six-1.12.0-py2.py3-none-any.whl [cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> for f in $(ls *.whl); do unzip ${f} > /dev/null; done [cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> ls asn1crypto cffi-1.12.3-cp35-cp35m-linux_x86_64.whl cryptography-2.7-cp35-cp35m-linux_x86_64.whl pycparser-2.19-py2.py3-none-any.whl asn1crypto-1.0.1.dist-info cffi-1.12.3.dist-info cryptography-2.7.dist-info six-1.12.0.dist-info asn1crypto-1.0.1-py3-none-any.whl _cffi_backend.cpython-35m-x86_64-linux-gnu.so pycparser six-1.12.0-py2.py3-none-any.whl cffi cryptography pycparser-2.19.dist-info six.py [cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> PYTHONPATH=.:${PYTHONPATH} python3 ../code00.py Python 3.5.2 (default, Jul 10 2019, 11:58:48) [GCC 5.4.0 20160609] 64bit on linux OpenSSL version: OpenSSL 1.0.2t-fips 10 Sep 2019 FIPS_mode(): 0 FIPS_mode_set(1): 1 FIPS_mode(): 1 Success !!! Done.Related (more or less) posts:
- [SO]: Unable to build a working FIPS capable OpenSSL on HP-UX
- [SO]: How to enable FIPS mode for libcrypto and libssl packaged with Python? (@CristiFati's answer)
- [SO]: How to implement FIPS_mode() and FIPS_mode_set() in Python 3.6's ssl module? (@CristiFati's answer)
这篇关于OpenSSL FIPS_mode_set在Python加密库中不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
