HBase Kerberos连接更新策略 [英] HBase Kerberos connection renewal strategy

问题描述

最近我在群集中启用了kerberos，一切都很好，直到我的kerberos登录到期，比如说12个小时。在那个时候，我创建的任何连接，使用这些连接创建的任何表都会在我使用它时抛出。这可能会导致我的应用程序崩溃，具体取决于我如何处理。

我并不介意巨大的崩溃，因为我的应用程序是由滑块管理的，它会在应用程序重新启动时它会崩溃，但是这只会在HBase被使用时才会发生（也就是说，我调用了一个表格上的方法，这种方法可能会导致用户交互），这会导致用户体验差。

我不希望身份验证实现细节遍布我的应用程序，也不希望更频繁地创建连接对象，因为这是一项耗资巨大的操作，的RPC调用（zookeeper元数据位置开始）。

是否有共同的策略（最好内置在HBase客户端中）用于管理kerberos身份验证到期并更新HBase连接/表Kerberos TGT有一个生命周期（例如12h）和一个 。 >更新能够终身使用（例如7天）。只要票证仍然有效并且仍然可以更新，您可以申请免费续订 - 不需要密码 - 并重置终生计数器（例如，再次运行12小时）。

Hadoop认证库为当前TGT的自动续订生成特定的Java线程。这很丑陋，使用 kinit -R 命令行而不是JAAS库调用，但它起作用 - 请参阅 HADOOP-6656

因此，如果您在启动时让Slider创建可再生票，并且您可以贿赂您的系统管理员将默认值（cf. client conf）和最大值（参见KDC conf）可续订的生命期提高到30天，那么您的应用可以连续运行30天，最初的TGT。一个不错的改进。

~~~~~~~~~~

如果你真的渴望永恒...对不起，但你实际上会有一些编程工作要做。这意味着专用线程/进程负责或自动重新创建。

• Java方式：在启动时，在连接到HBase / HDFS之前，
  明确地创建一个 UGI with loginUserFromKeytab（）然后运行
  checkTGTAndReloginFromKeytab（）


• Shell命令：启动一个shell，它（a）用 kinit （b）
产生一个子进程，它会定期再次触发 kinit （c）
启动您的Java应用程序，然后杀死子进程应用终止



警告：如果某个其他线程在TGT重新启动时偶然打开或重新打开连接，创建时，该连接可能会失败，因为缓存在访问的确切时间为空（竞争条件）。下一次尝试将取得成功，但会在您的日志中出现一些流氓警告。

~~~~~~~~~~

最后建议：您可以为您的应用使用私有票证缓存（即，您可以在具有相同Linux帐户但不同Kerberos主体的同一节点上运行多个应用） KRB5CCNAME 环境变量，只要它是FILE：缓存即可。

Recently I enabled kerberos in my cluster, everything works great until my kerberos login expires, at say, 12 hours. At that point any connections I have created, any tables created with those connections etc will throw when I use them. This could potentially crash my app depending on how I handle this.

I don't mind crashing hugely because my app is managed by slider which will resurrect the app if and when it goes down, however this will only happen when HBase is "used" (i.e. I call a method on a table with a now stale connection) which will probably be caused by a user interaction and this would lead to poor UX.

I don't want authentication implementation details to pervade my application and also don't want to create connection objects more often than is necessary because it is a costly operation which makes a large number of RPC calls (zookeeper metadata location to start with).

Is there a common strategy (preferably inbuilt in HBase client) for managing kerberos authentication expiry and renewing HBase connections/tables when that happens?

解决方案

A Kerberos TGT has a lifetime (e.g. 12h) and a renewable lifetime (e.g. 7 days). As long as the ticket is still valid and is still renewable, you can request a "free" renewal -- no password required --, and the lifetime counter is reset (e.g. 12h to go, again).

The Hadoop authentication library spawns a specific Java thread for automatic renewal of the current TGT. It's kind of ugly, using a kinit -R command line instead of a JAAS library call, but it works - see HADOOP-6656

So, if you get Slider to create a renewable ticket on startup, and if you can bribe your SysAdmin to raise the default (cf. client conf) and the max (cf. KDC conf) renewable lifetime to, say, 30 days, then your app could run for 30 days straight with the initial TGT. A nice improvement.

~~~~~~~~~~

If you really crave for eternity... sorry, but you will actually have some programming to do. That means a dedicated thread/process in charge or re-creating automagically the TGT.

• The Java Way: on startup, before you connect to HBase/HDFS/whatever, create explicitly an UGI with loginUserFromKeytab() then run checkTGTAndReloginFromKeytab() from time to time
• The Shell Way: start a shell that (a) creates a TGT with kinit (b) spawns a sub-process that periodically fires kinit again (c) launches your Java app then kills the subprocess when/if your app ever terminates

Caveat: if some other thread happens to open, or re-open, a connection while the TGT is being re-created, that connection may fail because the cache was empty at the exact time it was accessed ("race condition"). The next attempt will be successful, but expect a few rogue warnings in your logs.

~~~~~~~~~~

Final advice: you can use a private ticket cache for your app (i.e. you can run multiple apps on the same node with the same Linux account but different Kerberos principals) by setting KRB5CCNAME environment variable, as long as it's a "FILE:" cache.

这篇关于HBase Kerberos连接更新策略的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！