获取有关气流on_failure_callback上下文的异常详细信息 [英] Get Exception details on Airflow on_failure_callback context

问题描述

有什么方法可以获取气流on_failure_callback上的异常详细信息吗?

Is there any way to get the exception details on the airflow on_failure_callback?

我注意到它不属于context.我想创建一个通用的异常处理机制，该机制将有关错误的信息(包括有关异常的详细信息)发布到Slack.我现在设法触发/执行了回调并将其发布到Slack，但是无法发布异常详细信息.

I've noticed it's not part of context. I'd like to create a generic exception handling mechanism which posts to Slack information about the errors, including details about the exception. I've now managed to trigger/execute the callback and post to Slack, but can't post the exception details.

谢谢.

推荐答案

可以将on_failure_callback提供给DAG和/或单个任务.

An on_failure_callback can be supplied to the DAG and/or individual tasks.

在第一种情况下(提供给DAG)，context中没有'exception'(参数Airflow调用您的on_failure_callback使用).

In the first case (supplying to the DAG), there is no 'exception' in the context (the argument Airflow calls your on_failure_callback with).

在第二种情况下(为任务提供)，有.

In the second case (supplying to a task), there is.

包含的对象应该是python Exception. 令人惊讶的是，从中获得类似堆栈跟踪的信息是不直观的，但是从此答案中，我使用以下内容来获得相当可读的堆栈跟踪:

The contained object should be a python Exception. It's surprisingly non-intuitive to get something like a stack trace from that, but from this answer I use the following to get a fairly readable stack trace:

import traceback

...

exception = context.get('exception')
formatted_exception = ''.join(
   traceback.format_exception(etype=type(exception), 
     value=exception, tb=exception.__traceback__
   )
).strip()

这篇关于获取有关气流on_failure_callback上下文的异常详细信息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！