达斯工人似乎已死，但找不到工人日志以找出原因 [英] Dask worker seem die but cannot find the worker log to figure out why

问题描述

我在本地计算机上运行了一段DASK代码，该代码可以在90％的时间内工作，但有时会卡住.卡住了.没有崩溃，没有错误打印出来的不是CPU的使用情况.永远不会结束.

I have a piece of DASK code run on local machine which work 90% of time but will stuck sometimes. Stuck mean. No crash, no error print out not cpu usage. never end.

我在Google上搜索并认为这可能是由于一些工人死亡所致.如果我可以看到工作人员日志并弄清楚原因，那么我将非常有用.但是我找不到我的工作日志.我去编辑config.yaml以添加日志记录，但仍然看不到stderr.然后我转到信息中心->信息->日志，然后看到空白页.

I google and think it maybe due to some worker dead. I will be very useful if I can see the worker log and figure out why. But I cannot find my worker log. I go to edit config.yaml to add loging but still see nothing from stderr. Then I go to dashboard --> info --> logs and see blank page.

卡住的代码是X_test = df_test.to_dask_array(长度=真)或者proba = y_pred_proba_train [:, 1] .compute()

The code it stuck is X_test = df_test.to_dask_array(lengths=True) or proba = y_pred_proba_train[:, 1].compute()

和我的〜/.config/dask/config.yaml或〜.dask/config.yaml看起来像记录:分布式:信息Distributed.client:警告分布式工作人员:调试散景:错误

and my ~/.config/dask/config.yaml or ~.dask/config.yaml look like logging: distributed: info distributed.client: warning distributed.worker: debug bokeh: error

我正在使用python 3.6达人1.1.4我需要的只是一种查看日志的方式，这样我就可以尝试找出问题所在.

I am using python 3.6 dask 1.1.4 All I need is a way to see the log so that I can try to figure out what goes wrong.

谢谢

约瑟夫

推荐答案

工作日志通常由用于设置Dask的任何系统管理.

Worker logs are usually managed by whatever system you use to set up Dask.

也许您使用过Kubernetes或Yarn或SLURM之类的东西?

Perhaps you used something like Kubernetes or Yarn or SLURM?

这些系统都有获取日志的方法.

These systems all have ways to get logs back.

不幸的是，一旦Dask工人不再运行，Dask本身就无法为您收集日志.您需要使用用于启动Dask的系统.

Unfortunately, once a Dask worker is no longer running, Dask itself has no ability to collect logs for you. You need to use the system that you use to launch Dask.

这篇关于达斯工人似乎已死，但找不到工人日志以找出原因的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！