Windows 上 Jupyter 中的 python 多处理:AttributeError:无法获取属性“abc" [英] python multiprocessing in Jupyter on Windows: AttributeError: Can't get attribute "abc"
本文介绍了Windows 上 Jupyter 中的 python 多处理:AttributeError:无法获取属性“abc"的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试运行一个简单的命令，该命令使用多处理按名称猜测性别.这段代码在以前的机器上工作过，所以也许我的设置与它有关.
I am trying to run a simple command that guesses gender by name using multiprocessing. This code worked on a previous machine so perhaps my setup had something to do with it.
下面是我的多处理代码:
Below is my multiprocessing code:
import sys
import gender_guesser.detector as gender
import multiprocessing
import time
d = gender.Detector()
def guess_gender (name):
n = name.title() # make first letter upper case and the rest lower case
g = d.get_gender(n) # guess gender
return g
ls = ['john','joe','amamda','derick','peter','ashley','john','joe','amamda','derick','peter','ashley']
t=time.time()
results=[]
def callBack(x):
results.append(x)
pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()-1, maxtasksperchild=1)
for n in ls:
print (n)
pool.apply_async(guess_gender,args=[n],callback=callBack)
pool.close()
pool.join()
results = pd.concat(results)
print(time.time()-t)
它只是运行，不做任何事情.在我的 cmd 窗口中，我在错误消息的末尾看到以下内容:
It simply runs and doesn't do anything. In my cmd window, I see the following at the end of an error message:
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
我在 Anaconda 上运行 python 版本 3.6.1:
Am running python version 3.6.1 on Anaconda:
import sys
print(sys.version)
3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:25:24) [MSC v.1900 64 bit (AMD64)]
更新:仍然无法让它工作.下面是我运行提供的代码时的整个 cmd 日志.感谢您的任何想法！
Update: Still cannot get it to work. Below is the entire cmd log when I ran the code provided. I appreciate any thoughts you may have!
C:UsersywuGoogle Drive>jupyter notebook
[I 10:13:43.954 NotebookApp] Serving notebooks from local directory: C:UsersywuGoogle Drive
[I 10:13:43.954 NotebookApp] 0 active kernels
[I 10:13:43.955 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=255a5c0c9af337a1c2187feb63f1c426fb903e5929a0b2f0
[I 10:13:43.956 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 10:13:43.959 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8888/?token=255a5c0c9af337a1c2187feb63f1c426fb903e5929a0b2f0
[I 10:13:44.264 NotebookApp] Accepting one-time-token-authenticated connection from ::1
[W 10:13:44.319 NotebookApp] 404 GET /api/kernels/aceb78ee-73e4-4481-9993-63e5ee8f72cb/channels?session_id=AEA3C6B2B0A440FC84FF3BAF5F5CB615 (127.0.0.1): Kernel does not exist: aceb78ee-73e4-4481-9993-63e5ee8f72cb
[W 10:13:44.328 NotebookApp] 404 GET /api/kernels/aceb78ee-73e4-4481-9993-63e5ee8f72cb/channels?session_id=AEA3C6B2B0A440FC84FF3BAF5F5CB615 (127.0.0.1) 20.07ms referer=None
[I 10:13:54.740 NotebookApp] Creating new notebook in /code/python
[I 10:13:55.241 NotebookApp] Kernel started: 45ab2da6-7466-408c-aa5a-98f7db54e711
[W 10:14:00.341 NotebookApp] Replacing stale connection: aceb78ee-73e4-4481-9993-63e5ee8f72cb:AEA3C6B2B0A440FC84FF3BAF5F5CB615
Process SpawnPoolWorker-2:
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
Process SpawnPoolWorker-1:
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
Process SpawnPoolWorker-4:
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
Process SpawnPoolWorker-3:
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
Process SpawnPoolWorker-5:
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
Process SpawnPoolWorker-6:
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
Traceback (most recent call last):
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
Process SpawnPoolWorker-7:
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
Process SpawnPoolWorker-8:
Process SpawnPoolWorker-9:
Process SpawnPoolWorker-10:
Traceback (most recent call last):
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
Traceback (most recent call last):
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
Process SpawnPoolWorker-11:
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
Process SpawnPoolWorker-14:
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
Traceback (most recent call last):
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
Traceback (most recent call last):
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 249, in _bootstrap
self.run()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingprocess.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingpool.py", line 108, in worker
task = get()
File "C:UsersywuAppDataLocalContinuumAnaconda3libmultiprocessingqueues.py", line 345, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'guess_gender' on <module '__main__' (built-in)>
[W 10:14:15.043 NotebookApp] 404 GET /api/kernels/c1224db6-69c6-470e-b74c-4c7b94fb48fe/channels?session_id=D8DC8A440B044EED8EBCA374EBEAF7C6 (127.0.0.1): Kernel does not exist: c1224db6-69c6-470e-b74c-4c7b94fb48fe
[W 10:14:15.046 NotebookApp] 404 GET /api/kernels/c1224db6-69c6-470e-b74c-4c7b94fb48fe/channels?session_id=D8DC8A440B044EED8EBCA374EBEAF7C6 (127.0.0.1) 7.48ms referer=None
推荐答案
通过将我的函数保存在单独的 .py 文件中并将该文件包含在我的笔记本中，我可以在 Windows 上的 Jupyter 笔记本中进行多处理.
I got multiprocessing to work from within a Jupyter notebook on Windows by saving my function in a separate .py file and including that file in my notebook.
例子:
f.py:
def f(name, output):
output.put('hello {0}'.format(name))
return
Jupyter 笔记本中的代码:
from multiprocessing import Process, Queue
#Having the function definition here results in
#AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
#The solution seems to be importing the function from a separate file.
import f
#Also, the original version of f only had a print statement in it.
#That doesn't work with Process - in the sense that it prints to the console
#instead of the notebook.
#The trick is to let f write the string to print into an output-queue.
#When Process is done, the result is retrieved from the queue and printed.
if __name__ == '__main__':
# Define an output queue
output=Queue()
# Setup a list of processes that we want to run
p = Process(target=f.f, args=('Bob',output))
# Run process
p.start()
# Exit the completed process
p.join()
# Get process results from the output queue
result = output.get(p)
print(result)
我是 Python 新手，我可能错过了各种细节，但这对我有用.
I'm a Python newby and I may have missed all sorts of details, but this works for me.
这篇关于Windows 上 Jupyter 中的 python 多处理:AttributeError:无法获取属性“abc"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
