我如何使用 spacy3.0 的 nlp.update 问题示例 [英] How can i work with Example for nlp.update problem with spacy3.0

问题描述

我正在尝试使用 spacy v3.0 训练我的数据，显然 nlp.update 不接受任何元组.这是一段代码:

i am trying to train my data with spacy v3.0 and appareantly the nlp.update do not accept any tuples. Here is the piece of code:

import spacy
import random
import json
nlp = spacy.blank("en")
ner = nlp.create_pipe("ner")
nlp.add_pipe('ner')
ner.add_label("label")
# Start the training
nlp.begin_training()
# Loop for 40 iterations
for itn in range(40):
    # Shuffle the training data
    random.shuffle(TRAINING_DATA)
    losses = {}
# Batch the examples and iterate over them
    for batch in spacy.util.minibatch(TRAINING_DATA, size=2):
        texts = [text for text, entities in batch]
        annotations = [entities for text, entities in batch]
# Update the model
        nlp.update(texts, annotations, losses=losses, drop=0.3)
    print(losses)

我收到错误

ValueError                                Traceback (most recent call last)
<ipython-input-79-27d69961629b> in <module>
     18         annotations = [entities for text, entities in batch]
     19 # Update the model
---> 20         nlp.update(texts, annotations, losses=losses, drop=0.3)
     21     print(losses)

~\Anaconda3\lib\site-packages\spacy\language.py in update(self, examples, _, drop, sgd, losses, component_cfg, exclude)
   1086         """
   1087         if _ is not None:
-> 1088             raise ValueError(Errors.E989)
   1089         if losses is None:
   1090             losses = {}

ValueError: [E989] `nlp.update()` was called with two positional arguments. This may be due to a backwards-incompatible change to the format of the training data in spaCy 3.0 onwards. The 'update' function should now be called with a batch of Example objects, instead of `(text, annotation)` tuples. 

我设置了我的火车数据格式:

I set my train data format:

TRAINING_DATA = []
for entry in labeled_data:
    entities = []
    for e in entry['labels']:
        entities.append((e[0], e[1],e[2]))
    spacy_entry = (entry['text'], {"entities": entities})
    TRAINING_DATA.append(spacy_entry)

我的火车数据如下所示:

My train data looks like this:

[('Part List', {'entities': []}), ('pending', {'entities': []}), ('3D Printing', {'entities': [(0, 11, 'Process')]}), ('Recommended to use a FDM 3D printer with PLA material.', {'entities': [(25, 36, 'Process'), (41, 44, 'Material')]}), ('', {'entities': []}), ('No need supports or rafts.', {'entities': []}), ('Resolution: 0.20mm', {'entities': []}), ('Fill density 20%', {'entities': []}), ('As follows from the analysis, part of the project is devoted to 3D', {'entities': [(64, 66, 'Process')]}), ('printing, as all static components were created using 3D modelling and', {'entities': [(54, 66, 'Process')]}), ('subsequent printing.', {'entities': []}), ('', {'entities': []}), ('In our project, we created several versions of the', {'entities': []}), ('model during modelling, which we will describe and document in the', {'entities': []}), ('following subchapters. As a tool for 3D modelling, we used the Sketchup', {'entities': [(37, 49, 'Process')]}), ('Make tool, version from 2017. The main reason was the high degree of', {'entities': []}), ('intuitiveness and simplicity of the tool, as we had not encountered 3D', {'entities': [(68, 70, 'Process')]}), ('modelling before and needed a relatively flexible and efficient tool to', {'entities': []}), ('guarantee the desired result. with zero previous experience.', {'entities': []}), ('In this version, which is shown in the figures Figure 13 - Version no. 2 side view and Figure 24 - Version no. 2 - front view, for the first time, the specific dimensions of the infuser were clarified and', {'entities': []}), ('modelled. The details of the lower servo attachment, the cable hole in', {'entities': []}), ('the main mast, the winding cylinder mounting, the protrusion on the', {'entities': [(36, 44, 'Process')]}), ('winding cylinder for holding the tea bag, the preparation for fitting', {'entities': []}), ('the wooden and aluminium plate and the shape of the cylinder end that', {'entities': [(15, 25, 'Material')]}), ('exactly fit the servo were also reworked.', {'entities': []}), ('After the creation of this', {'entities': []}), ('version of the model, this model was subsequently officially consulted', {'entities': []}), ('and commented on for the first time.', {'entities': []}), ('In this version, which is shown in the figures Figure 13 - Version no. 2 side view and Figure 24 - Version no. 2 - front view, for the first time, the specific dimensions of the infuser were clarified and', {'entities': []}), ('modelled. The details of the lower servo attachment, the cable hole in', {'entities': []}), ('the main mast, the winding cylinder mounting, the protrusion on the', {'entities': [(36, 44, 'Process')]})]

感谢您作为新贡献者的帮助.非常感谢！

I would appreciate your help as a new contributor. Thanks a lot!

推荐答案

你没有提供你的 TRAIN_DATA，所以我无法重现它.但是，您应该尝试以下操作:

You didn't provide your TRAIN_DATA, so I cannot reproduce it. However, you should try something like this:

from spacy.training.example import Example

for batch in spacy.util.minibatch(TRAINING_DATA, size=2):
    for text, annotations in batch:
        # create Example
        doc = nlp.make_doc(text)
        example = Example.from_dict(doc, annotations)
        # Update the model
        nlp.update([example], losses=losses, drop=0.3)

这篇关于我如何使用 spacy3.0 的 nlp.update 问题示例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！