将csv文件中的数据加载到Django模型中 [英] loading data from a csv file into a Django model

问题描述

我有一个简单却令人沮丧的问题，我似乎无法弄清.

I have a simple yet frustrating problem that I can't seem to figure out.

我正在尝试将csv文件中的数据加载到Django模型中.为此，我编写了以下脚本作为视图:

I'm trying to load data from a csv file into a Django model. To do so I've written the following script as a view:

import csv
def import_db(request):
    dataReader = csv.reader(open('/home/<name>/webapps/<name2>/employees.csv'), delimiter=',', quotechar='"')
    for row in dataReader:
        emp = Employee()
        emp.first_name = row[0]
        emp.last_name = row[1]
        emp.email = row[2]
        emp.level = row[3]
        emp.service_area = row[4]
        emp.service_line = row[5]
        emp.office = row[6]
        emp.save()
return HttpResponse("Completed", content_type="text/plain")

我已将视图链接到URL，如下所示:

I've linked the view to a url as follows:

from reviews import views as emp  
url(r'^load/$',   emp.import_db, name='importdb')

这个想法是，当我转到sitename.com/load链接时，我的数据将从我的employee.csv文件加载到我的Employee模型中.

The idea being that when I go to the link sitename.com/load, my data will get loaded from my employee.csv file into my Employee model.

问题是当我运行此脚本时，在Django模型中，我的csv文件中的每一行都有2个条目.我在csv中有1530个员工行，执行此操作时模型会填充3060个实例.更令人讨厌的是，模型中条目的顺序与csv文件不同，因此我不能简单地删除1530个模型实例的第二个组".即使当我尝试使用csv文件中的20行数据的子集进行尝试时，我也会获得40个模型实例.知道为什么会发生这种情况以及我该如何解决吗?

The problem is when I run this script, I get 2 entries in my Django model for every line in my csv file. I have 1530 employee lines in the csv and the model gets populated with 3060 instances when I do this. What's even more annoying is that the order of the entries in the model is not the same as the csv file so I can't simply delete the second 'group' of 1530 model instances. Even when I try it with a subset of 20 lines of data in the csv file I get 40 model instances. Any idea why this is happening and what i can do to fix it?

非常感谢！

推荐答案

在视图函数中执行此逻辑不是一个好主意.我不知道您为什么会这样，但是可以触发两次相同的视图功能(例如，将Google Chrome浏览器之类的浏览器粘贴到地址栏中后，往往会在后台预取网址-这可能会导致两个我不知道这是否是问题所在，但可能值得排除.)

Performing this logic in a view function is not a good idea. I don't know exactly why in your case, but it is possible to trigger the same view function twice (e.g., browsers like Google Chrome often prefetch URLs in the background as soon as you paste them into the address bar - this can cause two hits on that view. I don't know whether that is the problem here but it may be worth ruling out).

您应该考虑将此逻辑转移到自定义管理命令中可以确定地调用.像这样:

You should consider moving this logic into a custom management command that you can call deterministically. Something like:

# myapp/management/commands/import_csv.py
from django.core.management.base import BaseCommand, CommandError


class Command(BaseCommand):

    def add_arguments(self, parser):
        parser.add_argument('csv_file', nargs='+', type=str)

    def handle(self, *args, **options):
        for csv_file in options['csv_file']:
            dataReader = csv.reader(open(csv_file), delimiter=',', quotechar='"')
            for row in dataReader:
                emp = Employee()
                # etc...
                self.stdout.write(
                    'Created employee {} {}'.format(emp.first_name, emp.last_name)
                )

然后您可以通过以下方式调用它:

You would then call this with:

./manage.py import_csv --csvfile "/home/<name>/webapps/<name2>/employees.csv"

这种方法使您可以更好地控制正在发生的事情，并且可以更轻松地调试问题(如果仍有问题的话).

This approach gives you more control over what is going on and will make it easier to debug the issue (if there still is one).

这篇关于将csv文件中的数据加载到Django模型中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！