Django单元测试需要很长时间才能创建测试数据库 [英] Django Unit Testing taking a very long time to create test database

问题描述

一段时间以来，我的单元测试已经花费比预期的时间更长。我试图调试它几次，没有太多的成功，因为延迟是在我的测试甚至开始运行之前。这影响了我能够远程测试驱动开发的可能性（也许我的期望太高），所以我想看看我是否可以一劳永逸地解决这个问题。

当运行测试时，测试开始和实际开始之间有70到80秒的延迟。例如，如果我运行一个小模块的测试（使用 time python manage.py test myapp ），我得到

 < ...我从我的设置打印的一堆不重要的打印消息> 
 
创建别名default的测试数据库... 
 ...... 
 ----------------- ----------------------------------------------- 
 2.161s中的6个测试
 
 OK 
销毁别名default的测试数据库... 
 
 real 1m21.612s 
 user 1m17。 170s 
 sys 0m1.400s 
  

1m：21中约1m18位于

 为别名默认创建测试数据库... 
  

和

  ....... 
  

行。换句话说，测试需要3秒以下，但数据库初始化似乎是在1：18分钟之内。

我有大约30个应用程序，大多数使用1到3个数据库模型这应该给出项目规模的想法。我使用SQLite进行单元测试，并且已经实现了一些建议的改进。我不能发布我的整个设置文件，但很乐意添加所需的任何信息。

我使用跑步者

django.test.runner导入DiscoverRunner
从django.conf导入设置

类别ExcludeAppsTestSuiteRunner（DiscoverRunner）：

  覆盖默认的django'test'命令，排除测试
应用程序，我们知道将失败。
 
 def run_tests（self，test_labels，extra_tests = None，** kwargs） ：
如果没有test_labels：
＃在命令行上没有指定任何appnames，所以我们运行所有
＃测试，但删除那些我们知道是麻烦的。 
 test_labels =（
'app1'，
'app2'，
 .... 
）
 print（'Testing：'+ str（test_labels ）
 
 return super（ExcludeAppsTestSuiteRunner，self）.run_tests（
 test_labels，extra_tests，** kwargs）
  

在我的设置中：

  TEST_RUNNER ='config.test_runner.ExcludeAppsTestSuiteRunner'
  

我还尝试使用 django-nose django-nose-exclude

我已经阅读了很多关于如何加快测试本身，但没有发现任何线索如何优化或避免数据库初始化。我已经看到有关尝试不使用数据库进行测试的建议，但我不能或不知道如何完全避免这种情况。

请让我知道，如果

1. 这是正常和预期的


2. 不期望（希望能修复或导致做什么） / li>
   

再次，我不需要帮助如何加快测试本身，而是初始化（或开销）。我想要上面的例子需要10秒而不是80秒。

非常感谢

我运行测试单一应用程序）与 - verbose 3 ，并发现这与迁移有关：

 渲染模型状态... DONE（40.500s）
应用authentication.0001_initial ... OK（0.005s）
应用account.0001_initial ... OK（0.022s）
应用account.0002_email_max_length ... OK（0.016s）
应用contenttypes.0001_initial ... OK（0.024s）
应用contenttypes.0002_remove_content_type_name ... OK（0.048s）
应用s3video.0001_initial ... OK（0.021s）
应用s3picture.0001_initial ... OK（0.052s）
 ...更多的这样
  / pre> 
 
 
我压缩了所有的迁移，但仍然很慢。
解决方案
修复我的问题的最终解决方案是强制Django在测试期间禁用迁移，这可以从这样的设置完成
  TESTING = len（sys.argv）> 1和sys.argv [1] =='test'
如果TESTING：
 print（'===================== =='）
打印（'在测试模式 - 禁用迁移'）
 print（'====================== ='）
 
 class DisableMigrations（object）：
 
 def __contains __（self，item）：
 return True 
 
 def __getitem __自我，项目）：
返回notmigrations
 
 MIGRATION_MODULES = DisableMigrations（）
  
或使用 https://pypi.python.org/pypi/ django-test-without-migrations  
 
 
 
我的整个测试大约需要1分钟，一个小应用程序需要5秒钟。 
 
 
 
在我的情况下，迁移不需要进行测试，因为我在迁移时更新测试，并且不要使用迁移来添加数据。这对每个人都不行。
 
For some time now, my unit testing has been taking a longer than expected time. I have tried to debug it a couple of times without much success, as the delays are before my tests even begin to run. This has affected my ability to do anything remotely close to test driven development (maybe my expectations are too high), so I want to see if I can fix this once and for all.

When a run a test, there is a 70 to 80sec delay between the start and the actual beginning of the test. For example, if I run a test for a small module (using time python manage.py test myapp), I get
<... bunch of unimportant print messages I print from my settings>

Creating test database for alias 'default'...
......
----------------------------------------------------------------
Ran 6 tests in 2.161s

OK
Destroying test database for alias 'default'...

real    1m21.612s
user    1m17.170s
sys     0m1.400s
About 1m18 of the 1m:21 are between the 
Creating test database for alias 'default'...
and the 
.......
line. In other words, the test takes under 3sec, but the database initialization seems to be taking 1:18min

I have about 30 apps, most with 1 to 3 database models so this should give an idea of the project size. I use SQLite for unit testing, and have implemented some of the suggested improvements. I cannot post my whole setting file, but happy to add any information that is required.

I do use a runner
from django.test.runner import DiscoverRunner
from django.conf import settings

class ExcludeAppsTestSuiteRunner(DiscoverRunner):
    """Override the default django 'test' command, exclude from testing
    apps which we know will fail."""

    def run_tests(self, test_labels, extra_tests=None, **kwargs):
        if not test_labels:
            # No appnames specified on the command line, so we run all
            # tests, but remove those which we know are troublesome.
            test_labels = (
                'app1',
                'app2',
                ....
                )
            print ('Testing: ' + str(test_labels))

        return super(ExcludeAppsTestSuiteRunner, self).run_tests(
                test_labels, extra_tests, **kwargs)
and in my settings:
TEST_RUNNER = 'config.test_runner.ExcludeAppsTestSuiteRunner'
I have also tried using django-nose with django-nose-exclude

I have read a lot about how to speed up the test themselves, but have not found any leads on how to optimize or avoid the database initialization. I have seen the suggestions on trying not to test with the database but I cannot or don't know how to avoid that completely.

Please let me know if

This is normal and expected
Not expected (and hopefully a fix or lead on what to do)
Again, I don't need help on how to speed up the test themselves, but the initialization (or overhead). I want the example above to take 10sec instead of 80sec.

Many thanks

I run the test (for single app) with --verbose 3 and discovered this is all related to migrations:
  Rendering model states... DONE (40.500s)
  Applying authentication.0001_initial... OK (0.005s)
  Applying account.0001_initial... OK (0.022s)
  Applying account.0002_email_max_length... OK (0.016s)
  Applying contenttypes.0001_initial... OK (0.024s)
  Applying contenttypes.0002_remove_content_type_name... OK (0.048s)
  Applying s3video.0001_initial... OK (0.021s)
  Applying s3picture.0001_initial... OK (0.052s)
  ... Many more like this
I squashed all my migrations but still slow.
解决方案
The final solution that fixes my problem is to force Django to disable migration during testing, which can be done from the settings like this
TESTING = len(sys.argv) > 1 and sys.argv[1] == 'test'
if TESTING:
    print('=========================')
    print('In TEST Mode - Disableling Migrations')
    print('=========================')

    class DisableMigrations(object):

        def __contains__(self, item):
            return True

        def __getitem__(self, item):
            return "notmigrations"

    MIGRATION_MODULES = DisableMigrations()
or use https://pypi.python.org/pypi/django-test-without-migrations

My whole test now takes about 1 minute and a small app takes 5 seconds. 

In my case, migrations are not needed for testing as I update tests as I migrate, and don't use migrations to add data. This won't work for everybody

                        
这篇关于Django单元测试需要很长时间才能创建测试数据库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！