优化Jinja2环境创建 [英] Optimizing Jinja2 Environment creation

问题描述

我的应用程序正在Google App Engine上运行，并且由于CPU使用率过高，大多数请求不断获得黄色标记。使用事件探查器，我将问题追踪到创建 jinja2.Environment 实例的例程中。

我正在创建实例在模块级别：

  from jinja2 import Environment，FileSystemLoader 
 jinja_env = Environment（loader = FileSystemLoader（TEMPLATE_DIRS）） 
  

由于Google AppEngine操作模式（CGI），此代码可以在每次请求时运行（它们的模块导入缓存似乎在几秒钟内缓存模块，而不是几分钟）。

我正在考虑将环境实例存储在memcache中，但它似乎不可用。 FileSystemLoader 实例似乎是可挑选的并且可以被缓存，但是我没有发现使用这种方法在CPU使用方面有任何实质性的改进。

任何人都可以建议一种方法来减少创建 jinja2.Environment 实例的开销？

编辑：下面是profiler输出的（相关）部分。

  222172函数调用（215262原始调用）in 8.695 CPU秒
 
 ncalls tottime percall cumtime percall文件名：lineno（功能）
 33 1.073 0.033 1.083 0.033 {google3.apphosting.runtime._apphosting_runtime ___ python__apiproxy.Wait} 
 438/111 0.944 0.002 2.009 0.018 /base/python_dist/lib/python2.5/sre_parse.py:385(_parse）
 4218 0.655 0.000 1.002 0.000 /base/python_dist/lib/python2.5/pickle.py:1166(load_long_binput） 
 1 0.611 0.611 0.679 0.679 /base/data/home/apps/with-the-flow/1.331879498764931274/jinja 2 / environment.py：10（）
  

一次调用，但据我所知（和此在所有基于GAE的应用程序中都是一致的），这是整个请求处理周期中最昂贵的。

- 将Jinja2模板编译为python代码，并在生产环境中使用已编译的模板。所以我为此编写了一个编译器/加载程序，现在它以13倍的速度渲染一些复杂模板，并将 all 解析开销消除。有关存储库链接的相关讨论是此处。

My application is running on Google App Engine and most of requests constantly gets yellow flag due to high CPU usage. Using profiler I tracked the issue down to the routine of creating jinja2.Environment instance.

I'm creating the instance at module level:

from jinja2 import Environment, FileSystemLoader
jinja_env = Environment(loader=FileSystemLoader(TEMPLATE_DIRS))

Due to the Google AppEngine operation mode (CGI), this code can be run upon each and every request (their module import cache seems to cache modules for seconds rather than for minutes).

I was thinking about storing the environment instance in memcache, but it seems to be not picklable. FileSystemLoader instance seems to be picklable and can be cached, but I did not observe any substantial improvement in CPU usage with this approach.

Anybody can suggest a way to decrease the overhead of creating jinja2.Environment instance?

Edit: below is (relevant) part of profiler output.

222172 function calls (215262 primitive calls) in 8.695 CPU seconds

 ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     33    1.073    0.033    1.083    0.033 {google3.apphosting.runtime._apphosting_runtime___python__apiproxy.Wait}
438/111    0.944    0.002    2.009    0.018 /base/python_dist/lib/python2.5/sre_parse.py:385(_parse)
   4218    0.655    0.000    1.002    0.000 /base/python_dist/lib/python2.5/pickle.py:1166(load_long_binput)
      1    0.611    0.611    0.679    0.679 /base/data/home/apps/with-the-flow/1.331879498764931274/jinja2/environment.py:10()

One call, but as far I can see (and this is consistent across all my GAE-based apps), the most expensive in the whole request processing cycle.

解决方案

Armin suggested to pre-compile Jinja2 templates to python code, and use the compiled templates in production. So I've made a compiler/loader for that, and it now renders some complex templates 13 times faster, throwing away all the parsing overhead. The related discussion with link to the repository is here.

这篇关于优化Jinja2环境创建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！