什么是补救Magento的是与20,000+产品慢的最佳方法 [英] What is the best method to remedy Magento being slow with 20,000+ products
问题描述
我在3亚马逊EC2实例中运行的Magento。一是建立直接的管理面板访问的，另外两个是坐在后面的负载平衡器。
事情顺利运行，直到我们导入我们的数据与20K +产品，每一个可配置的产品，〜4个简单的产品（不同的尺寸，颜色等。）
运行缓慢唯一的东西似乎是事情，遍历产品 - 无论是管理和前端目录页需要5-10 +秒钟就可以从服务器的响应。静态/ CMS的网页加载的罚款。
他们都连接到运行正常一个RDS MySQL数据库 - 我可以做的查询，并让他们快速反击
我们拥有所有的缓存（包括全页面缓存）启用，启用持平目录，在速度上没有真正的变化。
Magento的目录是每个服务器的独立，除了媒体/ DIR是保持同步 lsyncd 。管理服务器表现为主机和两台负载平衡前端服务器是奴隶。
让我们打破这：
-
假设我要（请他们）
- 您正在运行相当强大的EC2实例，例如m3.large
- 您正在运行像APC一个PHP缓存
- 您使用的是Magento的编译器系统 - >工具 - >编辑
- 您正在使用Apache网络服务器
- '5至10秒，以获得来自服务器的响应是指响应时间，并且不包括图像和CSS和JS下载时间在浏览器
- 您对产品和类别平表（但如果你的缓存工作正常，然后这应该不会称霸速度也可以运行测试，没有平坦的表;它们并不总是快）
- 您的Web服务器配置为高容量的流量优化，保持有效指示和Expires头
- 在三重检查你的全页面缓存
真是个奇怪的东西对你的问题是，你说你有一个完整的页面缓存但产品前端页面正在采取5至10秒，然后服务器将它们发送给浏览器。
在我看来，这意味着你的全页面缓存无法正常工作。一个完整的页面缓存，当正确实施，将成为Apache的页面直接的没有的运行Magento的应用程序（ Mage.php ）所有，该就是为什么它是这么快。这意味着没有在请求缓存的页面的开销，这就是为什么一整页的高速缓存系统应该有'要求'小于0.25秒倍，有时甚至小于0.1秒。
我建议你把你的整页缓存，并查看它使什么区别。检查你的主题，你的缓存文件，因为他们是如何处理类似篮子的总结和显示用户名不可缓存的页面内容 - 但任何缓存应该缓存所有产品块反正这样做前端产品页面应该访问缓存而不是访问数据库。
当然，如果你有20000的产品（或10万？= 20,000配置+ 4 * 20000 simples），那么每一页都需要访问一次来填充缓存，所以你可能要设置一个链接检查运行在一夜之间，全部命中你的网址和总理完整页面缓存为每个类别和/或产品页面。
- 检查你的主题.phtml作为文件可怕
法师:: getModel（目录/产品） - ＆GT;负载（$ _产品 - ＆GT;的getId（））
此行击中了你的数据库中的硬盘的，如果你正在做的，对于每一个产品类别页面上，然后它会带来麻烦。如果你的主题是使用 - ＆GT;负载（）然后跟你的主题设计有关创建集合只有他们所需要的属性。但如果你有一个页面缓存，那么这将不一定重要（所以为什么我觉得你的缓存不工作）。
- 看看你的
core_url_rewrite表格
有机会，这个表是巨大的，由于你拥有的产品。这可能有助于使您简单的产品不可见的，只有使目录和搜索configurables可见。检查多少行的表有，截断它，然后去Magento管理和重新索引重写 - 这将重新建表，你可以看它是否有更少的行（Magento的，似乎在很多重写的翻倍时间）。你也将获得全套改写为在Magento每个存储，以便删除不使用任何的商店。
在高速缓存现在另一个音符。我找到 core_url_rewrite 的瓶颈，所以我把周围的缓存之一 .phtml作为生成因为店里的菜单菜单并没有改变太多，这样做节省了大量数据库时间。但如果你有一个缓存中已经那么这将不会是一个问题，除非你的缓存不工作或没有正确设置。
- 获取瓦瑞恩探查工作
所以，你可以看到正在Magento的那么长。我想你会需要关闭缓存为它工作。这是href="http://www.mgt-commerce.com/magento-developer-toolbar.html" rel="nofollow">速度分析有用的工具 但是，这是不是测试你的数据库是否工作正常与否。也许你知道这一切，但你可以用phpmyadmin来检查，如果你的my.cnf的设置进行了优化。 phpmyadmin->岗位>顾问会给你提示 下面是一个真正优秀的＃1 Q＆放大器;关于ibdata文件，表格优化和InnoDB管理。警告：我不知道最佳的InnoDB设置了Magento的，但是当我读到这样一个文章中，我觉得它好像走正确的路 反正你应该确保你的my.cnf设置为使用可用内存给它的最佳方式（有没有一个神奇的设置，我不能告诉你，但研究这些参数：）。
innodb_buffer_pool 和的key_buffer_size 和的table_cache 。不管你做什么，Magento的不具有优化索引，使MySQL会总是有很多工作要做。你可能想看看你的InnoDB的文件，就像这里什么建议 （我这是需要读得），但如果你的ibdata1中的文件，你InnoDB日志文件不是太大，你没有慢查询日志中的任何东西，你是不是患了太多的锁等待再有，也许没有一个优势与<$ C运行$ C>innodb_file_per_table。有人建议 innodb_file_format =酷鱼 ，但我认为我们正在进入微调挤过去的毫秒出来。
的max_allowed_packet =
table_cache的=
sort_buffer_size的值=
read_buffer_size =
read_rnd_buffer_size =
myisam_sort_buffer_size =
tmp_table_size =
max_heap_table_size =
query_cache_size变量=
query_cache_type的=
thread_cache_size的=
innodb_fast_shutdown = 0
innodb_file_per_table
innodb_buffer_pool_size =
innodb_additional_mem_pool_size =
innodb_log_file_size =
innodb_log_buffer_size =
的innodb_flush_log_at_trx_commit =
innodb_lock_wait_timeout =
innodb_thread_concurrency参数=
跳外锁
MAX_CONNECTIONS =
read_buffer_size =
sort_buffer_size的值=
key_buffer_size的=
- SSH到您的箱子
和运行顶观看HTTP守护进程的内存和CPU负载和MySQL服务器 - 我有时会这样在运行我的链接检查工具，所以我可以看到系统中的至少一个小负荷下。如果有很小的负载，也许你的httpd.conf和my.cnf中未设置为使用的CPU和内存。如果你的CPU和内存是越来越累坏了，然后的也许的，你需要更大的盒子，但如果你的整个页面的缓存工作正常...
使用顶也显示，如果您的服务器已被入侵，所有的CPU cylces被挤兑一些脚本小子的比特币矿工。
- 在扔钱它
转到M3超大箱子，电话的Percona并采取一切其建议，得到一吨的RAM，并在ramdisk运行数据库，聘请来自Facebook的一些孩子对HHVM运行Magento的，或者说拧，并支付Magento的专家托管公司做这一切为您服务。但如果你的整个页面缓存工作...
----------
什么办法，我希望你玩得开心。我喜欢做Magento的运行速度更快。有一吨的旋钮来调整的，这是非常有意义的，看的页面加载时间一点一点下降。
呵呵，我觉得慢管理区将不会被全页面缓存得到帮助，但也有一些模块，在那里，使管理大型产品目录要简单得多。
I'm running magento on 3 Amazon EC2 instances. One is set up to be accessed directly for the admin panel, the other two are sitting behind a load balancer.
Things were running smoothly until we imported our data with 20k+ products, each a configurable product with ~4 simple products (for different sizes, colours etc.)
The only things running slow seem to be things that loop over products - both admin and frontend catalog pages take 5-10+ seconds to get a response from the server. Static/CMS pages load fine.
They're all connected to the one RDS MySQL database which is running fine - I can do queries and get them back quickly.
We have all caching (including full page cache) enabled and have enabled flat catalogs with no real change in speed.
The magento directories are independent per-server, except for the media/ dir which is kept in sync with lsyncd. The admin server behaves as the master and the two load-balanced frontend servers are the slaves.
Let's break this down:
Assumptions I will make (please check them)
- You are running on fairly powerful EC2 instances eg m3.large
- You are running a PHP cache like APC
- You are using the Magento compiler system->tools->compilation
- You are using Apache webserver
- '5 to 10 seconds to get a response from the server' means the response time and does not include image and CSS and JS download time to the browser
- You have flat tables for products and categories (but if your cache is working properly then this shouldn't dominate speed. You should also run tests without the flat tables; they are not always quicker)
- Your webserver configuration is optimized for high volume traffic for 'keep alives' and 'expires headers'
- Triple check your full-page caching
The really strange thing about your question is that you say you have a full page cache yet product front-end pages are taking 5 to 10 seconds before the server sends them to the browser.
In my opinion it means your full page cache is not working. A full page cache, when implemented properly, will serve the page from Apache directly without running the Magento app (Mage.php) at all, that is why it is so quick. It means there is no overhead when a cached page is requested and it is why a full page cache system should have 'requesting' times of less than 0.25 seconds and sometimes less than 0.1 seconds.
I suggest you turn your full page cache off and see what difference it makes. Check your theme and your cache documentation for how they handle non-cacheable page content like basket summary and display user name - but any cache should cache all the product blocks anyway so making a front-end product page should access the cache and not access the database.
Of course if you have 20000 products (or 100,000? = 20,000 configured + 4*20,000 simples) then every page needs to be visited once to populate your caches so you might want to set a link checker running overnight to hit all your URLs and prime the full page cache for each category and/or product page.
- Check your theme .phtml files for the dreaded
Mage::getModel(catalog/product)->load($_product->getId())
this line hits your database hard and if you are doing that for every product on a category page then it will spell trouble. If your theme is using ->load() then talk to your theme designers about creating collections with just the attributes they need. BUT if you have a page cache then this won't necessarily matter (hence why I think your cache isn't working).
- Take a look at your
core_url_rewritetable
Chances are that this table is huge due to all the products you have. It might help to make your simple products not visible and only make the configurables visible in catalog and search. Check how many rows the table has, truncate it, then go to the Magento Admin and re-index the rewrites - this will re build the table and you can see if it has less rows (Magento seems to double up a lot of rewrites over time). Also you will get a full set of rewrites for each store in Magento so delete any stores you aren't using.
Now another note on caches. I find the core_url_rewrite one of the bottle necks so I put a cache around the .phtml that generates the store menu because the menu doesn't change much and doing so saves a lot database time. BUT if you have a cache already then this won't be an issue unless your cache isn't working or isn't set up properly.
- Get the Varien profiler working
So that you can see what is taking magento so long. I think you will need to turn off the caches for it to work. This is a useful tool for speed profiling, but other free tools exist and actually you can just use the Varien profiler without a tool. The tool will give you an indication of what is taking a long time to build on the page (but if your cache is working properly then it won;t matter how long the page tales to build because the page would be served from the cache which is why I think your cache isn't working).
- You say 'MySQL database which is running fine - I can do queries and get them back quickly.'
But that isn't the test for whether your database is working fine or not. Maybe you know all this but you can use phpmyadmin to check if your my.cnf settings are optimised. phpmyadmin->status->advisor will give you hints for innodb_buffer_pool and key_buffer_size and table_cache. Whatever you do, Magento does not have optimised indexes so mysql will always have a lot of work to do. You might want to look at your innodb files like what is suggested here (and I this is required reading too), but if your ibdata1 file and you innodb log files aren't too big and you don't have anything in the slow query logs and you aren't suffering too many lock waits then there maybe isn't an advantage to running with 'innodb_file_per_table'. Some people suggest innodb_file_format=Barracuda, but I think we are getting into the fine tuning to squeeze the last millisecond out.
Here is a truly excellent Stackoverflow Q&A about ibdata files, table optimization and innodb management. Caveat: I don't know the optimum innodb set up for Magento but when I read articles like that one I think it seems like the correct way to go.
Anyway you should ensure your my.cnf is set to use the memory available to it in the optimum way (there is no one magic set up, I cannot tell you it, but study these parameters: ).
max_allowed_packet =
table_cache =
sort_buffer_size =
read_buffer_size =
read_rnd_buffer_size =
myisam_sort_buffer_size =
tmp_table_size =
max_heap_table_size =
query_cache_size =
query_cache_type =
thread_cache_size =
innodb_fast_shutdown = 0
innodb_file_per_table
innodb_buffer_pool_size =
innodb_additional_mem_pool_size =
innodb_log_file_size =
innodb_log_buffer_size =
innodb_flush_log_at_trx_commit =
innodb_lock_wait_timeout =
innodb_thread_concurrency =
skip-external-locking
max_connections =
read_buffer_size =
sort_buffer_size =
key_buffer_size =
- SSH into your boxes
and run 'top' to watch the memory and cpu loads of the http daemon and the mysql server - I sometimes do this while running my link checker so I can see the system under at least a little load. If there is very little load, maybe your httpd.conf and my.cnf are not set to utilise the CPU and memory available. If your CPU and memory is getting maxed out then maybe you need bigger boxes but if your full page cache is working properly...
Using top will also show if your server has been compromised and all the CPU cylces are being hogged by some script kiddie's bitcoin miner.
- Throw money at it
Go to M3 Extra large boxes, telephone Percona and take all its advice, get a tonne of RAM and run your database in a ramdisk, hire some kid from Facebook to run Magento on HHVM, or say 'screw it' and pay a Magento expert hosting company to do it all for you. But if your full page cache is working...
----------
Any way, I hope you have fun. I enjoy making Magento run faster. There are a tonne of knobs to tweak and it is very rewarding to see the page load times dropping bit by bit.
Oh, I think the slow admin area won't be helped by full page caching but there are some modules out there that make managing large product catalogues much simpler.
这篇关于什么是补救Magento的是与20,000+产品慢的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
