如何使用Laravel和MySQL获取巨大的记录? [英] How can fetching huge records using Laravel and MySQL?
问题描述
我需要专家的建议和解决方案.我们正在这里开发工作门户网站，处理大约一百万条记录.我们正在面对获取超时错误的记录.如何使用laravel和MySql处理这些记录?
I Need experts Suggestions and Solutions. We are developing job portal website here by handle around 1 million records. We are facing records fetching timeout errors. How can I handle those records using laravel and MySql?
我们正在尝试执行以下步骤:
- 增加PHP执行时间
- MySql索引
- 分页
推荐答案
使用大型数据集时，应该对结果进行分块.这允许您处理较小的负载，减少内存消耗，并允许您在其余部分正在获取/处理时将数据返回给用户.请参阅laravel文档中有关分块的信息:
You should be chunking results when working with large data sets. This allows you to process smaller loads, reduces memory consumption and allows you to return data to the User while the rest is being fetched/processing. See the laravel documentation on chunking:
https://laravel.com/docs/5.5/eloquent#chunking-results
要进一步加快处理速度，您可以利用多线程并生成并发进程，每个进程一次处理一个大块. Symfony的Symfony\Component\Process\Process类使此操作变得容易.
To further speed things up you can leverage multithreading and spawn concurrent processes that each handle a chunk at a time. Symfony's Symfony\Component\Process\Process class makes this easy to do.
https://symfony.com/doc/current/components/process.html
这篇关于如何使用Laravel和MySQL获取巨大的记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
