'safe'json_decode(,,,)以防止耗尽内存 [英] 'safe' json_decode( ,,, ) to prevent exhausting memory
问题描述
在我的应用中，我经常调用一个返回json字符串的外部api.
In my app I often call an external api that returns an json string.
$url = 'api.example.com/xyz';
$blah = json_decode( file_get_contents( $url ) );
但是在某些情况下我会得到
But in some cases I get
PHP致命错误:在...中耗尽了xxx个字节的内存大小(尝试分配32个字节).
PHP Fatal error: Allowed memory size of xxx bytes exhausted (tried to allocate 32 bytes) in ...
我无法控制外部API，当然可以增加php的内存，但这有一些缺点.
I cannot control the external API, and of course I could increase the memory for php, but that has some drawbacks.
1-无论我设置什么尺寸，都可能仍然太小. 2-如果将内存大小设置为"infinite"(无限)，则可能会导致服务器被杀.
1- Whatever size I set, could still be too little. 2- If I set the memory size to 'infinite' then I could run the risk of killing my server.
理想情况下，在调用json_decode(...)之前，我想先检查一下该字符串是否导致内存耗尽.
Ideally I would like to 'check' before I call json_decode( ... ) that the string result into a memory exhaustion.
有可能吗?
推荐答案
如果它们设法耗尽服务器的内存，则必须获得大量的JSON响应.以下是一些指标，这些指标包含一个1 MB的文件，其中包含一个多维关联的数组(包含准备输入到具有不同数据类型的三个MySQL表中的数据).
You must be getting some massive JSON responses if they manage to exhaust your server's memory. Here are some metrics with a 1 MB file containing a multidimensional associated array (containing data prepared for entry into three MySQL tables with diverse data-types).
当我include并且文件作为数组加载到内存中时，我的内存使用量达到9 MB.如果我通过file_get_contents()获取原始数据，则它将占用预期的1 MB内存.然后，PHP数组与数据strlen()的比率约为1:9(最初以var_export()输出).
When I include and the file is loaded into memory as an array, my memory usage goes to 9 MB. If I get the raw data with file_get_contents(), it takes 1 MB memory as expected. Then, a PHP array has an approximate ratio of 1:9 to the strlen() of the data (originally output with var_export()).
当我运行json_encode()时，峰值内存使用量不会增加. (PHP在块中分配内存，因此通常会产生一些开销，在这种情况下，足够包含JSON的字符串数据；但是它可能会使您增加一个块.)作为字符串的结果JSON数据需要670 KB.
When I run json_encode(), peak memory usage doesn't increase. (PHP allocates memory in blocks so there's often a bit of overhead, in this case enough to include the string data of the JSON; but it could bump you up one block more.) The resulting JSON data as a string takes 670 KB.
当我使用file_get_contents将JSON数据加载到字符串中时，它需要0.75 MB的内存.当我在其上运行json_decode()时，它将占用7 MB的内存.然后，对于要解码为 JSON-data-bytesize 到 PHP本地对象数组 的RAM要求，我将最小比例为1:10.
When I load the JSON data with file_get_contents into a string, it takes an expected 0.75 MB of memory. When I run json_decode() on it, it takes 7 MB of memory. I would then factor a minimum ratio of 1:10 for JSON-data-bytesize decoded to native PHP array-or-object for RAM requirement.
要在解码之前对JSON数据进行测试，则可以执行以下操作:
To run a test on your JSON data before decoding it, you could then do something like this:
if (strlen($my_json) * 10 > ($my_mb_memory * 1024 * 1024)) {
die ('Decoding this would exhaust the server memory. Sorry!');
}
...，其中$my_json是原始JSON响应，而$my_mb_memory是已分配的RAM，已将其转换为字节以与传入数据进行比较. (当然，您也可以使用intval(ini_get('memory_limit'))来获取整数的内存限制.)
...where $my_json is the raw JSON response, and $my_mb_memory is your allocated RAM that's converted into bytes for comparison with the incoming data. (You can of course also use intval(ini_get('memory_limit')) to get your memory limit as an integer.)
如下所述，RAM的使用也将取决于您的数据结构.相比之下，由于我很好奇自己，所以又有一些快速的测试用例:
As pointed out below, the RAM usage will also depend on your data structure. For contrast, a few more quick test cases because I'm curious myself:
- 如果我创建一个整数为1-60000的一维数组，则保存的PHP数组大小为1 MB，但是RAM的最大使用量在10.5和12.5 MB之间(好奇振荡)，或比例为1:12-ish .
- 如果我创建一个1 MB文件的值的数据作为12000个随机字符串作为基本关联数组，则加载后的内存使用量仅为5 MB；比例为1:5.
- 如果我创建一个价值1 MB的文件作为相似的关联数组，其中一半条目是带有数字索引的字符串数组，则内存使用率为7 MB，比例为1:7.
因此，您的实际RAM里程可能会相差很大.另外请注意，如果您将大量数据循环传递并做了一些处理，那您的内存使用量可能会增加很多(或成倍地取决于您的代码经济性)高于json_decode()单独导致的后果.
So your actual RAM mileage may vary a good deal. Also be aware that if you pass that bulk of data around in circles and do a bit of this and that, your memory usage may get much (or exponentially, depending on your code economy) higher than what json_decode() alone will cause.
要调试内存使用情况，可以在代码中的主要间隔使用memory_get_usage()和/或memory_get_peak_usage()来记录或输出代码不同部分中使用的内存.
To debug memory usage, you can use memory_get_usage() and/or memory_get_peak_usage() at major intervals in your code to log or output the memory used in different parts of your code.
这篇关于'safe'json_decode(,,,)以防止耗尽内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
