Ruby TempFile在不同类之间的行为 [英] Ruby TempFile behaviour among different classes

问题描述

我们的处理服务器主要与TempFiles一起使用，因为它使我们这边变得更容易:在它们被收集到垃圾或处理名称冲突等情况时，无需照顾删除它们.

Our processing server works mainly with TempFiles as it makes things easier on our side: no need to take care of deleting them as they get garbage collected or handle name collisions, etc.

最近，我们遇到了TempFiles在过程中过早获得GC的问题.特别是通过我们的一项服务，该服务会将Foo文件从url转换为某些Bar文件，并将其上传到我们的服务器.

Lately, we are having problems with TempFiles getting GCed too early in the process. Specially with one of our services that will convert a Foo file from a url to some Bar file and upload it to our servers.

为清楚起见，我在下面添加了一个案例方案，以简化讨论并提供示例.

For sake of clarity I added bellow a case scenario in order to make discussion easier and have an example at hand.

此工作流程执行以下操作:

This workflow does the following:

1. 获取网址作为参数
2. 将Foo文件下载为TempFile
3. 将其复制到新的TempFile
4. 将相关资产下载到TempFiles
5. 将相关资产链接到本地​​dup TempFile
6. 将Foo转换为Bar格式
7. 将其上传到我们的服务器

有时转换失败，并且所有内容都指向一个事实，即我们的本地Foo文件指向转换前已创建和GCed的相关资产.

At times the conversion fail and everything points to the fact that our local Foo file is pointing to related assets that have been created and GCed before the conversion.

我的两个问题:

1. 我的TempFiles是否有可能过早被GC?我读到有关Ruby GCed系统的信息，避免出现这种情况非常保守.

1. Is it possible that my TempFiles get GCed too early? I read about Ruby GCed system it was very conservative to avoid those scenarios.

如何避免这种情况发生?我可以尝试保存download_and_replace_uri(node)中的所有相关资产，并将其作为返回值传递，以在ConvertService实例仍然存在时使其保持活动状态.但是我不确定这是否可以解决.

How can I avoid this from happening? I could try to save all related assets from download_and_replace_uri(node) and passing them as a return to keep it alive while the instance of ConvertService is still existing. But I'm not sure if this would solve it.

myfile.foo

{
  "buffers": [
    { "uri": "http://example.com/any_file.jpg" },
    { "uri": "http://example.com/any_file.png" },
    { "uri": "http://example.com/any_file.jpmp3" }
  ]
}

main.rb

  ConvertService.new('http://example.com/myfile.foo')

ConvertService

class ConvertService
  def initialize(url)
    @url = url
    @bar_file = Tempfile.new
  end

  def call
    import_foo
    convert_foo
    upload_bar
  end

  private

  def import_foo
    @foo_file = ImportService.new(@url).call.edited_file
  end

  def convert_foo
    `create-bar "#{@foo_file.path}" "#{@bar_file.path}"`
  end

  def upload_bar
    UploadBarService.new(@bar_file).call
  end
end

ImportService

class ImportService
  def initialize(url)
    @url = url
    @edited_file ||= Tempfile.new
  end

  def call
    download
    duplicate
    replace
  end

  private

  def download
    @original = DownloadFileService.new(@url).call.file
  end

  def duplicate
    FileUtils.cp(@original.path, @edited_file.path)
  end

  def replace
    file = File.read(@edited_file.path)
    json = JSON.parse(file, symbolize_names: true)
    json[:buffers]&.each do |node| 
      node[:uri] = DownloadFileService.new(node[:uri]).call.file.path
    end
    write_to_disk(@edited_file.path, json.to_json)
  end
end

DownloadFileService

module Helper
  class DownloadFileService < ApplicationHelperService
    def initialize(url)
      @url = url
      @file = Tempfile.new
    end

    def call
      uri = URI.parse(@url)
      Net::HTTP.start(
        uri.host, 
        uri.port, 
        use_ssl: uri.scheme == 'https'
      ) do |http|
        response = http.request(Net::HTTP::Get.new(uri.path))
        @file.binmode
        @file.write(response.body)
        @file.flush
      end
    end
  end
end

UploadBarService

module Helper
  class UploadBarService < ApplicationHelperService
    def initialize(file)
      @file = file
    end

    def call
      HTTParty.post('http://example.com/upload', body: { file: @file })
      # NOTE: End points returns the url for the uploaded file
    end
  end
end

推荐答案

由于代码的复杂性和可能使我们感到困惑的缺失部分，对您的问题的简单答案是确保将tempfile实例对象保留在其中内存需要它们整个生命周期，否则它们将立即被垃圾回收，从文件系统中删除临时文件，并导致丢失所遇到的临时文件状态.

Because of the complexity of your code and missing parts which may be obfuscated to us, the simple answer to your problem is to insure that your tempfile instance objects remain in memory throughout the lifecycle in which they are needed, otherwise they will get garbage collected immediately, removing the tempfile from the file system, and will lead to the the missing tempfile state you've encountered.

用于临时文件状态的Ruby文档当临时文件对象被垃圾回收时，或者当Ruby解释器退出时，其关联的临时文件将被自动删除."

The Ruby Document for Tempfile states "When a Tempfile object is garbage collected, or when the Ruby interpreter exits, its associated temporary file is automatically deleted."

根据评论，其他人可能会发现此对话很有帮助遇到这个问题时.

As per comments, others may find this conversation helpful when running into this problem.

这篇关于Ruby TempFile在不同类之间的行为的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！