将文件传递给活动作业/后台作业 [英] Pass file to Active Job / background job

问题描述

我正在通过标准文件输入接收请求参数中的文件

def create
  file = params[:file]
  upload = Upload.create(file: file, filename: "img.png")
end

但是，对于大型上传，我想在后台执行此操作. 诸如Sidekiq或Resque之类的流行后台作业选项依赖Redis来存储参数，因此我不能只通过Redis传递文件对象.

我可以使用Tempfile，但是在某些平台(例如Heroku)上，本地存储不可靠.

我必须采取什么措施才能使其在任何"平台上可靠?

解决方案

没有临时文件

听起来您想加快图像上传速度或将其推入背景.这是我在其他帖子中的建议.如果您正在寻找这些东西，也许他们会为您提供帮助.

我发现此问题的原因是因为我想保存一个CSV文件，并将后台作业与该文件中的信息一起添加到数据库中.

我有解决办法.

因为您的问题有点不清楚，而且我太懒了，无法发布自己的问题并回答自己的问题，所以我只在此处发布答案.大声笑

与其他帅哥说的一样，将文件保存在某些云存储服务上.对于亚马逊，您需要:

# Gemfile
gem 'aws-sdk', '~> 2.0' # for storing images on AWS S3
gem 'paperclip', '~> 5.0.0' # image processor if you want to use images

您也需要此.在production.rb

中使用相同的代码，但使用不同的存储桶名称

# config/environments/development.rb
Rails.application.configure do
  config.paperclip_defaults = {
    storage: :s3,
    s3_host_name: 's3-us-west-2.amazonaws.com',
    s3_credentials: {
      bucket: 'my-bucket-development',
      s3_region: 'us-west-2',
      access_key_id: ENV['AWS_ACCESS_KEY_ID'],
      secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
    }
  }
end

您还需要迁移

# db/migrate/20000000000000_create_files.rb
class CreateFiles < ActiveRecord::Migration[5.0]
  def change
    create_table :files do |t|
      t.attachment :import_file
    end
  end
end

和模型

class Company < ApplicationRecord
  after_save :start_file_import

  has_attached_file :import_file, default_url: '/missing.png'
  validates_attachment_content_type :import_file, content_type: %r{\Atext\/.*\Z}

  def start_file_import
    return unless import_file_updated_at_changed?
    FileImportJob.perform_later id
  end
end

和一份工作

class FileImportJob < ApplicationJob
  queue_as :default

  def perform(file_id)
    file = File.find file_id
    filepath = file.import_file.url

    # fetch file
    response = HTTParty.get filepath
    # we only need the contents of the response
    csv_text = response.body
    # use the csv gem to create csv table
    csv = CSV.parse csv_text, headers: true
    p "csv class: #{csv.class}" # => "csv class: CSV::Table"
    # loop through each table row and do something with the data
    csv.each_with_index do |row, index|
      if index == 0
        p "row class: #{row.class}" # => "row class: CSV::Row"
        p row.to_hash # hash of all the keys and values from the csv file
      end
    end
  end
end

在您的控制器中

def create
  @file.create file_params
end

def file_params
  params.require(:file).permit(:import_file)
end

I'm receiving a file in a request params through a standard file input

def create
  file = params[:file]
  upload = Upload.create(file: file, filename: "img.png")
end

However, for large uploads, I'd like to do this in a background job. Popular background jobs options like Sidekiq or Resque depend on Redis to store the parameters, so I can't just pass a file object through redis.

I could use a Tempfile, but on some platforms such as Heroku, local storage is not reliable.

What options do I have to make it reliable on "any" platform ?

解决方案

No tempfile

It sounds like you want to either speed up image uploading or push it into background. Here are my suggestions from another post. Maybe they'll help you if that's what you're looking for.

The reason I found this question is because I wanted to save a CSV file and have my background job add to the database with the info in that file.

I have a solution.

Because you the question is a bit unclear and I'm too lazy to post my own question and answer my own question, I'll just post the answer here. lol

Like the other dudes said, save the file on some cloud storage service. For Amazon, you need:

# Gemfile
gem 'aws-sdk', '~> 2.0' # for storing images on AWS S3
gem 'paperclip', '~> 5.0.0' # image processor if you want to use images

You also need this. Use the same code but different bucket name in production.rb

# config/environments/development.rb
Rails.application.configure do
  config.paperclip_defaults = {
    storage: :s3,
    s3_host_name: 's3-us-west-2.amazonaws.com',
    s3_credentials: {
      bucket: 'my-bucket-development',
      s3_region: 'us-west-2',
      access_key_id: ENV['AWS_ACCESS_KEY_ID'],
      secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
    }
  }
end

You also need a migration

# db/migrate/20000000000000_create_files.rb
class CreateFiles < ActiveRecord::Migration[5.0]
  def change
    create_table :files do |t|
      t.attachment :import_file
    end
  end
end

and a model

class Company < ApplicationRecord
  after_save :start_file_import

  has_attached_file :import_file, default_url: '/missing.png'
  validates_attachment_content_type :import_file, content_type: %r{\Atext\/.*\Z}

  def start_file_import
    return unless import_file_updated_at_changed?
    FileImportJob.perform_later id
  end
end

and a job

class FileImportJob < ApplicationJob
  queue_as :default

  def perform(file_id)
    file = File.find file_id
    filepath = file.import_file.url

    # fetch file
    response = HTTParty.get filepath
    # we only need the contents of the response
    csv_text = response.body
    # use the csv gem to create csv table
    csv = CSV.parse csv_text, headers: true
    p "csv class: #{csv.class}" # => "csv class: CSV::Table"
    # loop through each table row and do something with the data
    csv.each_with_index do |row, index|
      if index == 0
        p "row class: #{row.class}" # => "row class: CSV::Row"
        p row.to_hash # hash of all the keys and values from the csv file
      end
    end
  end
end

In your controller

def create
  @file.create file_params
end

def file_params
  params.require(:file).permit(:import_file)
end

这篇关于将文件传递给活动作业/后台作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！