使用Yesod进行高效的大文件上传 [英] Efficient large file upload with Yesod

问题描述

我想用我的Yesod应用程序实现大文件上传。现在我有：

  module Handler.File其中
 
导入
 
导入System.Random 
导入System.FilePath $ b $导入Control.Monad 
导入限定的Data.ByteString.Lazy作为LBS 
导入限定的Data.Text.Encoding 
 
  -  upload 
 $ b uploadDirectory :: FilePath  -  FIXME：使此可配置$ b $ uploadDirectory =传入
 
 randomFileName :: IO FilePath 
 randomFileName = do 
 fname'base<  -  replicateM 20（randomRIO（'a'，'z'））
 let fname = uploadDirectory< /> fname'base<。> bin
 return fname 
 $ b $ fileUploadForm :: Form（FileInfo，Textarea）
 fileUploadForm = renderDivs $（，）
< $> fileAFormReq选择一个文件
< *> areq textareaField文件上有什么？ Nothing 
 
 getFileNewR :: Handler RepHtml 
 getFileNewR = do 
（formWidget，formEnctype）<  -  generateFormPost fileUploadForm 
 defaultLayout $ do 
 setTitleUpload new文件。 
 $（widgetFilefile-new）
 
 postFileNewR :: Handler RepHtml 
 postFileNewR = do 
 user<  -  requireAuth 
（（result， formWidget），formEnctype）<  -  runFormPost fileUploadForm 
 
的结果FormSuccess（fi，info） - > （文件内容类型fi）$ b $ f $<  -  liftIO randomFileName 
 liftIO（LBS.writeFile fn（fileContent fi））
 let newFile = File（entityKey user） $ b fid<  -  runDB $ insert newFile 
 redirect（FileViewR fid）
 _  - > return（）
 
 defaultLayout $ do 
 setTitle上传新文件。 
 $（widgetFilefile-new）
  

：

1. 文件的最大大小大约为2兆字节。我有一个解决方法，但是这样做是正确的吗？我的修复程序覆盖了Yesod实例中maximumContentLength方法的缺省实现，如下所示： 1024 - 2 GB
   maximumContentLength _ _ = 2 * 1024 * 1024 - 2 MB




2. 使用的内存量等于的文件。这实在是不理想。我想从 http://hackage.haskell.org/packages/archive/wai-extra/1.2.0.4/doc/html/Network-Wai-Parse.html ，但我不知道如何将其实际连接到我的请求，并使它与表单逻辑（隐藏的_token字段等）工作。

上传是'单一镜头'：任何如何使它的例子如何使用基于Flash / Html5的上传程序来显示用户的进度？


3. 您的解决方案是正确的。 maximumContentLength设置的目的是允许您为需要更大上传量的特定路由覆盖此值。

这是当前文件设置的一个缺点在yesod-core中处理。目前它使用内存文件上传进行硬编码。我们之前在邮件列表中讨论过这并不理想。我刚刚为此创建了一个 Github问题，修复程序将包含在Yesod 1.1（没有发布时间表）。


4. 我没有这方面的例子，对不起。

I want to implement large file upload with my Yesod application. Right now I have:

module Handler.File where

import Import

import System.Random
import System.FilePath
import Control.Monad
import qualified Data.ByteString.Lazy as LBS
import qualified Data.Text.Encoding

-- upload

uploadDirectory :: FilePath -- FIXME: make this configurable
uploadDirectory = "incoming"

randomFileName :: IO FilePath
randomFileName = do
  fname'base <- replicateM 20 (randomRIO ('a','z'))
  let fname = uploadDirectory </> fname'base <.> "bin"
  return fname

fileUploadForm :: Form (FileInfo, Textarea)
fileUploadForm = renderDivs $ (,)
    <$> fileAFormReq "Choose a file"
    <*> areq textareaField "What's on the file?" Nothing

getFileNewR :: Handler RepHtml
getFileNewR = do
  (formWidget, formEnctype) <- generateFormPost fileUploadForm
  defaultLayout $ do
       setTitle "Upload new file."
       $(widgetFile "file-new")

postFileNewR :: Handler RepHtml
postFileNewR = do
  user <- requireAuth
  ((result, formWidget), formEnctype) <- runFormPost fileUploadForm
  case result of
    FormSuccess (fi,info) -> do
                 fn <- liftIO randomFileName
                 liftIO (LBS.writeFile fn (fileContent fi))
                 let newFile = File (entityKey user) fn info (fileName fi) (fileContentType fi)
                 fid <- runDB $ insert newFile
                 redirect (FileViewR fid)
    _ -> return ()

  defaultLayout $ do
       setTitle "Upload new file."
       $(widgetFile "file-new")

It is mostly fine, except few issues:

1. Maximum size of a file is around 2 megabytes. I have a fix, but is it right way to do this? My fix is overriding default implementation of maximumContentLength method in Yesod instance for my app, like this:

   maximumContentLength _ (Just (FileNewR _)) = 2 * 1024 * 1024 * 1024 -- 2 gigabytes maximumContentLength _ _ = 2 * 1024 * 1024 -- 2 megabytes

2. The amount of memory used is equal to the size of a file. This is really suboptimal. I would like to use tempFileBackEnd from http://hackage.haskell.org/packages/archive/wai-extra/1.2.0.4/doc/html/Network-Wai-Parse.html but I have no idea how to actually wire that into my request and make it work with forms logic (hidden _token field etc.).

3. The upload is 'single shot': any examples of how to make it work with Flash/Html5 based uploaders that show progress to user?

解决方案

1. Your solution is correct. The purpose of the maximumContentLength setting is to allow you to override this value for specific routes that need larger uploads.

2. This is a shortcoming of the current setup of file handling in yesod-core. It's currently hard-coded to use in-memory file uploads. We've discussed on the mailing list in the past that this is suboptimal. I've just created a Github issue for this, and the fix will be included in Yesod 1.1 (no timetable on release though).

3. I don't have an example of this, sorry.

这篇关于使用Yesod进行高效的大文件上传的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！