存储脱机Web应用程序的图像数据（客户端存储数据库） [英] Storing Image Data for offline web application (client-side storage database)

问题描述

我有一个使用appcaching的离线Web应用程序。我需要提供大约10MB - 20MB的数据，它将保存（客户端），主要由PNG图像文件组成。操作如下：

1. Web应用程序在appcache中下载并安装（使用清单）


2. 有时，Web应用会与服务器重新同步，并会对PNG数据库进行小部分更新/删除/添加操作。


3. FYI：服务器是一个JSON REST服务器，可以将文件放入wwwroot中以便取出

这是我的对处理二进制BLOB存储的基于客户端的数据库的当前分析

底部的SEE UPDATE

• AppCache （通过清单添加所有PNG，然后根据需要更新）
  
  
  
  
  • CON： PNG数据库项目将意味着完全下载清单中的所有项目（真的是坏消息！）
  
  


• WebStorage $ CON：为JSON存储而设计


• CON：只能通过base64存储blob编码（可能是由于解码成本造成的致命缺陷）
  
• CON：webStorage的硬性限制为5MB http://htmlui.com/blog/2011-08-23-5-obscure-facts-about-html5-localstorage .html

PhoneGap& SQLLite

• CON：赞助商会拒绝它作为需要认证的原生应用程序
>

• ZIP文件
  
  
  
  
  • 服务器创建一个zip文件，将其放入wwwroot中，并通知客户端
  
  
  • 用户必须手动解压（至少我是这么看的）并保存到客户端文件系统中
    
  • Web应用程序使用FileSystem API来引用文件
  
  
  • CON：ZIP可能太大（zip64？），需要很长时间才能创建
  
  
  • CON：不确定FileSystem API是否总能读出（我认为是这样的）
  
  


• USB或SD卡（回到石器时代... 。）
  
  
  
  
  • 用户在离线前会在服务器本地。
  
  
  • 因此，我们可以让他插入SD卡，让服务器填充PNG文件
  
  
  • 然后用户将其插入笔记本电脑，平板电脑
  
  
  • Web应用程序将使用FileSystem API阅读文件
  
  
  • C ON：不确定FileSystem API是否可以始终读出沙箱（我认为是这样）
  
  


• WebSQL
  
  
  
  
  • CON：w3c放弃了它（非常糟糕）
  
  
  • 我可能会考虑使用IndexedDB和WebSQL的Javascript包装器作为后退
  
  


• FileSystem API
  
  
  
  
  • Chrome支持读/写blob
  
  
  • CON：不清楚IE和FireFox（IE10，具有非标准msSave）
  
  
  • caniuse.com报告IOS和Android支持（但是，这又只是JSON的r / w，还是包括用于编写的完整blob API？
  
  
  • CON：FireFox人不喜欢FileSystem API&不清楚他们是否支持保存blob： https：/ /hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/
  
  
  • PRO：很快根据jsperf的bloed IndexedDB http://jsperf.com/indexeddb-vs-localstorage/15（第2页）
  
  


• IndexedDB
  
  
  
  
  • IE10中的良好支持，FireFox（保存，读取blob）
  
  
  • 比文件系统更快速，更易于管理（删除，更新）
  
  
  • PRO：查看速度测试： http://jsperf.com/indexeddb-vs-localstorage/ 15
  
  
  • 请参阅有关在IndexedDB中存储和显示图像的文章： https：// hac ks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/
  
  
  • CON：我确认Chrome尚不支持blob写入（目前的错误，但不清楚它何时将被修复）
  
  
  • 更新：Chrome开发者确认他们正在为桌面和Android工作！没有时间限制。
  lawnchair /rel =noreferrer> http://brian.io/lawnchair/
  
  
  
  
  • PRO：非常干净的IndexedDB，WebSQL包装器或者你拥有的任何数据库（想想polyfill）
  
  
  • CON：不能存储二进制blob，只能是数据：uri（base64编码）（由于解码成本的原因可能是致命缺陷）
  
  
  
  
  • IndexedDB JQUERY polyFill https://github.com/axemclion/jquery-indexeddb
    
    
    
    
    • Parashuram为原始文件写了一个很好的JQUERY包装器IndexedDB接口
    
    
    • PRO：极大地简化了使用IndexedDB的操作，我希望为Chrome FileSystemAPI添加填充/填充元素。
    
    
    • CON：应该处理斑点，但是我无法让它工作
    
    
  
  
  • idb.filesystem.js http://ericbidelman.tumblr.com/post/21649963613/idb-filesystem- js-bring-the-html5-filesystem-api
    
    
    
    
    • Eric Bidelman @ Google编写了一个经过充分测试的PolyFill FileSystem API，它使用了Indexed DB作为后退
    
    
    • PRO：FileSystem API非常适合存储blob
    
    
    • PRO：适用于FireFox和Chrome
      
      
      
      
      • PRO：非常适合与基于云的CouchDB同步
      
      
    
    
    • CON：没有明确原因，但它不适用于IE10
    
    
  
  
  • PouchDB JavaScript库 http://pouchdb.com/
    
    
    
    
    • 适用于同步本地数据库的CouchDB使用WebSQL或IndexedDB（但不​​是我的问题）
    
    
    • CON：NO缺点，PouchDB现在支持所有最新浏览器的二进制blob（IE，Chrome，Fi refox，手机上的Chrome等）以及许多旧版浏览器。当我第一次发布这篇文章时，情况并非如此。
    
    
  
  
  
  
  

  查看数据：PNG的uri编码我在以下位置创建了一个示例： http://jsbin.com/ivefak/1 / b>
  
  

  期望/有用/不需要的功能
  
  

  
  
  • 客户端（纯Web应用程序）上没有本机（EXE，PhoneGap，ObjectiveC等）应用程序
  
  
  • 只需在最新的Chrome，FireFox，IE10上运行便携式计算机
  
  
  • 非常想要Android平板电脑的相同解决方案（IOS也不错）但只需要一个浏览器就可以工作（FF，Chrome等）
  
  
  • 快速初始数据库人群
  
  
  • 要求：通过Web应用程序从存储（数据库，文件）快速检索图像 不适合消费者。我们可以限制浏览器，并要求用户进行特殊设置&任务，但让我们最小化
  
  
  
  
  

  IndexedDB实施

  
  
  
  
  
  • 有一篇关于IE，FF和Chrome如何在内部实现这个功能的精彩文章： http://www.aaron-powell.com/web/indexeddb-storage
  
  
  • 简而言之：
    
    
    
    
    • IE使用与Exchange和Active Directory相同的数据库格式用于IndexedDB
    
    
    • Firefox正在使用SQLite，所以有点在SQL数据库中实现NoSQL数据库
    
    
    • Chrome（和WebKit）正在使用Key / Value商店，该商店在BigTable中具有传统
    
    
  
  
  
  
  

  我的当前结果

  
  
  
  
  
  • 我选择使用IndexedDB的方法（并使用FileSystemAPI for Chrome对它们进行填充，直到它们提供blob支持）
    
  • 为了获取tile，我有一个困境，因为JQUERY的人们正在为添加这对于AJAX
  
  
  • 我使用Phil Parsons的XHR2-Lib，它非常像JQUERY .ajax（） https://github.com/pmp/xhr2-lib
  
  
  • 100MB下载性能（IE10 4s，Chrome 6s，FireFox 7s ）。
  
  
  • 我无法使用任何IndexedDB包装工具来处理blob（lawnchair，PouchDB，jquery-indexeddb等）。
  
  
  • 自己的包装，性能是（IE10 2s，Chrome 3s，FireFox 10s）使用FF，我假设我们看到使用关系数据库（sqllite） sql storage
  
  
  • 注意，Chrome有一些出色的调试工具（开发人员选项卡，资源），用于检查IndexedDB的状态。
  
  
  
  
  

  最终结果作为回答发布如下
  
  

  更新

  
  
  

  PouchDB现在支持所有最新浏览器（IE，Chrome，Firefox，Chrome on mobile等）以及许多旧版浏览器的二进制blob。当我第一次发布这篇文章时，并不是这种情况。

  解决方案

  结果PNG slippy maps的离线blob缓存

  
  
  

  测试

  
  
  
  
  
  • 171个PNG文件（总计3.2MB）
  
  
  • 测试过的平台：Chrome v24，FireFox 18，IE 10
  
  
  • 还应该使用Chrome&适用于Android的FF
  
  
  
  
  

  从网络服务器提取

  
  
  
  
  
  • 使用XHR2（几乎所有浏览器支持）从服务器进行Blob下载
  
  
  • 我与Phil Parsons的XHR2-Lib一起使用，非常类似JQUERY .ajax（）
    
    
    
    • https://github.com/pmp/xhr2 -lib
    
    
  
  
  
  
  

  存储

  
  
  
  
  
  • 针对IE和FireFox的IndexedDB
  
  
  • Chrome：Polyfill（使用FileSystem API存储的blob， IndexedDB）polyfill
  
  
  • 必须阅读关于浏览器如何存储IndexedDB数据的文章
    
    
    
    • http://www.aaron-powell.com/web/indexeddb-storage
    
    
  li>
  
  • 注意：FireFox使用SQLlite作为NOSQL IndexedDB。这可能是性能缓慢的原因。 （blob单独存储）
  
  
  • 注意：Microsoft IE使用可扩展存储引擎：
    
    
    
    • http://en.wikipedia.org/wiki/Extensible_Storage_Engine
    注意：Chrome使用LevelDB http://code.google.com/p/leveldb/
    
    
    
    

    显示

    
    
    
    
    
    • 传单 http://leafletjs.com/ 显示地图图块
    
    
    • 我使用了功能性图块层Ishmael Smyrnow用于从DB
      
      
      
      • 获取切片图层的插件 https：// github.com/ismyrnow/Leaflet.functionaltilelayer
      
      
      
      • 我将基于DB的图层与纯粹的本地（本地主机：//）存储
      
      
      • 性能没有明显差异！在使用IndexedDB和本地文件之间！
      
      
      
      
      

      结果
      
      

      
      
      • FireFox：Fetch（6.551s），Store（8.247s），总耗用时间：（13.714s） （0.422秒），商店（31.519秒），总计经过时间：（32.836秒）
        
      • IE 10：抓取时间（0.668秒），存储时间（0.896秒） （3.758秒）
        
      
      

      I have an offline web application using appcaching. I need to provide it about 10MB - 20MB of data that it will save (client-side) consisting mainly of PNG image files. The operation is as follows:

      1. Web application downloads and installs in appcache (uses manifest)
      2. Web app requests from server PNG data files (how? - see alternatives below)
      3. Occasionally web app resyncs with server, and does small partial updates/deletes/additions to PNG database
      4. FYI: Server is a JSON REST server, that can place files in wwwroot for pickup

      Here is my current analysis of client-based "databases" that handle binary blob storage

      SEE UPDATE at Bottom

      • AppCache (via manifest add all the PNG and then update on demand)
        • CON: any change of a PNG database item will mean complete download of all items in manifest (Really bad news!)
      • WebStorage
        • CON: Designed for JSON storage
        • CON: can only store blobs via base64 encoding (probably fatal flaw due to cost of de-encoding)
        • CON: Hard limit of 5MB for webStorage http://htmlui.com/blog/2011-08-23-5-obscure-facts-about-html5-localstorage.html
      • PhoneGap & SQLLite
        • CON: Sponsor will reject it as a native app requiring certification
      • ZIP file
        • Server creates a zip file, places it in wwwroot, and notifies client
        • user has to manually unzip (At least that is how I see it) and save to client file system
        • Web app uses FileSystem API to reference files
        • CON: ZIP might be too large (zip64?), long time to create
        • CON: Not sure if FileSystem API can always read out of the sandbox (I think so)
      • USB or SD card (back to the stone age....)
        • The user will be local to the server before going offline
        • So we could have him insert a SD card, let the server fill it with PNG files
        • Then the user will plug it into the laptop, tablet
        • Web app will use FileSystem API to read the files
        • CON: Not sure if FileSystem API can always read out of the sandbox (I think so)
      • WebSQL
        • CON: w3c has abandoned it (pretty bad)
        • I might consider a Javascript wrapper that uses IndexedDB and WebSQL as a fall-back
      • FileSystem API
        • Chrome supports read/write of blobs
        • CON: not clear about IE and FireFox (IE10, has non-standard msSave)
        • caniuse.com reports IOS and Android support (but again, is this just r/w of JSON, or does it include the full blob API for writing?
        • CON: FireFox folks dislike FileSystem API & not clear if they are supporting saving blobs: https://hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/
        • PRO: Much faster than IndexedDB for blobs according to jsperf http://jsperf.com/indexeddb-vs-localstorage/15 (page 2)
      • IndexedDB
        • Good support in IE10, FireFox (save, read blobs)
        • Good speed and easier management than a file system (deletes, updates)
        • PRO: see speed tests: http://jsperf.com/indexeddb-vs-localstorage/15
        • See this article on storing and display of images in IndexedDB: https://hacks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/
        • CON: I confirmed that Chrome does not yet support blob writing (current bug, but not clear when it will be fixed)
        • UPDATE: Chrome developers confirm they are working on this for both desktop and android! no timeline yet.
      • LawnChair JavaScript wrapper http://brian.io/lawnchair/
        • PRO: very clean wrapper for IndexedDB, WebSQL or whatever database you have (think polyfill)
        • CON: cannot store binary blobs, only data:uri (base64 encoding) (probably fatal flaw due to cost of de-encoding)
      • IndexedDB JQUERY polyFill https://github.com/axemclion/jquery-indexeddb
        • Parashuram has writtent a nice JQUERY wrapper for the raw IndexedDB interface
        • PRO: greatly simplifies using IndexedDB, I was hoping to add a shim/polyfill for Chrome FileSystemAPI
        • CON: It should handle blobs, but I was unable to get it to work
      • idb.filesystem.js http://ericbidelman.tumblr.com/post/21649963613/idb-filesystem-js-bringing-the-html5-filesystem-api
        • Eric Bidelman @ Google has written a well tested PolyFill the FileSystem API that uses Indexed DB as a fall back
        • PRO: FileSystem API is well suited for storing blobs
        • PRO: works great on FireFox and Chrome
          • PRO: great for synchronizing with cloud based CouchDB
        • CON: no clear why, but it is not working on IE10
      • PouchDB JavaScript Library http://pouchdb.com/
        • great for syncing a CouchDB with a local DB (uses either WebSQL or IndexedDB (not my problem though)
        • CON: NO CONS, PouchDB now supports binary blobs for all recent browsers (IE, Chrome, Firefox, Chrome on mobile, etc.) as well as many older browsers. That was not the case when I first did this post.

      NOTE: to see a data:uri encoding of PNG I created an example at: http://jsbin.com/ivefak/1/edit

      Desired/Usefull/Uneeded Features

      • No native (EXE, PhoneGap, ObjectiveC, etc) app on client (pure web application)
      • Only needs to run on latest Chrome, FireFox, IE10 for laptops
      • Strongly want same solution for Android Tablet (IOS would be nice too) but only need one browser to work (FF, Chrome, etc.)
      • Fast initial DB population
      • REQUIREMENT: Very fast retrieval of images by web application from storage (DB, file)
      • Not meant for consumers. We can restrict browsers, and ask user to do special setup & tasks, but let's minimize that

      IndexedDB Implementations

      • There is an excellent article on how IE,FF,and Chrome internally implement this at: http://www.aaron-powell.com/web/indexeddb-storage
      • In short:
        • IE uses the same database format as Exchange and Active Directory for IndexedDB
        • Firefox is using SQLite so are kind of implementing a NoSQL database in to SQL database
        • Chrome (and WebKit) are using a Key/ Value store which has heritage in BigTable

      My Current Results

      • I chose to use an IndexedDB approach (and polyfill with FileSystemAPI for Chrome until they ship blob support)
      • For fetching the tiles, I had a dilemna since the JQUERY folks are kvetching about adding this to AJAX
      • I went with XHR2-Lib by Phil Parsons, which is very much like JQUERY .ajax() https://github.com/p-m-p/xhr2-lib
      • Performance for 100MB downloads (IE10 4s, Chrome 6s, FireFox 7s).
      • I could not get any of the IndexedDB wrappers to work for blobs (lawnchair, PouchDB, jquery-indexeddb, etc.)
      • I rolled my own wrapper, and performance is (IE10 2s, Chrome 3s, FireFox 10s)
      • With FF, I assume we are seeing the performance issue of using a relational DB (sqllite) for a non-sql storage
      • NOTE, Chrome has outstanding debug tools (developer tab, resources) for inspecting the state of the IndexedDB.

      FINAL Results posted below as answer

      Update

      PouchDB now supports binary blobs for all recent browsers (IE, Chrome, Firefox, Chrome on mobile, etc.) as well as many older browsers. That was not the case when I first did this post.

      解决方案

      Results Offline blob cache for PNG slippy maps

      Testing

      • 171 PNG files (total of 3.2MB)
      • Platforms tested: Chrome v24, FireFox 18, IE 10
      • Should also work with Chrome & FF for Android

      Fetch from web server

      • using XHR2 (supported on almost all browsers) for blob download from web server
      • I went with XHR2-Lib by Phil Parsons, which is very much like JQUERY .ajax()
        • https://github.com/p-m-p/xhr2-lib

      Storage

      • IndexedDB for IE and FireFox
      • Chrome: Polyfill (blob stored using FileSystem API, reference kept in IndexedDB) polyfill
      • A Must read article on "How the browsers store IndexedDB data"
        • http://www.aaron-powell.com/web/indexeddb-storage
      • Note: FireFox uses SQLlite for the NOSQL IndexedDB. That might be the reason for the slow performance. (blobs stored separately)
      • Note: Microsoft IE uses the extensible storage engine:
        • http://en.wikipedia.org/wiki/Extensible_Storage_Engine
      • Note: Chrome uses LevelDB http://code.google.com/p/leveldb/

      Display

      • I am using Leaflet http://leafletjs.com/ to show the map tiles
      • I used the functional tile layer plugin by Ishmael Smyrnow for fetching the tile layer from the DB
        • https://github.com/ismyrnow/Leaflet.functionaltilelayer
      • I compared the DB-based tiles layer with a purely local (localhost://) storage
      • There is no noticeable difference in performance! between using IndexedDB and local files!

      Results

      • Chrome: Fetch (6.551s), Store (8.247s), Total Elapsed Time: (13.714s)
      • FireFox: Fetch (0.422s), Store (31.519s), Total Elapsed Time: (32.836s)
      • IE 10: Fetch (0.668s), Store: (0.896s), Total Elapsed Time: (3.758s)

      这篇关于存储脱机Web应用程序的图像数据（客户端存储数据库）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！