将noSQL和ORM结合在一个实例应用程序的MVC框架中 [英] Combining noSQL and ORM in an MVC framework for a real-case application
问题描述
在过去的几年里，我一直在尝试一些关于noSQL（couchDB，mongoDB，Redis ...）阅读的酷事物。
我很习惯用Django编写应用程序，并开始使用Play！当Java是唯一可接受的部署选项（并享受它）。两个模块都可以使用，例如， MongoDB ，django也有 nonrel 。但是我从来没有觉得需要noSQL。
直到我终于找到我以为是面向文档的存储的一个很好的用例，如MongoDB。
用例
假设我们必须管理一些复杂项目的排序和跟进（无论如何）。这些项目可能有很多不同的属性，例如。过度简化我们可以有：
- 可以有
- 一个或两个门，
- 为A，B或C类，
- 表面颜色
- 独立或内置
- 一个可以具有以下功能的烤箱：
- 天然气或电力两者
- 自洁或
- 独立或内置
SQL / ORM解决方案
正如你所看到的，每个对象有几个属性可以被类型限制。
在我通常的RDBMS通过ORM我会去定义一个产品模型，然后继承两个模型，一个冰箱和烤箱。
如果冰箱在一段时间之后再获得一个属性，那么我修改模型，因此，模式，运行迁移，并添加一列。
< h2> noSQL解决方案
noSQL解决方案我可以想到的是：
- 使用RDF（如 Virtuoso 或构建我自己的简化三倍体存储），
- 使用面向文档的数据库，如MongoDB
问题
但是我不明白如何以实际的方式切换到使用正确的适配器（特别是DODB）的框架ORM的noSQL解决方案。 / p> 假设我通过mongodb引擎在MongoDB中使用Django。 我仍然使用相同的ORM，所以我仍然将这些对象描述为模型，列出所有属性。
ORM因此做的完全一样！
如果模型发生变化，生产迁移的成本非常有限，因为ORM（特别是像南方这样的东西），没有什么需要自己学习一项新技术。
$ b $对于DODB可能有其他/优点，而某些特定于MongoDB（可扩展性，数据处理，可能是性能），但是...我正在描述的确切用例和问题呢？
我很有可能缺少一点，所以这里有真正的问题：
对于这个具体的用例：
- 这个例子是DODB的好或坏的（你有没有好的）？
将复杂对象的基本东西（用户，订单）和noSQL ORM的ORM组合在一起的ORM是有意义的，是否有完全切换到noSQL的令人信服的理由，还是应该留在股票ORM / SQL？
我明白回答这些问题可能是部分主观的，所以你可以假设p对noSQL和SQL理论的知识不完全，以及股票ORM;存在从股票ORM到noSQL DB的良好桥梁。我们假设我们正在谈论MongoDB的这个用例，作为noSQL的替代方案。
但是有一个更一般的问题 - 这是SO文章的核心问题： p>
- 不是一个很好的ORM（例如JPA，ActiveRecord或Django的），使noSQL，特别是面向文档的数据库很少使用？
- ...是否值得使用noSQL与经典ORM？
（从编程和维护的角度来看，很少使用，性能和类似标准是一个不同的问题，需要精确的产品与产品的比较）
我也想了解的是，如果不是最好使用ORM放弃切换到noSQL。拥有更多动态模型是很好的，例如。我可以有一个描述冰箱和烤箱模型是什么（表格）的表格，代码中的冰箱和烤箱模型将能够动态构建视图（用于编辑和列表显示的窗体）
相关问题：
：这些都在这里显示我的研究，但也澄清，我所要求的不是关于noSQL与SQL的通用。
- 使用NoSQL API进行ORM冗余？：类似！但是我想看看为什么大多数框架（见上文）通过其ORM提供noSQL访问。这是个好主意吗？
- 为什么使用ORM与NoSql（如MongoDB）：比前一个更具体，但仍然是另一回事。我争辩说， ORM使得noSQL无法使用，而不是其他方式。
- 何时用NoSQL替换RDBMS / ORM
- 何时使用MongoDB或其他面向文档的数据库系统？
- https：// softwareengineering。 stackexchange.com/questions/54373/when-would-someone-use-mongodb-or-similar-over-traditional-rdms
编辑
和链接：
- 锡耶纳：Java的持久化API，灵感来源于Google App Engine Python Datastore，试图绘制SQL和NoSQL世界之间的桥梁。
- minimongo ：轻量级，图形化，面向对象的面向MongoDB的接口
这是我为拖动堆栈流。一旦遇到一个悬而未决的问题，我被迫提供2美分（有自己的项目时间表的风险）。
我刚刚完成了一个项目，我不得不从模型中取消ORM，所以我可以实现一个NoSQL解决方案，发现并不困难，虽然它有时试图找出最好的方法。所以没有太具体地说明我的执行情况，我将会谈到我必须做些什么来使之发挥作用，因为当你沿着同样的道路行进时，可能会提供一些启示。
我的设置：
- 框架 - Symfony 1.4
- ORM - Doctrine 1.4
- NoSQL - 我自己的专有解决方案
目标：
- 存储xml文件中的图像路径与数据库
- 将xml文件中的html描述路径存储到数据库
我不想将图像作为blob存储在持久存储（数据库）中，我不想将图像路径存储在数据库中，因为我没有想要支付创建数据库连接和查询路径的开销。所以我决定将路径信息存储在NoSQL永久存储（文件系统）中。
同样的html描述，我不想在我的表和存储数据库中可能有数百行html的内容，以及与上述相同的原因。
我所有的NoSQL文件都与一个对象（冰箱） ）。这些文件包含其相关资产的路径（html描述和图像），我称之为指向文件系统上的资源的指针。我选择使用XML格式存储数据，看起来像这样：
//指向文件的路径
/home/files/app/needle/myApp/refrigerator/1/1.xml
//示例指针
< pointer> / home / files / app / file / myApp /冰箱/ 1.png< /指针>
现在，在框架内我不得不重写save（）方法，所以我可以保存上述资产使用NoSQL API。这很简单，我只是检查了父调用并维护了进入方法的值，所以他们不会破坏任何链逻辑（使用相同参数调用其他方法的方法），我不知道。我还使我的自定义NoSQL API调用抛出异常，因为主要的save（）调用被包装在一个try / catch块中。您唯一需要注意的是确定您的NoSQL资产是否值得停止整个交易。在我的例子中，我必须弄清楚，如果上传图像会破坏保存数据库中其余的表单域（我选择中断交易）。
I还必须使用NoSQL API和标准模型逻辑来更改load（）方法来检索资产。与保存方法一样，这也不是太难。我只需要看看父类正在做什么，而不是用任何参数值进行处理。
当所有的描述和完成，我能够存储图像和html描述在文件系统上，使用由指向其位置的指针组成的xml文件。所以现在我每次需要一个资产时都不会发出数据库调用。
有些注意事项（这些可能包含在其他NoSQL解决方案中，我必须自己写）：
- 您将无法查询持久存储的图像的冰箱。您将不得不在您的应用程序中写入一些逻辑，以从NoSQL存储中提取资产。
- 备份：在备份持久存储数据时，还需要备份NoSQL数据。
- 孤儿：现在，您的模式不知道您可能拥有的任何资源，从永久存储中删除一行将孤立文件系统上的资产。所以请确保您的应用程序具有在删除行时清除NoSQL存储的逻辑。
我想我打了所有主要
- 编辑 -
回应评论：
-
正如我所说，我不想创建数据库连接和查询只是为了获得资产的路径。对于这种类型的信息，我觉得最好使用NoSQL解决方案，因为真的没有理由对这种类型的信息（图像或html描述）进行查询。
- <开发我自己的NoSQL解决方案更是一个自我挑战。在工作中有一个项目实施一个自定义NoSQL解决方案（与MogileFS有不好的经验），坦率地说，设计不善，执行不力。但是，而不仅仅是指出坏处，我挑战自己提供一个更好的解决方案，但是对于一个侧面项目。而且由于挑战方面，我没有研究任何已经可用的NoSQL解决方案，但事后我可能应该有。
我仍然认为您可以通过使用ORM的Model层覆盖原始函数来实现MongoDB或任何NoSQL解决方案，比较容易事实上，我不仅实现了我的NoSQL解决方案，还添加了在crud函数中将数据索引到SOLR（用于全文搜索）的能力，所以有可能。
I've been trying for some time to put some 'cool' things I've been reading about noSQL (couchDB, mongoDB, Redis...) in the past years into practical use.
I'm quite used to writing apps with Django, and started using Play! when Java is the only acceptable deployment option (and enjoying it too). Both have modules to work, for instance, with MongoDB, and django also has nonrel. But i never felt a need for noSQL.
Until I finally found what I thought was a good use case for document-oriented storage, such as MongoDB.
Use Case
Let's say we have to manage ordering and follow up (whatever) of some complex items. The items might have a lot of different properties, eg. oversimplifying we could have:
- a Fridge that can have
- one or two doors,
- be of class A,B or C,
- a surface color
- standalone or built-in
- an Oven that can have:
- gas or electricity or both
- self cleaning or not
- standalone or built-in
SQL/ORM solution
As you can see, each object can have several properties that can be constrained by type.
In my usual RDBMS through an ORM I would go with defining a "product" model, then inherit two models, a Fridge and an Oven. If a Fridge gets one more property some time after, I modify the model - and as such, the schema-, run a migration, and add a column.
noSQL solutions
noSQL solutions I can think of are:
- using RDF (with something like Virtuoso or building my own simplified triples storage)
- using a document-oriented db such as MongoDB
The Problem
But I fail at understanding how different (easier) development would pragmatically be switching to a noSQL solution still using the framework ORM with the right adapter (especially with DODB).
Let's say I'm using Django with MongoDB through mongodb-engine.
I'm still using the same ORM, so I still describe those objects as models, listing all properties. The ORM is thus doing exactly the same job! The cost of producing a migration if the model changes is very limited with an ORM (in particular with something like South), nothing requiring to learn a new technology by itself.
There might be /other/ advantages to a DODB, and some specific to MongoDB (scalability, data-processing, maybe performance) but... what about the exact use case and problem I'm describing?
I am most likely missing a point, so here come the real question(s):
For this specific use case:
- is this example a good or bad one for DODB (do you have a good one)?
- would it make sense to combine an ORM for basic stuff (Users, Orders) and usage of noSQL without ORM for complex objects, is there a compelling reason to switch to noSQL completely, or should I stay with the stock ORM/SQL?
I understand answering these questions could be partially subjective, so you can assume perfect knowledge of both noSQL and SQL theory, and stock ORM; existence of good bridges from stock ORMs to noSQL DB. Let's assume we are talking about this use case with MongoDB as noSQL alternative.
But there is a more general question - which is the core question of this SO post:
- Isn't a good ORM (such as JPA, ActiveRecord, or Django's one) making noSQL and in particular document-oriented databases of little use?
- ...and is it worth using noSQL with a 'classic' ORM?
("little use" from a programming and maintenance standpoint, performance and similar criteria are a different matter and would require a precise product-to-product comparison)
[edit]
What I am also trying to understand is if it wouldn't be better to drop using an ORM when switching to noSQL. It would be nice to have more "dynamic" models, eg. I could have a table describing what the Fridge and Oven models are (fields), and the Fridge and Oven models in the code would be able to construct their views dynamically (forms for editing and listings for displaying).
Related questions:
[edit]: these are here to show my research, but also to clarify that what I am asking is not generic about noSQL vs. SQL
- Is an ORM redundant with a NoSQL API? : similar! but I am trying to see why most frameworks (see above) are providing noSQL access through their ORMs. Is that a good idea or not?
- why the use of an ORM with NoSql (like MongoDB) : more specific than the previous one, but still is the other way around. I'm arguing that ORMs make noSQL unuseful, rather than the other way around!
- When to replace RDBMS/ORM with NoSQL
- When to use MongoDB or other document oriented database systems?
- https://softwareengineering.stackexchange.com/questions/54373/when-would-someone-use-mongodb-or-similar-over-traditional-rdms
EDIT And links:
- Siena: a persistence API for Java inspired by the Google App Engine Python Datastore trying to draw a bridge between SQL and NoSQL worlds.
- minimongo: lightweight, schemaless, Pythonic Object-Oriented interface to MongoDB
This is what I get for trolling stackoverflow. Once in a great while an outstanding question gets asked and I am compelled to offer my 2 cents (at the risk of my own project timelines).
I just finished up a project where I had to de-couple an ORM from the model so I could implement a NoSQL solution, and found it not that difficult, although it was rough at times trying to figure out the best approach. So without getting too specific about my implementation, I will touch on what I had to do to make it work, as it may offer some enlightenment when you travel down the same path.
My setup:
- Framework - Symfony 1.4
- ORM - Doctrine 1.4
- NoSQL - My own proprietary solution
Goal:
- Store image paths within xml files vs the database
- Store html description paths within xml files vs the database
I didn't want to store images as blobs within the persistent store (database), and I didn't want to store image paths in the database, as I didn't want to pay the overhead of creating a database connection and querying for the path. So I decided to store the path information within a NoSQL persistent store (filesystem).
And ditto for html descriptions, I didn't want to create a text column on my table and store what could potentially be hundreds of lines of html within the database, and the same reasons as above.
All my NoSQL files relate to an object (refrigerator for example). These files contain paths to their related assets (html description and images), in what I call pointers, which point to the assets on the filesystem. I opted to use XML format for storing the data so it looks something like this:
// Path to pointer file
/home/files/app/needle/myApp/refrigerator/1/1.xml
// Example pointer
<pointer>/home/files/app/file/myApp/refrigerator/1.png</pointer>
Now, within the framework I had to override the save() methods so I could save the aforementioned assets using the NoSQL API. It was pretty easy, I just checked the parent calls and maintained the values coming into the methods, so they wouldn't break any chain logic (methods calling other methods with the same arguments), that I wasn't aware of. I also made my custom NoSQL API calls throw exceptions as the main save() call was wrapped in a try/catch block. The only thing you have to be careful of here, is determining whether your NoSQL assets are worth stopping the entire transaction. In my example, I had to figure out if uploading an image would break saving the rest of the form fields in the database (I opted to break the transaction).
I also had to alter the load() methods to retrieve the assets using the NoSQL API vs the standard model logic. As with the save methods, this wasn't too hard to do either. I just had to see what the parent classes were doing and not muck with any argument values.
When all was said and done, I was able to store images and html descriptions on the filesystem, with an xml file made up of pointers pointing to their location. So now I don't incur a database call every time I need an asset.
Some considerations (these may be included in other NoSQL solutions, I had to write my own):
- You will not be able to query for refrigerators that have images from your persistent store. You will have to write some logic in your application to pull in the assets from the NoSQL store.
- Backups: As you backup your persistent store data, you also need to backup your NoSQL data.
- Orphans: Now that your schema is unaware of any assets you may have, deleting a row from your persistent store will orphan an asset on the filesystem. So be sure your application has the logic to clean the NoSQL store when a row has been deleted.
I think I hit all the major hurdles I faced when implementing a NoSQL solution with an ORM, if you have any other questions feel free to hit me up.
-- Edit --
Responses to comments:
As I mentioned I didn't want to create a database connection and query just to get a path to an asset. I feel it's better to use a NoSQL solution for this type of information as there is really no reason run queries against this type of information (images or html descriptions).
Developing my own NoSQL solution was more of an ego challenge. At work there was a project to implement a custom NoSQL solution (had bad experiences with MogileFS), and to be frank, was poorly designed and poorly implemented. But rather than just point out the bad, I challenged myself to offer up a better solution, but for a side project. And because of the challenge aspect, I didn't research any already available NoSQL solutions, but in hindsight I probably should have.
I still think you can implement MongoDB or any NoSQL solution by overriding crud functions with the Model layer of your ORM, relatively easy. In fact, not only did I implement my NoSQL solution, I also added the ability to index data into SOLR (for full-text searching) during crud functions as well, so anything is possible.
这篇关于将noSQL和ORM结合在一个实例应用程序的MVC框架中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
