你如何使用 LinqToSQL/实体框架/NHibernate 实现管道和过滤器模式? [英] How do you implement Pipes and Filters pattern with LinqToSQL/Entity Framework/NHibernate?

问题描述

在使用 DAL Repository 构建时，我偶然发现了一个叫做管道和过滤器的概念.我在这里阅读了相关信息a>，在这里看到了来自这里的截屏视频.我仍然不确定如何实现这种模式.理论上一切听起来都不错，但我们如何在企业场景中真正实现这一点?

如果您在问题中提到的数据映射器/ORM 的上下文中对此模式有任何资源、提示或示例，我将不胜感激.

提前致谢！！

解决方案

最终，IEnumerable 上的 LINQ 是管道和过滤器的实现.IEnumerable<T> 是一个 streaming API - 意味着数据会根据您的要求延迟返回(通过迭代器块)，而不是一次加载所有内容，并返回一个大缓冲区的记录.

这意味着您的查询:

var qry = from row in source//IEnumerable;其中 row.Foo == "abc"选择新的 {row.ID, row.Name};

是:

var qry = source.Where(row => row.Foo == "abc").Select(row = > new {row.ID, row.Name});

当您对此进行枚举时，它将懒惰地消耗数据.您可以使用 Jon Skeet 的 Visual LINQ.唯一能破坏管道的是强制缓冲的东西；OrderBy、GroupBy 等.对于大量工作，Jon 和我自己在 推送 LINQ 以在这种情况下进行聚合而无需缓冲.>

IQueryable(由大多数 ORM 工具公开 - LINQ-to-SQL、实体框架、LINQ-to-NHibernate)是一个稍微不同的野兽；因为数据库引擎将完成大部分繁重的工作，所以大部分步骤可能已经完成 - 剩下的就是使用 IDataReader 并将其投影到对象/值 -但这通常仍然是一个管道(IQueryable 实现 IEnumerable)，除非您调用 .ToArray(), .ToList() 等

关于在企业中使用...我的观点是可以使用 IQueryable 在存储库中编写可组合查询，但它们不应该离开存储库 - 因为这会使存储库受调用者的影响，因此您将无法正确进行单元测试/分析/优化等.我已经在存储库中做了一些聪明的事情，但返回列表/数组.这也意味着我的存储库不知道实现.

这是一种耻辱——因为从存储库方法返回"IQueryable 的诱惑非常大；例如，这将允许调用者添加分页/过滤器/等 - 但请记住，他们尚未实际使用数据.这使得资源管理变得痛苦.此外，在 MVC 等中，您需要确保 controller 调用 .ToList() 或类似方法，以便控制数据访问的不是视图(否则，您将无法正确地对控制器进行单元测试).

安全 (IMO) 在 DAL 中使用过滤器如下:

public Customer[] List(string name, string countryCode) {使用(var ctx = new CustomerDataContext()){IQueryable<客户>qry = ctx.Customers.Where(x=>x.IsOpen);if(!string.IsNullOrEmpty(name)) {qry = qry.Where(cust => cust.Name.Contains(name));}if(!string.IsNullOrEmpty(countryCode)) {qry = qry.Where(cust => cust.CountryCode == countryCode);}返回 qry.ToArray();}}

这里我们即时添加了过滤器，但在调用 ToArray 之前什么也没有发生.此时，获取并返回数据(处理过程中的数据上下文).这可以完全单元测试.如果我们做了类似的事情但只是返回了 IQueryable，调用者可能会做这样的事情:

 var custs = customerRepository.GetCustomers().Where(x=>SomeUnmappedFunction(x));

突然之间，我们的 DAL 开始失败(无法将 SomeUnmappedFunction 转换为 TSQL 等).不过，您仍然可以在存储库中做很多有趣的事情.

这里唯一的痛点是它可能会促使您使用一些重载来支持不同的调用模式(带/不带分页等).在 可选/命名参数到来之前，我找到了最好的这里的答案是在接口上使用扩展方法；这样，我只需要一个具体的存储库实现:

class CustomerRepository {公共客户[]列表(字符串名称，字符串国家代码，内部?页面大小，整数?页码){...}}接口 ICustomerRepository {客户[]列表(字符串名称，字符串国家代码，内部?页面大小，整数?页码);}静态类 CustomerRepositoryExtensions {公共静态客户[]列表(这个 ICustomerRepository 存储库，字符串名称，字符串国家代码){返回 repo.List(name, countryCode, null, null);}}

现在我们在 ICustomerRepository 上有虚拟重载(作为扩展方法) - 所以我们的调用者可以使用 repo.List("abc","def") 而不必指定分页.

<小时>

最后 - 如果没有 LINQ，使用管道和过滤器会变得更加痛苦.您将编写某种基于文本的查询(TSQL、ESQL、HQL).您显然可以附加字符串，但它不是非常管道/过滤器"-ish.Criteria API"要好一些 - 但不如 LINQ 优雅.

While building by DAL Repository, I stumbled upon a concept called Pipes and Filters. I read about it here, here and saw a screencast from here. I am still not sure how to go about implementing this pattern. Theoretically all sounds good , but how do we really implement this in an enterprise scenario?

I will appreciate, if you have any resources,tips or examples ro explanation for this pattern in context to the data mappers/ORM mentioned in the question.

Thanks in advance!!

解决方案

Ultimately, LINQ on IEnumerable<T> is a pipes and filters implementation. IEnumerable<T> is a streaming API - meaning that data is lazily returns as you ask for it (via iterator blocks), rather than loading everything at once, and returning a big buffer of records.

This means that your query:

var qry = from row in source // IEnumerable<T>
          where row.Foo == "abc"
          select new {row.ID, row.Name};

is:

var qry = source.Where(row => row.Foo == "abc")
            .Select(row = > new {row.ID, row.Name});

as you enumerate over this, it will consume the data lazily. You can see this graphically with Jon Skeet's Visual LINQ. The only things that break the pipe are things that force buffering; OrderBy, GroupBy, etc. For high volume work, Jon and myself worked on Push LINQ for doing aggregates without buffering in such scenarios.

IQueryable<T> (exposed by most ORM tools - LINQ-to-SQL, Entity Framework, LINQ-to-NHibernate) is a slightly different beast; because the database engine is going to do most of the heavy lifting, the chances are that most of the steps are already done - all that is left is to consume an IDataReader and project this to objects/values - but that is still typically a pipe (IQueryable<T> implements IEnumerable<T>) unless you call .ToArray(), .ToList() etc.

With regard to use in enterprise... my view is that it is fine to use IQueryable<T> to write composable queries inside the repository, but they shouldn't leave the repository - as that would make the internal operation of the repository subject to the caller, so you would be unable to properly unit test / profile / optimize / etc. I've taken to doing clever things in the repository, but return lists/arrays. This also means my repository stays unaware of the implementation.

This is a shame - as the temptation to "return" IQueryable<T> from a repository method is quite large; for example, this would allow the caller to add paging/filters/etc - but remember that they haven't actually consumed the data yet. This makes resource management a pain. Also, in MVC etc you'd need to ensure that the controller calls .ToList() or similar, so that it isn't the view that is controlling data access (otherwise, again, you can't unit test the controller properly).

A safe (IMO) use of filters in the DAL would be things like:

public Customer[] List(string name, string countryCode) {
     using(var ctx = new CustomerDataContext()) {
         IQueryable<Customer> qry = ctx.Customers.Where(x=>x.IsOpen);
         if(!string.IsNullOrEmpty(name)) {
             qry = qry.Where(cust => cust.Name.Contains(name));
         }
         if(!string.IsNullOrEmpty(countryCode)) {
             qry = qry.Where(cust => cust.CountryCode == countryCode);
         }
         return qry.ToArray();
     }
}

Here we've added filters on-the-fly, but nothing happens until we call ToArray. At this point, the data is obtained and returned (disposing the data-context in the process). This can be fully unit tested. If we did something similar but just returned IQueryable<T>, the caller might do something like:

 var custs = customerRepository.GetCustomers()
       .Where(x=>SomeUnmappedFunction(x));

And all of a sudden our DAL starts failing (cannot translate SomeUnmappedFunction to TSQL, etc). You can still do a lot of interesting things in the repository, though.

The only pain point here is that it might push you to have a few overloads to support different calling patterns (with/without paging, etc). Until optional/named parameters arrives, I find the best answer here is to use extension methods on the interface; that way, I only need one concrete repository implementation:

class CustomerRepository {
    public Customer[] List(
        string name, string countryCode,
        int? pageSize, int? pageNumber) {...}
}
interface ICustomerRepository {
    Customer[] List(
        string name, string countryCode,
        int? pageSize, int? pageNumber);
}
static class CustomerRepositoryExtensions {
    public static Customer[] List(
          this ICustomerRepository repo,
          string name, string countryCode) {
       return repo.List(name, countryCode, null, null); 
    }
}

Now we have virtual overloads (as extension methods) on ICustomerRepository - so our caller can use repo.List("abc","def") without having to specify the paging.

---

Finally - without LINQ, using pipes and filters becomes a lot more painful. You'll be writing some kind of text based query (TSQL, ESQL, HQL). You can obviously append strings, but it isn't very "pipe/filter"-ish. The "Criteria API" is a bit better - but not as elegant as LINQ.

这篇关于你如何使用 LinqToSQL/实体框架/NHibernate 实现管道和过滤器模式?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！