如何在拥有大量用户的 LDAP 服务器上进行分页搜索? [英] How to do a paged search on an Ldap server with lots of users?

问题描述

我需要使用 .NET Core 创建一个在 linux 上运行的 LDAP 客户端.我在互联网上搜索过，唯一支持 .Net Standard 的库是 Novell.Directory.Ldap(开源，iei - https://github.com/dsbenghe/Novell.Directory.Ldap.NETStandard).目录服务库在 .Net Core for Linux 中不支持，仅在 Windows 上支持.

我查看了文档并成功创建了一个基本的 Ldap 客户端应用程序.

现在的问题:我需要同步很多用户(10.000、200.000 个用户)，默认情况下我的 ldap 服务器有一个最大大小的页面 1000(我不想改变它).我使用 VirtualListControl 来创建页面，它适用于 10k 用户.

对于 200k 用户，它崩溃并显示错误 53 - 不愿意执行 LdapSortControl 响应.Novell 库需要一个 LdapSortControl 来执行分页操作(用于索引)，我认为我的 ldap 无法对 200k 进行排序.我使用的代码:

 int startIndex = 1;int contentCount = 0;int afterIndex = 10;整数计数 = 0;做{LdapVirtualListControl ctrl = new LdapVirtualListControl(startIndex, 0, afterIndex, contentCount);LdapSortKey[] keys = new LdapSortKey[1];键[0] = new LdapSortKey("sn");LdapSortControl sort = new LdapSortControl(keys, true);LdapSearchConstraints 约束 = _ldapConnection.SearchConstraints;Constraints.setControls(new LdapControl[] { ctrl, sort});_ldapConnection.Constraints = 约束；LdapSearchResults lsc = _ldapConnection.Search(searchBase, searchScope, searchFilter, attributes, typesOnly, cons);而 (lsc.HasMore()){尝试{LdapEntry nextEntry = lsc.Next();Console.WriteLine(nextEntry.DN);}捕获(LdapException e){Console.WriteLine($"错误:{e.LdapErrorMessage}");//抛出异常，进入下一个条目继续;}}LdapControl[] 控件 = lsc.ResponseControls;如果(控制 == 空){Console.Out.WriteLine("没有返回控件");}别的{foreach(控件中的LdapControl控件){if (control.ID == "2.16.840.1.113730.3.4.10"){LdapVirtualListResponse response = new LdapVirtualListResponse(control.ID, control.Critical, control.getValue());开始索引 += 后索引 + 1;contentCount = response.ContentCount;计数 += 后索引；}}}Console.WriteLine(i);} while (count <= contentCount);

文档很小，没有足够的信息，我不知道如何以更好的方式使用 Novell 库进行分页.这里有没有人使用 Novell Ldap 库并且有分页经验并且可以帮助我?我散了

谢谢

解决方案

要使用 Novell.Directory.Ldap 进行分页查询，您必须使用 LdapVirtualListControl 作为请求"控件.

LdapVirtualListControl 遵守 Ldap Sorting Request Control: VLV (Virtual List View) 的参数:

before:after:index:content_count

其中before"是您想要在索引之前返回的项目数，after"是您想要在索引之后返回的项目数索引和content_count"是服务器中项目的预期总数.如果您不知道，您必须使用 0 作为值.

如果您想通过ldapsearch"cli 返回前 5 个元素，则必须使用:'0:4:1:0' 和 '0:4:5:0' 用于后续五个元素.

LdapVirtualListControl 拥有一个具有相同参数但顺序不同的构造函数:

LdapVirtualListControl(int startIndex, int beforeCount, int afterCount, int contentCount)

就我个人而言，我使用这个函数来正确设置参数:

public static LdapSearchConstraints AddPagination(这个LdapSearchConstraints约束，int页，int pageSize){int startIndex = (page - 1) * pageSize;开始索引++；int beforeCount = 0;int afterCount = pageSize - 1;int contentCount = 0;//0 表示我不知道总数var lvlc = new LdapVirtualListControl(startIndex, beforeCount, afterCount, contentCount);约束.setControls(lvlc);退货限制；}

之后，另一个问题需要你注意:如果你请求一组数据，它位于数据集的末尾，你会收到数据集的第一项.

说明:

ldap 中存在的数据示例:
|1 |2 |3 |4 |
如果我们要套装
________|3 |4 |5 |<- 5 不存在
LDAP 返回:
________|3 |4 |1 |<- 从头开始​​

为了解决这个问题，我在返回之前删除了超出的元素:

var lastIndex = (page * pageSize);if (lastIndex > result.Total){var itemsToReturn = (int) (result.Total - (lastIndex - pageSize));如果 (itemsToReturn <1){items = new List();}别的{items = items.Take(itemsToReturn).ToList();}}

最后，获取总数的函数(在 searchResults.HasMore() 方法之后执行)

protected int?GetTotal(LdapSearchResults searchResult){如果(搜索结果 == 空){throw new ArgumentNullException(nameof(searchResult));}if (searchResult.ResponseControls != null && searchResult.ResponseControls.Any()){foreach(searchResult.ResponseControls 中的 LdapControl 控件){if (control.ID == "2.16.840.1.113730.3.4.10")//响应控件的id{LdapVirtualListResponse 响应 =新的 LdapVirtualListResponse(control.ID, control.Critical, control.getValue());返回 response.ContentCount;}}}返回空；}

您可以获得关于本书的更多见解和信息:LDAP 服务目录和部署>

I need to create a LDAP client that works on linux using .NET Core. I searched trought the internet and the only library that support .Net Standard is Novell.Directory.Ldap (open source, iei - https://github.com/dsbenghe/Novell.Directory.Ldap.NETStandard). Directory Service library does not have support in .Net Core for Linux, only on Windows.

I looked to the documentation and succeded to create a basic Ldap client application.

Now the problem: I need to synchronize a lot of users (10.000, 200.000 users) and by default my ldap server has a maximum size page 1000 (and I don't want to change it). I used a VirtualListControl in order to create pages and it works ok for 10k users.

For 200k user it crashes with error 53 - unwilling to perform on the LdapSortControl response. The Novell library need a LdapSortControl in order to perform paging operation (for index) and I think that my ldap is not able to sort 200k. The code that I used:

        int startIndex = 1;
        int contentCount = 0;
        int afterIndex = 10;
        int count = 0;
        do
        {
            LdapVirtualListControl ctrl = new LdapVirtualListControl(startIndex, 0, afterIndex, contentCount);
            LdapSortKey[] keys = new LdapSortKey[1];
            keys[0] = new LdapSortKey("sn");
            LdapSortControl sort = new LdapSortControl(keys, true);

            LdapSearchConstraints constraints = _ldapConnection.SearchConstraints;
            constraints.setControls(new LdapControl[] { ctrl, sort});

            _ldapConnection.Constraints = constraints;
            LdapSearchResults lsc = _ldapConnection.Search(searchBase, searchScope, searchFilter, attributes, typesOnly, cons);
            while (lsc.HasMore())
            {
                try
                {
                    LdapEntry nextEntry = lsc.Next();
                    Console.WriteLine( nextEntry.DN);
                }
                catch (LdapException e)
                {
                    Console.WriteLine($"Error: {e.LdapErrorMessage}");
                    //Exception is thrown, go for next entry
                    continue;
                }

            }

            LdapControl[] controls = lsc.ResponseControls;
            if (controls == null)
            {
                Console.Out.WriteLine("No controls returned");
            }
            else
            {
                foreach (LdapControl control in controls)
                {
                    if (control.ID == "2.16.840.1.113730.3.4.10")
                    {
                        LdapVirtualListResponse response = new LdapVirtualListResponse(control.ID, control.Critical, control.getValue());
                        startIndex += afterIndex + 1;
                        contentCount = response.ContentCount;
                        count += afterIndex;
                    }
                }
            }
            Console.WriteLine(i);

        } while (count <= contentCount);

The documentation is small and does not have enough information and I have no idea how to use Novell library for paging in a better way. Is anybody here that use Novell Ldap library and have any experience with paging and also can help me? I'm disperate

Thank you

解决方案

To make a paged query with Novell.Directory.Ldap you have to use LdapVirtualListControl as "Request" Control.

LdapVirtualListControl respects the arguments of the Ldap Sorting Request Control: VLV (Virtual List View) that are:

before:after:index:content_count

where "before" is the number of items that you want to be returned BEFORE the index, "after" the number of items that you want to be returned AFTER the index and "content_count" is the expected total count of items in the server. In case you do not know it, you have to use 0 as value.

If you want to return the first 5 elements via "ldapsearch" cli, you have to use: '0:4:1:0' and '0:4:5:0' for the subsequent five ones.

The LdapVirtualListControl owns a constructor with the same arguments but with a different order:

LdapVirtualListControl(int startIndex, int beforeCount, int afterCount, int contentCount)

Personally, I use this function to set the arguments correctly:

public static LdapSearchConstraints AddPagination(this LdapSearchConstraints constraints, int page,
            int pageSize)
{
    int startIndex = (page - 1) * pageSize;
    startIndex++;
    int beforeCount = 0;
    int afterCount = pageSize - 1;
    int contentCount = 0; //0 means that i don't know the total count

    var lvlc = new LdapVirtualListControl(startIndex, beforeCount, afterCount, contentCount);
    constraints.setControls(lvlc);
    return constraints;
}

After that, another issue needs your attention: if you ask for a set of data, which are positioned after the end of the data-set, you will receive back the first items of the dataset.

Explanation:

Example of data present in ldap:
| 1 | 2 | 3 | 4 |
if we ask for the set
________| 3 | 4 | 5 | <- 5 does not exists
Ldap returns:
________| 3 | 4 | 1 | <- it starts over from the beginning

To fix this issue, I remove the exceeded elements before return:

var lastIndex = (page * pageSize);
if (lastIndex > result.Total)
{
    var itemsToReturn = (int) (result.Total - (lastIndex - pageSize));
    if (itemsToReturn < 1)
    {
        items = new List<LdapQueryItem>();
    }
    else
    {
        items = items.Take(itemsToReturn).ToList();
    }
}   

Finally, the function to get the totals (to execute AFTER the searchResults.HasMore() method)

protected int? GetTotal(LdapSearchResults searchResult)
{
    if (searchResult == null) {
       throw new ArgumentNullException(nameof(searchResult));
    }
    if (searchResult.ResponseControls != null && searchResult.ResponseControls.Any())
    {
        foreach (LdapControl control in searchResult.ResponseControls)
        {
            if (control.ID == "2.16.840.1.113730.3.4.10") // the id of the response control
            {
                LdapVirtualListResponse response =
                    new LdapVirtualListResponse(control.ID, control.Critical, control.getValue());
                return response.ContentCount;
            }
        }
    }

    return null;
}

You can get further insights and info on this book: Understanding and Deploying LDAP Directory Services

这篇关于如何在拥有大量用户的 LDAP 服务器上进行分页搜索?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！