从数据库游标缓冲数据，同时保持 UI 响应 [英] Buffer data from database cursor while keeping UI responsive

问题描述

我有一个已填充的数据库目录和一个可用于检索对象的游标.这个目录显然可以非常大，我想要做的是使用 ReactiveUI 来缓冲数据，同时保持 UI 数据绑定和响应.我按照此处的步骤操作将我的 IEnumerable 转换为 IObservable，如下所示:

I have a database catalog that is populated, and a cursor that can be used to retrieve objects. This catalog can obviously be very large, and what I'd like to do is use ReactiveUI to buffer the data in, while keeping the UI data-bound and responsive. I followed the steps here to translate my IEnumerable into an IObservable, as shown here:

public class CatalogService
{
   ...

   public IObservable<DbObject> DataSource
   {
        get
        {
            return Observable.Create<DbObject>(obs =>
            {
                var cursor = Database.Instance.GetAllObjects();
                var status = cursor.MoveToFirst();

                while (status == DbStatus.OK)
                {
                    var dbObject= Db.Create(cursor);
                    obs.OnNext(dbObject);

                    status = cursor.MoveToNext();
                }

                obs.OnCompleted();

                return Disposable.Empty;
            });
        }
    }
}

在我的视图类(特别是 Loaded 事件)中，我订阅了数据源并使用缓冲区方法，希望保持 UI 响应.

In my view class (specifically, the Loaded event), I am subscribing to the data-source and using the buffer method in hopes of keeping the UI responsive.

    public ObservableCollection<DbObject> DbObjects { get; set; }

    private async void OnLoad(object sender, RoutedEventArgs e)
    {
        var observableData = CatalogService.Instance.DataSource.Publish();
        var chunked = observableData.Buffer(TimeSpan.FromMilliseconds(100));
        var dispatcherObs = chunked.ObserveOnDispatcher(DispatcherPriority.Background);
        dispatcherObs.Subscribe(dbObjects =>
        {
            foreach (var dbObject in dbObjects)
            {
                DbObjects.Add(dbObject);
            }
        });

        await Task.Run(() => observableData.Connect());
        await dispatcherObs.ToTask();
    }

不幸的是，结果恰恰相反.当我的视图控件(它包含一个简单的 ListBox 数据绑定到 DbObjects 属性)加载时，它不会显示任何数据，直到枚举整个目录.只有这样，UI 才会刷新.

The result is unfortunately quite the opposite. When my view control (which contains a simple ListBox data-bound to the DbObjects property) loads, it does not show any data until the entire catalog has been enumerated. Only then does the UI refresh.

我是 ReactiveUI 的新手，但我确信它能够完成手头的任务.如果我使用不当，有没有人有任何建议或指示?

I am new to ReactiveUI, but I am sure that it is capable for the task at hand. Does anyone have any suggestions or pointers if I am using it incorrectly?

推荐答案

等待进一步信息，我的猜测是您可能有几个零长度的缓冲区，具体取决于数据库查询花费的时间，后面紧跟一个非零的缓冲区包含所有结果的长度缓冲区.您最好通过长度和时间来限制缓冲区大小.

Pending further information, my guess is you've got possibly several zero-length buffers depending on how long the DB query takes, followed by exactly one non-zero length buffer containing all the results. You're probably better off limiting buffer size by length as well as time.

编辑 - 我只是想对原始实现中涉及的各种线程进行分析.我不同意 Paul 的分析，我不相信 UI 线程因为 DB 查询而被阻塞.我相信它是由于缓冲了大量结果而被阻止的.

EDIT - I just wanted to do an analysis of the various threads involved in the original implementation. I don't agree with Paul's analysis, I do not believe the UI Thread is blocked because of the DB query. I believe it's blocked due to large numbers of results being buffered.

查理 - 拜托，你能不能用代码(而不是调试器)对数据库查询进行计时并转储你得到的缓冲区长度.

Charlie - please, can you time the DB query in code (not with the debugger) and dump the buffer lengths you are getting too.

我将注释代码以显示所涉及的所有三个线程的顺序:

I will annotate the code to show the order of all three threads involved:

首先，在提供的代码之外，我假设通过 Loaded 事件调用了 OnLoad.

First of all, outside of the provided code, I am assuming a call is made to OnLoad via the Loaded event.

(1) - UI 线程调用 OnLoad

(1) - UI Thread calls OnLoad

public ObservableCollection<DbObject> DbObjects { get; set; }

private async void OnLoad(object sender, RoutedEventArgs e)
{
    // (2) UI Thread enters OnLoad

    var observableData = CatalogService.Instance.DataSource.Publish();

    var chunked = observableData
        // (6) Thread A OnNext passes into Buffer
        .Buffer(TimeSpan.FromMilliseconds(100));
        // (7) Thread B, threadpool thread used by Buffer to run timer 

    var dispatcherObs = chunked
        // (8) Thread B still
        .ObserveOnDispatcher(DispatcherPriority.Background);
        // (9) Non blocking OnNexts back to UI Thread

    dispatcherObs.Subscribe(dbObjects =>
    {
        // (10) UI Thread receives buffered dbObjects            
        foreach (var dbObject in dbObjects)
        {
            // (11) UI Thread hurting while all these images are
            // stuffed in the collection in one go - This is the issue I bet.
            DbObjects.Add(dbObject);
        }
    });

    await Task.Run(() =>
    {
        // (3) Thread A - a threadpool thread,
        // triggers subscription to DataSource
        // UI Thread is *NOT BLOCKED* due to await
        observableData.Connect()
    });
    // (13) UI Thread - Dispatcher call back here at end of Create call
    // BUT UI THREAD WAS NOT BLOCKED!!!

    // (14) UI Thread - This task will be already completed
    // It is causing a second subscription to the already completed published observable
    await dispatcherObs.ToTask();


}

public class CatalogService
{
   ...

   public IObservable<DbObject> DataSource
   {
        get
        {
            return Observable.Create<DbObject>(obs =>
            {
                // (4) Thread A runs Database query synchronously
                var cursor = Database.Instance.GetAllObjects();
                var status = cursor.MoveToFirst();

                while (status == DbStatus.OK)
                {
                    var dbObject= Db.Create(cursor);
                    // (5) Thread A call OnNext
                    obs.OnNext(dbObject);

                    status = cursor.MoveToNext();
                }

                obs.OnCompleted();
                // (12) Thread A finally completes subscription due to Connect()
                return Disposable.Empty;
            });
        }
    }
}

我认为问题在于一次将大量结果卸载到 ObservableCollection 中的大缓冲区，从而为列表框创建了大量工作.

I think the issue is a large buffer unloading tons of results into the ObservableCollection in one go, creating a ton of work for the listbox.

这篇关于从数据库游标缓冲数据，同时保持 UI 响应的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！