文件索引作为附件的FileSystemwatcher [英] FileSystemwatcher with Documents Indexing as Attachments

问题描述

我正在尝试重新生成一个控制台应用程序，该应用程序会监视一个文件夹，并将文件夹中的任何新添加的文档索引到ES。

I'm trying to reproduce a console application which watches a folder and any new additions of documents to the folder are to be Indexed to ES .

正在工作罚款如果我一次移动/添加3-4个文件并且能够索引。但是，如果我一次移动30个文档，则不会对所有文档进行索引，而是仅索引一个文档。但是如果我使用断点运行代码，那么即使30个文档也被索引。
有些人可以帮我解决这个问题。

It is working fine If I move/add 3-4 documents at a time and able to index. But if I move around 30 documents at a time, It is not indexing all the documents, instead indexing only one. But If I run the code with break points , then even 30 documents are also getting indexed. Can some one help me in solving this.

 static void OnCreated(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine("File Created: Path: {0}, \n Name: {1}", e.FullPath, e.Name);
            Indexdoc(e.FullPath);
        }

如果我不在上述代码中调用Indexdoc（e.FullPath）方法而是打印更改，它显示所有的文件名完美添加。所以filesystemwatcher没有问题。我认为索引文档需要时间才能生成响应并返回到onCreated方法。

If I dont call the Indexdoc(e.FullPath) method in the above code and instead print the changes, it is showing all the filenames added perfectly. so there is no problem with the filesystemwatcher. I think indexing documents is taking time to generate response and come back to onCreated method.

    public static void Indexdoc(string newFilePath)
            {
                List<Document> list = new List<Document>(); //list of Document class objects
                List<string> filesList = new List<string>(); //list of files in the path received on method call
                string path = string.Empty;

                client = ConfigSettings.connection();

                if (newFilePath == null) //for FULL Indexing
                {
                   //some code here
                }
                else //for new files indexing
                {
                    filesList.Add(newFilePath); //adds only one file everytime the method is called.
                    //the newFilePath will be of type C:/Documents/abc.txt
                }

                try
                {
                    foreach (string file in filesList)
                    {
                        Attachment attach = new Attachment
                        {
                            Name = Path.GetFileNameWithoutExtension(file),
                            Content = Convert.ToBase64String(File.ReadAllBytes(file)),
                            ContentType = Path.GetExtension(file)
                        };

                        var doc = new Document()
                        {
                            Title = Path.GetFileNameWithoutExtension(file),
                            FilePath = Path.GetFullPath(file), //added to get the path of the file
                            File = attach
                        };

                        list.Add(doc);
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine(e.Message.ToString());
                }

                var response = client.IndexMany(list, "trial");
            }

有些人可以帮我解决这个问题。

Can some one help me in solving this.

TIA

推荐答案

首先，创建一个队列，用于放置信息。这在类的范围内：

First, create a queue that you'll use to put the information in. This is at class scope:

private BlockingCollection<string> CreatedQueue = new BlockingCollection<string>();

在启动时，您要创建一个任务那个观察该队列更改：

At startup, you want to create a Task that watches that queue for changes:

var queueProcessor = Task.Factory.StartNew(() => ProcessQueue, TaskCreationOptions.LongRunning);

您的 OnCreated 事件处理程序如下所示：

Your OnCreated event handler looks like this:

static void OnCreated(object sender, FileSystemEventArgs e)
{
    Console.WriteLine("File Created: Path: {0}, \n Name: {1}", e.FullPath, e.Name);
    CreatedQueue.Add(e.FullPath);
}

而$ code> ProcessQueue 方法如下所示：

And the ProcessQueue method looks like this:

void ProcessQueue()
{
    foreach (var fileName in CreatedQueue.GetConsumingEnumerable())
    {
        Indexdoc(fileName);
    }
}

ProcessQueue 方法将继续从队列中删除项目（或等待添加项目，以便它可以删除它们），直到队列被标记为已完成。也就是说，你说你完成添加项目，队列是空的。所以当你关闭时，你必须调用 CompleteAdding ，然后在退出之前确保队列是空的。要做到这一点，你的主程序是这样做的：

The ProcessQueue method will continue removing items from the queue (or waiting for items to be added, so that it can remove them) until the queue is marked as completed. That is, you say you're done adding items and the queue is empty. So when you shut down, you have to call CompleteAdding, and then make sure the queue is empty before you exit. To do that, your main program does this:

// Tell the queue that no more items will be added
CreatedQueue.CompleteAdding();

// Wait for the task to complete
queueProcessor.Wait();

这篇关于文件索引作为附件的FileSystemwatcher的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！