如何在elasticsearch中将一个索引文档复制到另一个索引? [英] how to copy one index documents to other index in elasticsearch?

问题描述

我用映射创建了一个新索引.其中存储了 500 000 个文档.

I created a new index with mappings. 500 000 documents are stored in it.

我想更改索引的映射，但在弹性搜索中是不可能的.所以我用新的新映射创建了另一个索引，现在我试图将文档从旧索引复制到新索引.

I want to change the mapping of the index, but it is not possible in elastic search.so I created another index with new new mappings and now I am trying to copy the documents from old index to new index.

我正在使用扫描和滚动类型从旧索引中检索文档并将其复制到新索引.因为复制需要更多时间并且系统速度变慢.

I am using scan and scroll type to retrieve the documents from old index and copying to new index.for copying its taking more time and the system is slowing down.

以下是我正在使用的代码.

Below is the code that I am using.

$client= app('elastic_search');
    $params = [
        "search_type" => "scan",    // use search_type=scan
        "scroll" => "30s",          // how long between scroll requests. should be small!
        "size" => 500000,               // how many results *per shard* you want back
        "index" => "admin_logs422",
        "body" => [
            "query" => [
                "match_all" => []
            ]
        ]
    ];

    $docs = $client->search($params);   // Execute the search

    $scroll_id = $docs['_scroll_id'];   


    while (	rue) {

        // Execute a Scroll request
        $response = $client->scroll([
                "scroll_id" => $scroll_id,  //...using our previously obtained _scroll_id
                "scroll" => "500s"           // and the same timeout window
            ]
        );

        if (count($response['hits']['hits']) > 0) {
            foreach($response['hits']['hits'] as $s)
            {

                $params =
                    [
                        'index' => 'admin_logs421',
                        'type' => 'admin_type421',
                        'id'=> $s['_id'],
                        'client' => [
                            'ignore' => [400, 404],
                            'verbose' => true,
                            'timeout' => 10,
                            'connect_timeout' => 10
                        ],
                        'body' => $s['_source']
                    ];

                $response = app('elastic_search')->create($params);

            }


            $scroll_id = $response['_scroll_id'];
        } else {
            // No results, scroll cursor is empty.  You've exported all the data
            return response("completed");
        }
    }

推荐答案

您不应该编写类似的代码.有一些出色的工具可以为您完成.

You should not have to code stuff like that. There are some excellent tools around to do it for you.

只需看看 Taskrabbit 的 elasticdump 实用程序，它完全符合您的要求.

Just look at Taskrabbit's elasticdump utility which does exactly what you want.

elasticdump 
  --input=http://localhost:9200/source_index 
  --output=http://localhost:9200/target_index 
  --type=data

或者您也可以非常轻松地利用 Logstash，如 这个其他答案

Or you can also very easily leverage Logstash as is shown in this other answer

最后，由于您使用的是 Python，您还可以使用 elasticsearch-py 重新索引实用程序

Finally, since you're using Python, you can also use the elasticsearch-py reindex utility

这篇关于如何在elasticsearch中将一个索引文档复制到另一个索引?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！