mgo - 查询性能似乎一直很慢(500-650 毫秒) [英] mgo - query performance seems consistently slow (500-650ms)

问题描述

我的数据层大量使用 Mongo 聚合，平均而言，查询需要 500-650 毫秒才能返回.我正在使用 mgo.

My data layer uses Mongo aggregation a decent amount, and on average, queries are taking 500-650ms to return. I am using mgo.

下面显示了一个示例查询函数，它代表了我的大多数查询的样子.

A sample query function is shown below which represents what most of my queries look like.

func (r userRepo) GetUserByID(id string) (User, error) {
    info, err := db.Info()
    if err != nil {
        log.Fatal(err)
    }

    session, err := mgo.Dial(info.ConnectionString())
    if err != nil {
        log.Fatal(err)
    }
    defer session.Close()

    var user User
    c := session.DB(info.Db()).C("users")
    o1 := bson.M{"$match": bson.M{"_id": id}}
    o2 := bson.M{"$project": bson.M{
        "first":           "$first",
        "last":            "$last",
        "email":           "$email",
        "fb_id":           "$fb_id",
        "groups":          "$groups",
        "fulfillments":    "$fulfillments",
        "denied_requests": "$denied_requests",
        "invites":         "$invites",
        "requests": bson.M{
            "$filter": bson.M{
                "input": "$requests",
                "as":    "item",
                "cond": bson.M{
                    "$eq": []interface{}{"$$item.active", true},
                },
            },
        },
    }}
    pipeline := []bson.M{o1, o2}
    err = c.Pipe(pipeline).One(&user)
    if err != nil {
        return user, err
    }
    return user, nil
}

我拥有的 user 结构如下所示..

The user struct I have looks like the following..

type User struct {
    ID             string        `json:"id" bson:"_id,omitempty"`
    First          string        `json:"first" bson:"first"`
    Last           string        `json:"last" bson:"last"`
    Email          string        `json:"email" bson:"email"`
    FacebookID     string        `json:"facebook_id" bson:"fb_id,omitempty"`
    Groups         []UserGroup   `json:"groups" bson:"groups"`
    Requests       []Request     `json:"requests" bson:"requests"`
    Fulfillments   []Fulfillment `json:"fulfillments" bson:"fulfillments"`
    Invites        []GroupInvite `json:"invites" bson:"invites"`
    DeniedRequests []string      `json:"denied_requests" bson:"denied_requests"`
}

根据我提供的信息，是否有任何明显的迹象表明为什么我的查询平均需要 500-650 毫秒?

Based on what I have provided, is there anything obvious that would suggest why my queries are averaging 500-650ms?

我知道使用聚合管道可能会对性能造成一些影响，但我不认为它会如此糟糕.

I know that I am probably swallowing a bit of a performance hit by using aggregation pipeline, but I wouldn't expect it to be this bad.

推荐答案

.. 是否有任何明显的迹象可以表明为什么我的查询者平均需要 500-650 毫秒?

.. is there anything obvious that would suggest why my queriers are averaging 500-650ms?

是的，有.您在执行之前调用 mgo.Dial()每个查询.mgo.Dial() 每次都必须连接到 MongoDB 服务器，在查询后立即关闭.连接很可能需要数百毫秒才能建立，包括身份验证、资源分配(服务器端和客户端)等.这是非常浪费的.

Yes, there is. You are calling mgo.Dial() before executing each query. mgo.Dial() has to connect to the MongoDB server every time, which you close right after the query. The connection may very likely take hundreds of milliseconds to estabilish, including authentication, allocating resources (both at server and client side), etc. This is very wasteful.

对于给定的集群，此方法通常只调用一次. 然后在获得的会话上使用 New 或 Copy 方法建立到同一集群的进一步会话.这将使它们共享底层集群，并适当地管理连接池.

This method is generally called just once for a given cluster. Further sessions to the same cluster are then established using the New or Copy methods on the obtained session. This will make them share the underlying cluster, and manage the pool of connections appropriately.

创建一个全局会话变量，在启动时连接一次(使用例如包 init() 函数)，并使用该会话(或它的副本/克隆)，通过 Session.Copy() 获得或 Session.Clone()).例如:

Create a global session variable, connect on startup once (using e.g. a package init() function), and use that session (or a copy / clone of it, obtained by Session.Copy() or Session.Clone()). For example:

var session *mgo.Session
var info *db.Inf // Use your type here

func init() {
    var err error
    if info, err = db.Info(); err != nil {
        log.Fatal(err)
    }
    if session, err = mgo.Dial(info.ConnectionString()); err != nil {
        log.Fatal(err)
    }
}

func (r userRepo) GetUserByID(id string) (User, error) {
    sess := session.Clone()
    defer sess.Close()

    // Now we use sess to execute the query:
    var user User
    c := sess.DB(info.Db()).C("users")
    // Rest of the method is unchanged...
}

这篇关于mgo - 查询性能似乎一直很慢(500-650 毫秒)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！