如何自定义Go中的http.Client或http.Transport超时后重试？ [英] How to customize http.Client or http.Transport in Go to retry after timeout?
本文介绍了如何自定义Go中的http.Client或http.Transport超时后重试？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我想为标准 http.Client 实现自定义 http.Transport ，如果客户超时。
PS由于某种原因，自定义 http.Transport 是必备。我已经检查过 hashicorp / go-retryablehttp ，但是它不允许我使用我的自己的 http.Transport 。
这是我的尝试，自定义组件：
type CustomTransport结构{
http.RoundTripper
// ...私有字段
}
func NewCustomTransport（上游* http.Transport）* CustomTransport {
上游.TLSClientConfig =& tls.Config {InsecureSkipVerify：true}
// ...其他运输定制
返回& CustomTransport {上游}
}
函数（ct * CustomTransport）RoundTrip（req * http.Request）（resp * http.Response，错误错误）{
req.Header.Set （ Secret， Blah blah blah）
// ...每个请求的其他自定义
for i：= 1;我< = 5; i ++ {
resp，err = ct.RoundTripper.RoundTrip（req）
如果有错误。Is（err，context.DeadlineExceeded）{
log.Warnf（＃％d将会超时重试-％v，i，错误）
//time.Sleep(time.Duration(100*i）* time.Millisecond）
Continue
} else {
break
}
}
log.Debugf（获得最终结果：％v，错误）
返回值，错误
}
呼叫者代码：
func main（）{
transport：= NewCustomTransport（http.DefaultTransport。（* http.Transport））
client：=& http.Client {
超时：8 *时间第二，
运输：运输，
}
apiUrl：= https://httpbin.org/delay/10；
log.Debugf（开始获取％q，apiUrl）
start：= time.Now（）
resp，err：= client.Get（apiUrl）
if err！= nil {
log.Warnf（客户端收到错误：％v，err）
}否则{
推迟resp.Body.Close（）
}
log.Debugf（结束以获取％q，时间成本：％v，apiUrl，时间。Since（start））
if resp！= nil {
数据，错误：= httputil.DumpResponse（resp，true）
，如果错误！= nil {
log.Warnf（无法转储resp：％v，错误）
}
fmt.Println（string（data））
}
}
我的实现没有按预期工作，一旦客户端超时，重试实际上就不会发生。参见以下日志：
2020-07-15T00：53：22.586调试开始获取" https：//httpbin.org/delay/ 10英寸
2020-07-15T00：53：30.590警告＃1超时将重试-上下文截止日期已超过
2020-07-15T00：53：30.590警告＃2超时将重试-上下文截止日期已超过
2020-07-15T00：53：30.590警告＃3超时将重试-上下文截止日期已超过
2020-07-15T00：53：30.590警告＃4超时将重试-上下文截止日期已超过
2020-07-15T00：53：30.590警告＃5超时将重试-上下文截止日期超过
2020-07-15T00：53：30.590调试得到最终结果：上下文截止日期超过
2020-07-15T00 ：53：30.590 WARN客户端发生错误：获取 https://httpbin.org/delay/10：超出上下文截止时间（在等待标题时超出了Client.Timeout）
2020-07-15T00：53：30.590调试结束即可获得 https://httpbin.org/delay/10，时间成本：8.00042786s
Can您请告诉我如何解决此问题，或如何实现此类 http.Client ？
解决方案
请注意，http.Client的超时字段已过时。现在的最佳实践是使用http.Request.Context（）进行超时。 – Flimzy
感谢@Flimzy的启发！我尝试将上下文用于超时控制，而不是使用http.Client方式。代码如下：
func（ct * CustomTransport）RoundTrip（req * http.Request）（resp * http。响应，错误错误）{
req.Header.Set（ Secret， Blah blah blah））
// ...每个请求的其他自定义
对于我：= 1;我< = 5; i ++ {
ctx，cancel：= context.WithTimeout（context.Background（），10 * time.Second）
defer cancel（）
// reqT：= req.WithContext（ctx）
resp，err = ct.RoundTripper.RoundTrip（req.WithContext（ctx））
如果有错误。Is（err，context.DeadlineExceeded）{
log.Warnf（＃％d超时将重试-％v，i，err）
//time.Sleep(time.Duration(100*i）* time.Millisecond）
Continue
} else {
break
}
}
根据日志，它可以正常工作（注意时间戳记在日志，它实际上已重试）：
2020-07-16T00：06：12.788 + 0800 DEBUG开始获取 https：// httpbin .org / delay / 10
2020-07-16T00：06：20.794 + 0800警告＃1超时将重试-上下文截止日期已超过
2020-07-16T00：06：28.794 + 0800 WARN＃2超时将重试-上下文超过
的最后期限2020-07-16T00：06：36.799 + 0800警告＃3超时将重试-上下文超过
2020-07-16T00：06：44.803 + 0800警告＃4超时将重试-上下文截止时间超过
2020-07-16T00：06：52.809 + 0800警告＃5超时将重试-上下文截止时间超过
2020-07-16T00：06：52.809 + 0800 DEBUG得到最终结果：上下文截止日期超过
2020-07-16T00：06：52.809 + 0800 WARN客户端收到错误：获取``https://httpbin.org/delay/10''：上下文截止日期超过
2020-07- 16T00：06：52.809 + 0800调试结束以获得 https://httpbin.org/delay/10，时间成本：40.019334668s
I want to implement a custom http.Transport for standard http.Client, which will retry automatically if the client got timeout.
P.S. for some reason, the custom http.Transport is a must-have. I've already checked hashicorp/go-retryablehttp, however it won't let me use my own http.Transport.
Here's my attempts, the custom component:
type CustomTransport struct {
http.RoundTripper
// ... private fields
}
func NewCustomTransport(upstream *http.Transport) *CustomTransport {
upstream.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
// ... other customizations for transport
return &CustomTransport{upstream}
}
func (ct *CustomTransport) RoundTrip(req *http.Request) (resp *http.Response, err error) {
req.Header.Set("Secret", "Blah blah blah")
// ... other customizations for each request
for i := 1; i <= 5; i++ {
resp, err = ct.RoundTripper.RoundTrip(req)
if errors.Is(err, context.DeadlineExceeded) {
log.Warnf("#%d got timeout will retry - %v", i, err)
//time.Sleep(time.Duration(100*i) * time.Millisecond)
continue
} else {
break
}
}
log.Debugf("got final result: %v", err)
return resp, err
}
The caller code:
func main() {
transport := NewCustomTransport(http.DefaultTransport.(*http.Transport))
client := &http.Client{
Timeout: 8 * time.Second,
Transport: transport,
}
apiUrl := "https://httpbin.org/delay/10"
log.Debugf("begin to get %q", apiUrl)
start := time.Now()
resp, err := client.Get(apiUrl)
if err != nil {
log.Warnf("client got error: %v", err)
} else {
defer resp.Body.Close()
}
log.Debugf("end to get %q, time cost: %v", apiUrl, time.Since(start))
if resp != nil {
data, err := httputil.DumpResponse(resp, true)
if err != nil {
log.Warnf("fail to dump resp: %v", err)
}
fmt.Println(string(data))
}
}
My implementations didn't work as expected, once it got the client timeout, the retry won't actually happen. See the log below:
2020-07-15T00:53:22.586 DEBUG begin to get "https://httpbin.org/delay/10"
2020-07-15T00:53:30.590 WARN #1 got timeout will retry - context deadline exceeded
2020-07-15T00:53:30.590 WARN #2 got timeout will retry - context deadline exceeded
2020-07-15T00:53:30.590 WARN #3 got timeout will retry - context deadline exceeded
2020-07-15T00:53:30.590 WARN #4 got timeout will retry - context deadline exceeded
2020-07-15T00:53:30.590 WARN #5 got timeout will retry - context deadline exceeded
2020-07-15T00:53:30.590 DEBUG got final result: context deadline exceeded
2020-07-15T00:53:30.590 WARN client got error: Get "https://httpbin.org/delay/10": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2020-07-15T00:53:30.590 DEBUG end to get "https://httpbin.org/delay/10", time cost: 8.004182786s
Can you please tell me how to fix this, or any methods/ideas to implement such a http.Client?
解决方案
Note that the Timeout field of http.Client is more or less obsolete. Best practice now is to use http.Request.Context() for timeouts. – Flimzy
Thanks for the inspiration from @Flimzy! I attempted to use context for timeout control instead of http.Client way. Here's the code:
func (ct *CustomTransport) RoundTrip(req *http.Request) (resp *http.Response, err error) {
req.Header.Set("Secret", "Blah blah blah")
// ... other customizations for each request
for i := 1; i <= 5; i++ {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
//reqT := req.WithContext(ctx)
resp, err = ct.RoundTripper.RoundTrip(req.WithContext(ctx))
if errors.Is(err, context.DeadlineExceeded) {
log.Warnf("#%d got timeout will retry - %v", i, err)
//time.Sleep(time.Duration(100*i) * time.Millisecond)
continue
} else {
break
}
}
As per the log, it works (note the timestamp in the logs, it actually retried):
2020-07-16T00:06:12.788+0800 DEBUG begin to get "https://httpbin.org/delay/10"
2020-07-16T00:06:20.794+0800 WARN #1 got timeout will retry - context deadline exceeded
2020-07-16T00:06:28.794+0800 WARN #2 got timeout will retry - context deadline exceeded
2020-07-16T00:06:36.799+0800 WARN #3 got timeout will retry - context deadline exceeded
2020-07-16T00:06:44.803+0800 WARN #4 got timeout will retry - context deadline exceeded
2020-07-16T00:06:52.809+0800 WARN #5 got timeout will retry - context deadline exceeded
2020-07-16T00:06:52.809+0800 DEBUG got final result: context deadline exceeded
2020-07-16T00:06:52.809+0800 WARN client got error: Get "https://httpbin.org/delay/10": context deadline exceeded
2020-07-16T00:06:52.809+0800 DEBUG end to get "https://httpbin.org/delay/10", time cost: 40.019334668s
这篇关于如何自定义Go中的http.Client或http.Transport超时后重试？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
