pd.read_html 错误客户端远程断开连接 [英] pd.read_html error client remote disconnected

问题描述

昨天下面的代码给了我没有问题的数据框.但是今天它开始给我以下错误:

Yesterday the following code was giving me the dataframe without problem. But today it started giving me the following error:

  http.client.RemoteDisconnected: Remote end closed connection without response

有什么解决办法吗?我知道问题出在网站上，但我需要一些解决方案来帮助我继续使用相同的代码

Any solution? I know the problem is from the website but I need some solution that can help me keep using the same code

import datetime
import pandas as pd
import sqlalchemy


from datetime import datetime
from datetime import date, timedelta

#d = datetime.today().strftime('%d-%m-%Y')
data_atual=datetime.now()
mes_atual = data_atual.month
ano_atual= data_atual.year

df = []


for j in range (2020, ano_atual+1):

    for i in range (9,13):

           frame = []


           df1 = pd.read_html('https://www.centrodeinformacao.ren.pt/userControls/GetExcel.aspx?T=REN_MENSAL&P='+str(j)+'&PP='+str(i)+'&PPP=26&PPPP=36&PPPPP=0&variation=PT', decimal=',', thousands=',')[0]
           df2 = pd.read_html('https://www.centrodeinformacao.ren.pt/userControls/GetExcel.aspx?T=REN_MENSAL&P='+str(j)+'&PP='+str(i)+'&PPP=26&PPPP=36&PPPPP=0&variation=PT', decimal=',', thousands=',')[2]


           frame.append(df1)
           frame.append(df2)
           print(frame)
           result = pd.concat(frame)
           print(result)
           engine = sqlalchemy.create_engine('mysql+pymysql://root:1234@localhost:3306/projeto')


           result.to_sql(
              name='renmensal'+str(i)+str(j),  # database table name
              con=engine,
              if_exists='replace',
              index=False
             )

推荐答案

错误在服务器端 - 有时服务器返回错误.一种解决方法是重复请求直到成功，例如:

The error is on the server side - sometimes the server returns error. One workaround is to repeat the requests until success, for example:

import http
import pandas as pd

j = 2020
i = 9

url = 'https://www.centrodeinformacao.ren.pt/userControls/GetExcel.aspx?T=REN_MENSAL&P='+str(j)+'&PP='+str(i)+'&PPP=26&PPPP=36&PPPPP=0&variation=PT'

while True:
    try:
        df1 = pd.read_html(url, decimal=',', thousands=',')[0]
        break
    except http.client.RemoteDisconnected as e:
        continue

print(df1)

打印:

                                  0  ...                    6
0                     CONSUMO [GWh]  ...  Variação Acumulada:
1                            Quebra  ...                  NaN
2                      Fios de Água  ...                 78.2
3                        Albufeiras  ...                 58.0
4    Total Hídrica Regime Ordinário  ...                 67.8
5                             Sines  ...                -77.7
6                          Ribatejo  ...                 -3.2
7                             Lares  ...                  3.0
8                         Pego C.C.  ...                 35.0
9                              Pego  ...                -64.6
10                   T.Outeiro C.C.  ...                  3.6
11   Total Térmica Regime Ordinário  ...                -20.7
12           TOTAL PRODUÇÃO LÍQUIDA  ...                  5.3
13           Importação (Comercial)  ...                -10.1
14           Exportação (Comercial)  ...                 66.0
15                 SALDO IMPORTADOR  ...                -43.9
16                       Hidráulica  ...                 37.9
17                          Térmica  ...                  3.9
18                           Eólica  ...                 -9.1
19                     Fotovoltaica  ...                 16.7
20                            Ondas  ...                  NaN
21   TOTAL PRODUÇÃO REGIME ESPECIAL  ...                 -1.3
22                    PRE Renovável  ...                 -0.6
23          BOMBAGEM HIDROELÉCTRICA  ...                 24.2
24                          CONSUMO  ...                 -3.5
25  Evolução corr.temp e dias úteis  ...                 -4.2
26     Consumo Mercado Liberalizado  ...                  ---
27         Consumo Mercado Regulado  ...                  ---

[28 rows x 7 columns]

这篇关于pd.read_html 错误客户端远程断开连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！