哪个MySQL版本与Airflow 1.10版本兼容？ [英] Which version of MySQL is compatible with Airflow version 1.10?

问题描述

我正在尝试使用LocalExecutor而不是默认的SequentialExecutor，后者强制使用与SQLlite不同的数据库。

I am trying to use LocalExecutor instead of the default SequentialExecutor which forces to use a different database then SQLlite.

我想尝试MySQL，但是我看到了问题与MySQL 5.6、5.7版一起使用？不确定是否与版本兼容性有关。

I wanted to try MySQL, however I am seeing issues with MySQL version 5.6, 5.7? Not sure if it is related to version compatibility.

很乐意看到与Airflow版本和兼容的MySQL版本有关的任何文档。

Would love to see any documentation related to Airflow versions and compatible MySQL versions.

更新：

这是我在使用MySQL后端时单击任何DAG相关按钮时在UI中看到的糟糕错误：

Here is the Ooops error I am seeing in th UI when click on any of the DAG related buttons while using MySQL backend:

     Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask_admin/base.py", line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask_admin/base.py", line 368, in _run_view
    return fn(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/flask_login.py", line 755, in decorated_view
    return func(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/airflow/www/utils.py", line 364, in view_func
    return f(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/airflow/www/utils.py", line 268, in wrapper
    session.commit()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 927, in commit
    self.transaction.commit()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 467, in commit
    self._prepare_impl()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
    self.session.flush()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2209, in flush
    self._flush(objects)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2329, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2293, in _flush
    flush_context.execute()
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
    execute(statement, params)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/home/ec2-user/.local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib64/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/usr/local/lib64/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/usr/local/lib64/python2.7/site-packages/pymysql/connections.py", line 516, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/local/lib64/python2.7/site-packages/pymysql/connections.py", line 727, in _read_query_result
    result.read()
  File "/usr/local/lib64/python2.7/site-packages/pymysql/connections.py", line 1066, in read
    first_packet = self.connection._read_packet()
  File "/usr/local/lib64/python2.7/site-packages/pymysql/connections.py", line 683, in _read_packet
    packet.check_error()
  File "/usr/local/lib64/python2.7/site-packages/pymysql/protocol.py", line 220, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib64/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
ProgrammingError: (pymysql.err.ProgrammingError) (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[(\\'dag_id\\', u\\'DataPipeline\\')]'')' at line 1") [SQL: u'INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s)'] [parameters: {'task_id': None, 'extra': "[('dag_id', u'DataPipeline')]", 'execution_date': None, 'event': 'tree', 'owner': 'anonymous', 'dttm': datetime.datetime(2018, 10, 6, 5, 28, 31, 180060, tzinfo=<Timezone [UTC]>), 'dag_id': u'DataPipeline'}]

推荐答案

这些时间我一直在使用Python27，一旦我将linux更新为Python36都开始工作，现在我将Airflow1.10与Mysql5.6.4一起使用

I was using Python27 all these time, once I update my linux to Python36 all started working. Now I am using Airflow1.10 with Mysql5.6.4

还有一点要记住，将python版本从python27更新为python36后，应清除浏览器cookie，以使其在用户界面中起作用，否则您将看到相同的错误。

One more thing to keep in mind, browser cookies should be cleared once you update your python version from python27 to python36 for this to work in the UI or else you will see the same error.

希望这对某人有帮助！

这篇关于哪个MySQL版本与Airflow 1.10版本兼容？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！