X3，如何填充更复杂的AST? [英] X3, how to populate a more complex AST?

问题描述

尝试生成一个像雇员示例这样的AST，它不仅包含雇员.以我目前的心态，RExpressions示例并没有帮助我.我没有编译示例，但是据我所知，我在员工示例中添加了团队，部门和公司.

Trying to generate an AST like the employee example that has more than just the employee. In my current mindset, the RExpressions example isn't helping me. The example I have doesn't compile, but I went as far as I understood in adding teams, departments, and corporations to the employee example.

我的问题是要了解如何将不同的结构添加到变体中，以及如何将变体添加到phrase_parse中.

My problem is in understanding how to add the different structs into a variant and add the variant to phrase_parse, if that's the idea.

在此示例中，可能有多条相同的线紧挨着.因此想知道是否需要进行AST递归.

In this example, there can be several of the same lines following each other. So wondering if that's what requires making an AST recursive.

#include <boost/config/warning_disable.hpp>       
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/adapt_struct.hpp>   
#include <boost/fusion/include/io.hpp>

#include <iostream>
#include <string>
#include <complex>

namespace client { namespace ast
{
    struct employee;
    struct team;
    struct department;
    struct corporation;

    typedef x3::variant<
        employee,
        team,
        department,
        corporation
     > var_types;

    struct employee
    {
        int age;
        std::string surname;
        std::string forename;
        double salary;
    };

    struct team 
    {
        std::string name;
        int num_employees;  
    };

    struct department
    { 
        std::string name;
        int num_teams;  
        double budget;
    };

    struct corporation
    { 
        std::string name;
        int num_depts;
    };
}}

BOOST_FUSION_ADAPT_STRUCT(client::ast::employee, age, surname, forename, salary)
BOOST_FUSION_ADAPT_STRUCT(client::ast::team, name, num_employees)
BOOST_FUSION_ADAPT_STRUCT(client::ast::department, name, num_teams, budget)
BOOST_FUSION_ADAPT_STRUCT(client::ast::corporation, name, num_depts)

namespace client
{
    namespace parser
    {
        namespace x3 = boost::spirit::x3;
        namespace ascii = boost::spirit::x3::ascii;

        using x3::int_;
        using x3::lit;
        using x3::double_;
        using x3::lexeme;
        using ascii::char_;
        using x3::eol;
        using x3::blank;
        using x3::skip;

        x3::rule<class employee, ast::employee> const employee = "employee";
        auto const employee_def = int_ >> *(char_ - eol) >> *(char_ - eol) >> double_;
        BOOST_SPIRIT_DEFINE(employee);

        x3::rule<class team, ast::team> const team = "team";
        auto const team_def = *(char_ - eol) >> int_;
        BOOST_SPIRIT_DEFINE(team);

        x3::rule<class department, ast::department> const department = "department"; 
        auto const department_def = *(char_ - eol) >> int_ >> double_;
        BOOST_SPIRIT_DEFINE(department);

        x3::rule<class corporation, ast::corporation> const corporation = "corporation";
        auto const corporation_def = *(char_ - eol) >> int_;
        BOOST_SPIRIT_DEFINE(corporation);

        auto pemployee = skip(blank) [
            *(employee >> eol)
        ];  

        auto pteam = skip(blank) [
            *(team >> eol)
        ];  

        auto pdepartment = skip(blank) [
            *(department >> eol)
        ];  

        auto pcorporation = skip(blank) [
            *(corporation >> eol)
        ];  

        auto const input = pemployee >> pteam >> pdepartment >> pcorporation;
    }
}

int main()
{
    namespace x3 = boost::spirit::x3;
    using boost::spirit::x3::ascii::blank;
    using x3::char_;
    using client::parser::input;
    using client::ast::var_types;

    var_types types;

    std::istringstream iss("30 joe smith 100000.00\n20 mary jo 100000.00\n25 john doe 100000.00\nteamA 1\nteamB 1\nteamC 1\naccounting 1 100000.00\nengineering 2 200000.00\nAcmeCorp 3\n");

    boost::spirit::istream_iterator iter(iss >> std::noskipws), eof;

    bool ok = phrase_parse(iter, eof, input, x3::char_(' '), types);

    std::cout << "ok = " << ok << std::endl;

    return 0;
}

推荐答案

除了缺少包含和名称空间别名外，您应该确保绑定属性ref允许多个条目，因为语法匹配多个员工，团队，部门和公司...:

Except for missing includes and namespace aliases, you should probably just make sure the bound attribute ref allows multiple entries, since the grammar matches multiple employees, teams, departments and corporations...:

std::vector<var_types> types;

让它为我编译.

假设您的ast是您想要的(为什么?！它不反映语法)，这是一个有效的示例.

Assuming your ast is what you want (why?! it doesn't reflect the grammar) here's a working example.

注意

• 您需要使用lexemes并将字符串的匹配限制为非空白，否则，您将始终匹配到行尾为止
• 需要在团队之前订购部门，否则您将获得团队"而不是部门的匹配
• 使用变体有点晦涩，从打印循环中可以看到
• 只要您的规则不是递归的，就不需要BOOST_SPIRIT_DEFINE
• 使用phrase_parse是伪造的，因为无论如何您都覆盖了船长.出于正确性和易用性的原因，我喜欢在语法定义中使用船长.

• you need to use lexemes and limit the matching for strings to non-blanks otherwise you will always match till the end of the line
• departments need to be ordered before teams, or you get "team" matched instead of departments
• using the variant is a bit obscure, as you can see from the printing loop
• as long as your rules are not recursive, no need for BOOST_SPIRIT_DEFINE
• using phrase_parse is bogus because you override the skipper anyways. I like to have the skipper in the grammar definition for reasons of correctness and ease of use anyways.

在Coliru上直播

#include <iostream>
#define BOOST_SPIRIT_X3_DEBUG
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>
#include <boost/spirit/include/support_istream_iterator.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/io.hpp>

namespace x3 = boost::spirit::x3;

namespace client { namespace ast {
    struct employee;
    struct team;
    struct department;
    struct corporation;

    typedef x3::variant<
        employee,
        team,
        department,
        corporation
    > var_types;

    struct employee
    {
        int age;
        std::string surname;
        std::string forename;
        double salary;
    };

    struct team
    {
        std::string name;
        int num_employees;
    };

    struct department
    {
        std::string name;
        int num_teams;
        double budget;
    };

    struct corporation
    {
        std::string name;
        int num_depts;
    };
}}

BOOST_FUSION_ADAPT_STRUCT(client::ast::employee, age, surname, forename, salary)
BOOST_FUSION_ADAPT_STRUCT(client::ast::team, name, num_employees)
BOOST_FUSION_ADAPT_STRUCT(client::ast::department, name, num_teams, budget)
BOOST_FUSION_ADAPT_STRUCT(client::ast::corporation, name, num_depts)

namespace client
{
    namespace parser
    {
        namespace ascii = boost::spirit::x3::ascii;

        using namespace x3;

        auto const string 
            = x3::rule<struct string_, std::string> {"string"}
            = lexeme[+graph];
        auto const employee
            = x3::rule<class employee, ast::employee>{"employee"}
            = int_ >> string >> string >> double_;

        auto const team
            = x3::rule<class team, ast::team>{"team"}
            = string >> int_;

        auto const department
            = x3::rule<class department, ast::department>{"department"}
            = string >> int_ >> double_;

        auto const corporation
            = x3::rule<class corporation, ast::corporation>{"corporation"}
            = string >> int_;

        auto any = employee|department|team|corporation;
        auto const input  = skip(blank) [ *(any >> eol) ];
    }
}

int main()
{
    namespace x3 = boost::spirit::x3;
    using boost::spirit::x3::ascii::blank;
    using x3::char_;
    using client::ast::var_types;

    std::vector<var_types> types;

    std::string const iss(R"(30 joe smith 100000.00
20 mary jo 100000.00
25 john doe 100000.00
teamA 1
teamB 1
teamC 1
accounting 1 100000.00
engineering 2 200000.00
AcmeCorp 3
)");

    auto iter = iss.begin(), eof = iss.end();

    bool ok = parse(iter, eof, client::parser::input, types);

    if (iter != eof) {
        std::cout << "Remaining unparsed: '" << std::string(iter, eof) << "'\n";
    }
    std::cout << "Parsed: " << (100.0 * std::distance(iss.begin(), iter) / iss.size()) << "%\n";
    std::cout << "ok = " << ok << std::endl;

    for (auto& item : types) {
        boost::apply_visitor([](auto& v) { std::cout << boost::fusion::as_deque(v) << "\n"; }, item);
    }
}

打印

Parsed: 100%
ok = 1
(30 joe smith 100000)
(20 mary jo 100000)
(25 john doe 100000)
(teamA 1)
(teamB 1)
(teamC 1)
(accounting 1 100000)
(engineering 2 200000)
(AcmeCorp 3)

具有很多调试信息(如果启用):

With a lot of debug information if enabled:

<employee>
  <try>30 joe smith 100000.</try>
  <string>
    <try> joe smith 100000.00</try>
    <success> smith 100000.00\n20 </success>
    <attributes>[j, o, e]</attributes>
  </string>
  <string>
    <try> smith 100000.00\n20 </try>
    <success> 100000.00\n20 mary j</success>
    <attributes>[s, m, i, t, h]</attributes>
  </string>
  <success>\n20 mary jo 100000.0</success>
  <attributes>[30, [j, o, e], [s, m, i, t, h], 100000]</attributes>
</employee>
<employee>
  <try>20 mary jo 100000.00</try>
  <string>
    <try> mary jo 100000.00\n2</try>
    <success> jo 100000.00\n25 joh</success>
    <attributes>[m, a, r, y]</attributes>
  </string>
  <string>
    <try> jo 100000.00\n25 joh</try>
    <success> 100000.00\n25 john d</success>
    <attributes>[j, o]</attributes>
  </string>
  <success>\n25 john doe 100000.</success>
  <attributes>[20, [m, a, r, y], [j, o], 100000]</attributes>
</employee>
<employee>
  <try>25 john doe 100000.0</try>
  <string>
    <try> john doe 100000.00\n</try>
    <success> doe 100000.00\nteamA</success>
    <attributes>[j, o, h, n]</attributes>
  </string>
  <string>
    <try> doe 100000.00\nteamA</try>
    <success> 100000.00\nteamA 1\nt</success>
    <attributes>[d, o, e]</attributes>
  </string>
  <success>\nteamA 1\nteamB 1\ntea</success>
  <attributes>[25, [j, o, h, n], [d, o, e], 100000]</attributes>
</employee>
<employee>
  <try>teamA 1\nteamB 1\nteam</try>
  <fail/>
</employee>
<department>
  <try>teamA 1\nteamB 1\nteam</try>
  <string>
    <try>teamA 1\nteamB 1\nteam</try>
    <success> 1\nteamB 1\nteamC 1\na</success>
    <attributes>[t, e, a, m, A]</attributes>
  </string>
  <fail/>
</department>
<team>
  <try>teamA 1\nteamB 1\nteam</try>
  <string>
    <try>teamA 1\nteamB 1\nteam</try>
    <success> 1\nteamB 1\nteamC 1\na</success>
    <attributes>[t, e, a, m, A]</attributes>
  </string>
  <success>\nteamB 1\nteamC 1\nacc</success>
  <attributes>[[t, e, a, m, A], 1]</attributes>
</team>
<employee>
  <try>teamB 1\nteamC 1\nacco</try>
  <fail/>
</employee>
<department>
  <try>teamB 1\nteamC 1\nacco</try>
  <string>
    <try>teamB 1\nteamC 1\nacco</try>
    <success> 1\nteamC 1\naccountin</success>
    <attributes>[t, e, a, m, B]</attributes>
  </string>
  <fail/>
</department>
<team>
  <try>teamB 1\nteamC 1\nacco</try>
  <string>
    <try>teamB 1\nteamC 1\nacco</try>
    <success> 1\nteamC 1\naccountin</success>
    <attributes>[t, e, a, m, B]</attributes>
  </string>
  <success>\nteamC 1\naccounting </success>
  <attributes>[[t, e, a, m, B], 1]</attributes>
</team>
<employee>
  <try>teamC 1\naccounting 1</try>
  <fail/>
</employee>
<department>
  <try>teamC 1\naccounting 1</try>
  <string>
    <try>teamC 1\naccounting 1</try>
    <success> 1\naccounting 1 1000</success>
    <attributes>[t, e, a, m, C]</attributes>
  </string>
  <fail/>
</department>
<team>
  <try>teamC 1\naccounting 1</try>
  <string>
    <try>teamC 1\naccounting 1</try>
    <success> 1\naccounting 1 1000</success>
    <attributes>[t, e, a, m, C]</attributes>
  </string>
  <success>\naccounting 1 100000</success>
  <attributes>[[t, e, a, m, C], 1]</attributes>
</team>
<employee>
  <try>accounting 1 100000.</try>
  <fail/>
</employee>
<department>
  <try>accounting 1 100000.</try>
  <string>
    <try>accounting 1 100000.</try>
    <success> 1 100000.00\nenginee</success>
    <attributes>[a, c, c, o, u, n, t, i, n, g]</attributes>
  </string>
  <success>\nengineering 2 20000</success>
  <attributes>[[a, c, c, o, u, n, t, i, n, g], 1, 100000]</attributes>
</department>
<employee>
  <try>engineering 2 200000</try>
  <fail/>
</employee>
<department>
  <try>engineering 2 200000</try>
  <string>
    <try>engineering 2 200000</try>
    <success> 2 200000.00\nAcmeCor</success>
    <attributes>[e, n, g, i, n, e, e, r, i, n, g]</attributes>
  </string>
  <success>\nAcmeCorp 3\n</success>
  <attributes>[[e, n, g, i, n, e, e, r, i, n, g], 2, 200000]</attributes>
</department>
<employee>
  <try>AcmeCorp 3\n</try>
  <fail/>
</employee>
<department>
  <try>AcmeCorp 3\n</try>
  <string>
    <try>AcmeCorp 3\n</try>
    <success> 3\n</success>
    <attributes>[A, c, m, e, C, o, r, p]</attributes>
  </string>
  <fail/>
</department>
<team>
  <try>AcmeCorp 3\n</try>
  <string>
    <try>AcmeCorp 3\n</try>
    <success> 3\n</success>
    <attributes>[A, c, m, e, C, o, r, p]</attributes>
  </string>
  <success>\n</success>
  <attributes>[[A, c, m, e, C, o, r, p], 3]</attributes>
</team>
<employee>
  <try></try>
  <fail/>
</employee>
<department>
  <try></try>
  <string>
    <try></try>
    <fail/>
  </string>
  <fail/>
</department>
<team>
  <try></try>
  <string>
    <try></try>
    <fail/>
  </string>
  <fail/>
</team>
<corporation>
  <try></try>
  <string>
    <try></try>
    <fail/>
  </string>
  <fail/>
</corporation>

AST FIX

如果您重新制作AST以更好地模仿语法:

AST FIX

If you rework your AST to mimick the grammar better:

namespace client { namespace ast {
    struct employee    { int age; std::string surname; std::string forename; double salary; };
    struct team        { std::string name; int num_employees;                               };
    struct department  { std::string name; int num_teams; double budget;                    };
    struct corporation { std::string name; int num_depts;                                   };

    struct input {
        std::vector<employee>    employees;
        std::vector<team>        teams;
        std::vector<department>  departments;
        std::vector<corporation> corporations;
    };
} }

现在，当发生这种情况时，所有属性强制规则都变得多余了，您可以简单地拥有以下语法:

Now, as it happens all the attribute coercion rules become redundant, and you can simply have this grammar:

namespace parser
{
    using namespace x3;

    static auto string      = lexeme[+graph];
    static auto employee    = int_ >> string >> string >> double_;
    static auto team        = string >> int_;
    static auto department  = string >> int_ >> double_;
    static auto corporation = string >> int_;

    auto const input  = skip(blank) [
           *(employee >> eol)
        >> *(team >> eol)
        >> *(department >> eol)
        >> *(corporation >> eol)
    ];
}

就是 all .我希望助手在行尾更具表现力:

That's all. I prefer a helper to be more expressive with the line ends:

static auto lines = [](auto p) { return *(p >> eol); };
auto const input  = skip(blank) [
       lines(employee)
    >> lines(team)
    >> lines(department)
    >> lines(corporation)
];

注意

• 没有更多的变体，打印就像您期望的那样简单:

• No more variant, printing is as simple as you'd expect:

for (auto& item : types.employees)    { std::cout << boost::fusion::as_deque(item) << "\n"; }
for (auto& item : types.teams)        { std::cout << boost::fusion::as_deque(item) << "\n"; }
for (auto& item : types.departments)  { std::cout << boost::fusion::as_deque(item) << "\n"; }
for (auto& item : types.corporations) { std::cout << boost::fusion::as_deque(item) << "\n"; }

团队/部门之间不再存在歧义，因为它们只能以固定顺序出现

No more ambiguity between team/department because they can only occur in fixed order

在Coliru上直播

#include <iostream>
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/io.hpp>

namespace x3 = boost::spirit::x3;

namespace client { namespace ast {
    struct employee    { int age; std::string surname; std::string forename; double salary; };
    struct team        { std::string name; int num_employees;                               };
    struct department  { std::string name; int num_teams; double budget;                    };
    struct corporation { std::string name; int num_depts;                                   };

    struct input {
        std::vector<employee>    employees;
        std::vector<team>        teams;
        std::vector<department>  departments;
        std::vector<corporation> corporations;
    };
} }

BOOST_FUSION_ADAPT_STRUCT(client::ast::employee, age, surname, forename, salary)
BOOST_FUSION_ADAPT_STRUCT(client::ast::team, name, num_employees)
BOOST_FUSION_ADAPT_STRUCT(client::ast::department, name, num_teams, budget)
BOOST_FUSION_ADAPT_STRUCT(client::ast::corporation, name, num_depts)
BOOST_FUSION_ADAPT_STRUCT(client::ast::input, employees, teams, departments, corporations)

namespace client
{
    namespace parser
    {
        namespace ascii = boost::spirit::x3::ascii;

        using namespace x3;

        auto const string 
            //= x3::rule<struct string_, std::string> {"string"}
            = lexeme[+graph];

        auto const employee
            //= x3::rule<class employee, ast::employee>{"employee"}
            = int_ >> string >> string >> double_;

        auto const team
            //= x3::rule<class team, ast::team>{"team"}
            = string >> int_;

        auto const department
            //= x3::rule<class department, ast::department>{"department"}
            = string >> int_ >> double_;

        auto const corporation
            //= x3::rule<class corporation, ast::corporation>{"corporation"}
            = string >> int_;

        auto lines = [](auto p) { return *(p >> eol); };
        auto const input  
             //= x3::rule<struct _input, ast::input>{"input"} 
             = skip(blank) [
               lines(employee)
            >> lines(team)
            >> lines(department)
            >> lines(corporation)
        ];
    }
}

int main()
{
    namespace x3 = boost::spirit::x3;
    using boost::spirit::x3::ascii::blank;
    using x3::char_;

    std::string const iss(R"(30 joe smith 100000.00
20 mary jo 100000.00
25 john doe 100000.00
teamA 1
teamB 1
teamC 1
accounting 1 100000.00
engineering 2 200000.00
AcmeCorp 3
)");

    auto iter = iss.begin(), eof = iss.end();

    client::ast::input types;
    bool ok = parse(iter, eof, client::parser::input, types);

    if (iter != eof) {
        std::cout << "Remaining unparsed: '" << std::string(iter, eof) << "'\n";
    }
    std::cout << "Parsed: " << (100.0 * std::distance(iss.begin(), iter) / iss.size()) << "%\n";
    std::cout << "ok = " << ok << std::endl;

    for (auto& item : types.employees)    { std::cout << boost::fusion::as_deque(item) << "\n"; }
    for (auto& item : types.teams)        { std::cout << boost::fusion::as_deque(item) << "\n"; }
    for (auto& item : types.departments)  { std::cout << boost::fusion::as_deque(item) << "\n"; }
    for (auto& item : types.corporations) { std::cout << boost::fusion::as_deque(item) << "\n"; }
}

打印

Parsed: 100%
ok = 1
(30 joe smith 100000)
(20 mary jo 100000)
(25 john doe 100000)
(teamA 1)
(teamB 1)
(teamC 1)
(accounting 1 100000)
(engineering 2 200000)
(AcmeCorp 3)

这篇关于X3，如何填充更复杂的AST?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！