rllib使用自定义注册环境 [英] rllib use custom registered environments

问题描述

Rllib文档提供了有关如何创建和训练自定义环境的一些信息.有一些有关注册该环境的信息，但我想它的工作方式与健身房注册.

Rllib docs provide some information about how to create and train a custom environment. There is some information about registering that environment, but I guess it needs to work differently than gym registration.

我正在使用 SimpleCorridor 环境.如果我将注册码添加到文件中，如下所示:

I'm testing this out working with the SimpleCorridor environment. If I add the registration code to the file like so:

from ray.tune.registry import register_env

class SimpleCorridor(gym.Env):
   ...


def env_creator(env_config):
    return SimpleCorridor(env_config)

register_env("corridor", env_creator)

然后我就可以使用字符串名称来训练算法了:

Then I am able to train an algorithm using the string name no problem:

if __name__ == "__main__":
    ray.init()
    tune.run(
        "PPO",
        stop={
            "timesteps_total": 10000,
        },
        config={
            "env": "corridor", # <--- This works fine!
            "env_config": {
                "corridor_length": 5,
            },
        },
    )

但是

将环境注册到定义环境的同一文件中是毫无意义的，因为您只能使用该类. OpenAI体育馆注册很不错，因为如果您安装了该环境，那么您只需编写

It is kinda pointless to register the environment in the same file that you define the environment because you can just use the class. OpenAI gym registration is nice because if you install the environment, then you can use it anywhere just by writing

include gym_corridor

对于我来说还不清楚是否有一种方法可以为rllib注册环境.有办法吗?

It's not clear to me if there is a way to do the same thing for registering environments for rllib. Is there a way to do this?

推荐答案

ray中的注册表功能让人头疼.我不知道为什么他们无法识别其他环境，例如OpenAI Gym.

The registry functions in ray are a massive headache; I don't know why they can't recognize other environments like OpenAI Gym.

无论如何，我解决此问题的方法是将自定义环境包装在另一个自动导入环境的函数中，以便我可以重用代码.例如:

Anyway, the way I've solved this is by wrapping my custom environments in another function that imports the environment automatically so I can re-use code. For example:

def env_creator(env_name):
    if env_name == 'CustomEnv-v0':
        from custom_gym.envs.custom_env import CustomEnv0 as env
    elif env_name == 'CustomEnv-v1':
        from custom_gym.envs.custom_env import CustomEnv1 as env
    else:
        raise NotImplementedError
    return env

然后，要使其与tune.register_env()一起使用，可以将自定义env与lambda函数一起使用:

Then, to get it to work with the tune.register_env(), you can use your custom env with a lambda function:

env = env_creator('CustomEnv-v0')
tune.register_env('myEnv', lambda: config, env(config))

从那里开始，tune.run()应该可以工作.这很烦人，但这是我发现可以解决此注册表问题的最佳方法.

From there, tune.run() should work. It's annoying, but that's the best way I've found to work around this registry issue.

这篇关于rllib使用自定义注册环境的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！