如何在运行时动态设置对象的方法(当对象是 Gym 环境的实例时)? [英] How to dynamically set an object's method at runtime (when the object is an instance of a Gym environment)?

问题描述

我想覆盖一个实例的方法.让我们称这个实例/对象为 env.实例方法有一个带有 f(self, other_parameter) 签名的方法，它实际上是不可公开访问的(不知道为什么，但你会在下面看到它没有列在类和对象:你能告诉我为什么吗?).但是，我得到了 new_f(other_parameter) 形式的函数，即没有 self 参数，我需要将其绑定到实例/对象，根据 这个答案.然而，这不是我的第一个问题.

我的第一个问题是，我从 这篇文章 中尝试过的任何方法似乎都不适用于我的情况，即新方法没有分配给对象:至少，这是我从该方法似乎没有被调用的事实中推断出来的.您可以运行以下示例来查看问题.

导入类型从 pprint 导入 pprint进口健身房类 MyEnv(gym.Env):def f(self, other_parameter):打印(其他参数(原始)="，其他参数)返回 0def step(self, action: int) ->元组:返回 [], self.f(action), False, {}定义重置(自我):返回 []env_id = "my_env-v1";env = MyEnv()健身房.envs.register(id=env_id,entry_point=lambda:环境，max_episode_steps=200)env = 健身房.make(env_id)pprint(目录(环境))环境重置()# 正确打印以下内容# other_parameter (原始) = 0# ([], 0, 假, {})打印(环境.步骤(0))打印(-"* 10)def f(self, other_parameter):打印(其他参数(新)="，其他参数)返回 10# 它似乎没有任何影响.# env.f = f# 即使我尝试其他方法，它似乎也不起作用.env.f = types.MethodType(f, env)# INCORRECTLY 打印以下内容# other_parameter (原始) = 7# ([], 0, 假, {})# 它应该打印# other_parameter (new) = 7# ([], 10, 假, {})打印(环境.步骤(7))

我正在使用 gym 0.17.3(只需执行 pip installgym==0.17.3 来安装它).

出现此问题的原因可能是 Gym 环境是通过调用 gym.make(env_id) 创建的，这可能会执行一些魔术或奇怪的技巧.我已经看过这个模块，其中make 已定义，但我还不确定问题出在哪里.我真的需要在创建环境后设置其方法，因此如果您有关于如何操作的建议，我将不胜感激.

解决方案

我刚刚注意到，如果我尝试将新方法 f 绑定到 env before 行 gym.envs.register(...)，它有效，但我想在注册后更改 f.

同时，我还注意到，gym 可以将您的环境对象包装到另一个类中.在上面的例子中，结果证明 MyEnv 对象是在 TimeLimit 对象内创建的，这就是上面例子中的 env 变量所指的至.我还注意到 env 有另一个名为 env 的属性，它是 MyEnv 类型的实际对象，所以我上面的例子只通过改变线

env.f = types.MethodType(f, env)

到

env.env.f = types.MethodType(f, env)

这也解释了为什么before注册环境有效，因为env注册环境之前实际上是MyEnv的一个实例.

I want to override an instance's method. Let's call this instance/object env. The instance method has a method with the signature f(self, other_parameter), which is actually not publically accessible (not sure why, but you will see below that it's not listed in the list of properties of the class and object: can you please tell me why?). However, I'm given functions of the form new_f(other_parameter), i.e. without the self parameter, which I need to bind to the instance/object, according to this answer. However, this is not my first problem.

My first problem is that no method that I've tried from this post seems to work in my case, i.e. the new method is not assigned to the object: at least, this is what I deduce from the fact that the method doesn't seem to be called. You can run the following example to see the problem.

import types
from pprint import pprint

import gym


class MyEnv(gym.Env):
    def f(self, other_parameter):
        print("other_parameter (original) =", other_parameter)
        return 0

    def step(self, action: int) -> tuple:
        return [], self.f(action), False, {}

    def reset(self):
        return []


env_id = "my_env-v1"
env = MyEnv()
gym.envs.register(id=env_id,
                  entry_point=lambda: env,
                  max_episode_steps=200)

env = gym.make(env_id)

pprint(dir(env))

env.reset()

# CORRECTLY prints the following
# other_parameter (original) = 0
# ([], 0, False, {})
print(env.step(0))

print("-" * 10)


def f(self, other_parameter):
    print("other_parameter (new) =", other_parameter)
    return 10


# IT DOES NOT SEEM TO TAKE ANY EFFECT.
# env.f = f

# EVEN IF I TRY OTHER METHODS, IT DOES NOT SEEM TO WORK.
env.f = types.MethodType(f, env)

# INCORRECTLY prints the following
# other_parameter (original) = 7
# ([], 0, False, {})
# IT SHOULD PRINT
# other_parameter (new) = 7
# ([], 10, False, {})
print(env.step(7))

I'm using gym 0.17.3 (just do pip install gym==0.17.3 to install it).

This problem probably occurs because Gym environments are created by calling gym.make(env_id), which probably does some magic or weird tricks. I've already looked at this module where make is defined, but I'm not yet sure where the problem could be. I really need to set the method of a created environment once it's been created, so if you have a suggestion of how to do it, I would appreciate it.

解决方案

I've just noticed that if I try to bind the new method f to env before the line gym.envs.register(...), it works, but I would like to change f after the registration.

Meanwhile, I've also noticed that gym can wrap your environment object into another class. In the example above, it turns out that an MyEnv object is created inside an TimeLimit object, which is what env variable in the example above points to. I also notice that env has another property called env, which is the actual object of type MyEnv, so my example above works by only changing the line

env.f = types.MethodType(f, env)

to

env.env.f = types.MethodType(f, env)

This also explains why before registering the environment it works, because env before registering the environment is actually an instance of MyEnv.

这篇关于如何在运行时动态设置对象的方法(当对象是 Gym 环境的实例时)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！