无法使用javaagent为Spring Boot uber jar应用程序检测apache httpclient [英] Unable to instrument apache httpclient using javaagent for spring boot uber jar application

查看：141
本文介绍了无法使用javaagent为Spring Boot uber jar应用程序检测apache httpclient的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我试图用Bytebuddy编写一个Javaagent来拦截apache httpclient请求，并且我想将此代理用于Spring Boot应用程序.当我从Idea启动我的测试Spring Boot应用程序时，代理工作正常(直接运行main方法).但是，当我将应用程序打包到spring boot uber jar中并使用java -javaagent:myagent.jar -jar myapplication.jar运行它时， 它会引发以下异常.

onError:org.apache.http.impl.client.AbstractHttpClient
java.lang.NoClassDefFoundError: org/apache/http/HttpHost
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.getDeclaredMethods(Class.java:1975)
    at net.bytebuddy.description.method.MethodList$ForLoadedType.<init>(MethodList.java:106)
    at net.bytebuddy.description.type.TypeDescription$ForLoadedType.getDeclaredMethods(TypeDescription.java:985)
    at net.bytebuddy.implementation.MethodDelegation$MethodContainer$ForExplicitMethods.ofStatic(MethodDelegation.java:1037)
    at net.bytebuddy.implementation.MethodDelegation.to(MethodDelegation.java:247)
    at net.bytebuddy.implementation.MethodDelegation.to(MethodDelegation.java:226)
    at com.yiji.dtrace.agent.httpclient4.interceptor.HttpClient4Interceptors$1.transform(HttpClient4Interceptors.java:48)
    at net.bytebuddy.agent.builder.AgentBuilder$Transformer$Compound.transform(AgentBuilder.java:457)
    at net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:2791)
    at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:3081)
    at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
    at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:170)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:142)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:170)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:142)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:170)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:142)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.yjf.common.net.HttpUtil.<init>(HttpUtil.java:118)
    at com.yjf.common.net.HttpUtil.<init>(HttpUtil.java:81)
    at com.yjf.common.net.HttpUtil.<clinit>(HttpUtil.java:78)
    at com.daidai.dtrace.agent.test.Main.main(Main.java:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.http.HttpHost
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 60 more

这是我与代理相关的代码.

public class DTraceAgent {

    public static TypeDescription abstractHttpClientDescription() {
        return new TypeDescription.Latent("org.apache.http.impl.client.AbstractHttpClient",
                Modifier.PUBLIC|Modifier.ABSTRACT,
                TypeDescription.OBJECT,
                Arrays.asList(httpClientDescription()));
    }

    public static TypeDescription httpHostDescription() {
        return new TypeDescription.Latent("org.apache.http.HttpHost",
                Modifier.PUBLIC|Modifier.FINAL,
                TypeDescription.OBJECT,
                Arrays.asList(new TypeDescription.ForLoadedType(Cloneable.class),
                        new TypeDescription.ForLoadedType(Serializable.class)));
    }

    public static TypeDescription httpContextDescription() {
        return new TypeDescription.Latent("org.apache.http.protocol.HttpContext",
                getInterfaceModifiers(),
                TypeDescription.OBJECT,
                null);
    }

    public static TypeDescription httpRequestDescription() {
        return new TypeDescription.Latent("org.apache.http.HttpRequest",
                getInterfaceModifiers(),
                httpMessageDescription(),
                null);
    }

    public static void premain(String arguments, Instrumentation instrumentation) {
        new AgentBuilder.Default()
                //.withBinaryLocator(binaryLocatorFor(instrumentation))
                .withListener(DebugListener.getListener())
                .type(is(abstractHttpClientDescription()))
                .transform(new AgentBuilder.Transformer() {
                    public DynamicType.Builder transform(DynamicType.Builder builder,
                                                         TypeDescription typeDescription) {
                        return builder.method(named("execute")
                                .and(takesArguments(httpHostDescription(), httpRequestDescription(), httpContextDescription()))
                                .and(returns(named("org.apache.http.HttpResponse"))))
                                .intercept(MethodDelegation.to(HttpClientInterceptor4dot3Plus.class));
                    }
                }).installOn(instrumentation);
    }
}

public class HttpClientInterceptor4dot3Plus {

    public static CloseableHttpResponse doExecute(
            @SuperCall Callable<CloseableHttpResponse> client, @Argument(1)HttpRequest request
            ) throws Exception {
        StringBuilder builder = new StringBuilder(1024);
        if (request != null && request.getRequestLine() != null) {
            RequestLine requestLine = request.getRequestLine();
            builder.append(requestLine.getMethod()).append(" ").append(requestLine.getUri());
        }
        try (TraceScope scope = Trace.startSpanForEntry(builder.toString())) {
            Trace.spanType(Span.SPAN_TYPE_HTTP);
            try {
                return client.call();
            } catch (Exception e) {
                Trace.exception(e);
                throw e;
            }
        }
    }
}

public class DebugListener {
    public static AgentBuilder.Listener getListener() {
        return new AgentBuilder.Listener() {
            @Override
            public void onTransformation(TypeDescription typeDescription, DynamicType dynamicType) {
                System.err.println("onTransformation:" + typeDescription.getCanonicalName());
                try {
                    dynamicType.saveIn(new File("generated_classes"));
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

            @Override
            public void onIgnored(TypeDescription typeDescription) {
                //System.err.println("onIgored:" + typeDescription.getCanonicalName());
            }

            @Override
            public void onError(String typeName, Throwable throwable) {
                System.err.println("onError:" + typeName);
                throwable.printStackTrace();
            }

            @Override
            public void onComplete(String typeName) {
                //System.err.println("onComplete:" + typeName);
            }
        };
    }
}

我认为这个问题是由spring boot uber jar引导应用程序的方式引起的. Spring Boot提供了一个名为LaunchedURLClassLoader的专用类加载器，用于从uber jar加载与应用程序相关的类，而javaagent jar默认情况下是由系统类加载器加载的(如果我的理解是正确的话).因此，apache httpclient库(包含在uber jar中)对于系统类加载器是不可见的.

我试图向AgentBuilder提供BinaryLocator，但是它没有用.也许BinaryLocator的构造不正确.无论如何，适当的BinaryLocator可能是一种解决方案.

非常感谢您提供任何解决方案或建议.

其他信息可能会有所帮助:
春季启动版1.3.1.RELEASE
字节伙伴0.7.7，使用maven-assembly-plugin的jar-with-dependencies描述符打包到代理中 apache httpclient 4.3.2

解决方案

我通过两个步骤解决了这个问题:

  1. 使用Spring Boot专用的ClassLoader来限定为卸载类型:

    public static void premain(String arguments, Instrumentation instrumentation) {
  ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
  ClassFileLocator.Compound compound = new ClassFileLocator.Compound(ClassFileLocator.ForClassLoader.of(classLoader), ClassFileLocator.ForClassLoader.ofClassPath());
  TypeDescription delegator = TypePool.Default.of(compound).describe(delegatorClass).resolve();
  new AgentBuilder.Default()
    //.withBinaryLocator(binaryLocatorFor(instrumentation))
    .withListener(DebugListener.getListener())
    .type(is(abstractHttpClientDescription()))
    .transform(new AgentBuilder.Transformer() {
      @Override
      public DynamicType.Builder transform(DynamicType.Builder builder,
          TypeDescription typeDescription) {
        return builder.method(named("execute")
          .and(takesArguments(httpHostDescription(), httpRequestDescription(), httpContextDescription()))
          .and(returns(named("org.apache.http.HttpResponse"))))
          .intercept(MethodDelegation.to(delegator));
      }
  }).installOn(instrumentation);
}

  2. 将javaagent jar 打包到Spring Boot的 uber-jar 中，以便委托者类可以引用与受感知类相关的类.

此问题在 Byte Buddy的问题跟踪器中进行了更详细的讨论./p>

I'm trying to write a javaagent with Bytebuddy to intercept apache httpclient requests and I want to use this agent for spring boot application. The agent works fine when I start my test spring boot application from Idea (run the main method directly). However, when I package the application into a spring boot uber jar and run it using java -javaagent:myagent.jar -jar myapplication.jar, it throws the following exception.

onError:org.apache.http.impl.client.AbstractHttpClient
java.lang.NoClassDefFoundError: org/apache/http/HttpHost
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.getDeclaredMethods(Class.java:1975)
    at net.bytebuddy.description.method.MethodList$ForLoadedType.<init>(MethodList.java:106)
    at net.bytebuddy.description.type.TypeDescription$ForLoadedType.getDeclaredMethods(TypeDescription.java:985)
    at net.bytebuddy.implementation.MethodDelegation$MethodContainer$ForExplicitMethods.ofStatic(MethodDelegation.java:1037)
    at net.bytebuddy.implementation.MethodDelegation.to(MethodDelegation.java:247)
    at net.bytebuddy.implementation.MethodDelegation.to(MethodDelegation.java:226)
    at com.yiji.dtrace.agent.httpclient4.interceptor.HttpClient4Interceptors$1.transform(HttpClient4Interceptors.java:48)
    at net.bytebuddy.agent.builder.AgentBuilder$Transformer$Compound.transform(AgentBuilder.java:457)
    at net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:2791)
    at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:3081)
    at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
    at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:170)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:142)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
    at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:170)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:142)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:170)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:142)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.yjf.common.net.HttpUtil.<init>(HttpUtil.java:118)
    at com.yjf.common.net.HttpUtil.<init>(HttpUtil.java:81)
    at com.yjf.common.net.HttpUtil.<clinit>(HttpUtil.java:78)
    at com.daidai.dtrace.agent.test.Main.main(Main.java:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.http.HttpHost
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 60 more

and here is my agent related code.

public class DTraceAgent {

    public static TypeDescription abstractHttpClientDescription() {
        return new TypeDescription.Latent("org.apache.http.impl.client.AbstractHttpClient",
                Modifier.PUBLIC|Modifier.ABSTRACT,
                TypeDescription.OBJECT,
                Arrays.asList(httpClientDescription()));
    }

    public static TypeDescription httpHostDescription() {
        return new TypeDescription.Latent("org.apache.http.HttpHost",
                Modifier.PUBLIC|Modifier.FINAL,
                TypeDescription.OBJECT,
                Arrays.asList(new TypeDescription.ForLoadedType(Cloneable.class),
                        new TypeDescription.ForLoadedType(Serializable.class)));
    }

    public static TypeDescription httpContextDescription() {
        return new TypeDescription.Latent("org.apache.http.protocol.HttpContext",
                getInterfaceModifiers(),
                TypeDescription.OBJECT,
                null);
    }

    public static TypeDescription httpRequestDescription() {
        return new TypeDescription.Latent("org.apache.http.HttpRequest",
                getInterfaceModifiers(),
                httpMessageDescription(),
                null);
    }

    public static void premain(String arguments, Instrumentation instrumentation) {
        new AgentBuilder.Default()
                //.withBinaryLocator(binaryLocatorFor(instrumentation))
                .withListener(DebugListener.getListener())
                .type(is(abstractHttpClientDescription()))
                .transform(new AgentBuilder.Transformer() {
                    public DynamicType.Builder transform(DynamicType.Builder builder,
                                                         TypeDescription typeDescription) {
                        return builder.method(named("execute")
                                .and(takesArguments(httpHostDescription(), httpRequestDescription(), httpContextDescription()))
                                .and(returns(named("org.apache.http.HttpResponse"))))
                                .intercept(MethodDelegation.to(HttpClientInterceptor4dot3Plus.class));
                    }
                }).installOn(instrumentation);
    }
}

public class HttpClientInterceptor4dot3Plus {

    public static CloseableHttpResponse doExecute(
            @SuperCall Callable<CloseableHttpResponse> client, @Argument(1)HttpRequest request
            ) throws Exception {
        StringBuilder builder = new StringBuilder(1024);
        if (request != null && request.getRequestLine() != null) {
            RequestLine requestLine = request.getRequestLine();
            builder.append(requestLine.getMethod()).append(" ").append(requestLine.getUri());
        }
        try (TraceScope scope = Trace.startSpanForEntry(builder.toString())) {
            Trace.spanType(Span.SPAN_TYPE_HTTP);
            try {
                return client.call();
            } catch (Exception e) {
                Trace.exception(e);
                throw e;
            }
        }
    }
}

public class DebugListener {
    public static AgentBuilder.Listener getListener() {
        return new AgentBuilder.Listener() {
            @Override
            public void onTransformation(TypeDescription typeDescription, DynamicType dynamicType) {
                System.err.println("onTransformation:" + typeDescription.getCanonicalName());
                try {
                    dynamicType.saveIn(new File("generated_classes"));
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

            @Override
            public void onIgnored(TypeDescription typeDescription) {
                //System.err.println("onIgored:" + typeDescription.getCanonicalName());
            }

            @Override
            public void onError(String typeName, Throwable throwable) {
                System.err.println("onError:" + typeName);
                throwable.printStackTrace();
            }

            @Override
            public void onComplete(String typeName) {
                //System.err.println("onComplete:" + typeName);
            }
        };
    }
}

I think this problem is caused by the way spring boot uber jar bootstraps an application. Spring boot provides a dedicated class loader named LaunchedURLClassLoader to load application related classes from the uber jar, while javaagent jar is loaded by default system classloader (if my understanding is correct). So the apache httpclient lib (included in the uber jar) is not visible to the system classloader.

I tried to provider a BinaryLocator to the AgentBuilder, but it didn't work. Maybe the BinaryLocator was not correctly constructed. Anyway, a proper BinaryLocator maybe a possible solution.

Thanks a lot for any solutions or suggestions.

Other infomation may be helpful:
spring-boot version 1.3.1.RELEASE
byte-buddy 0.7.7, packaged into the agent using maven-assembly-plugin's jar-with-dependencies descriptorRef
apache httpclient 4.3.2

解决方案

I solved this problem through two steps:

  1. Use Spring Boot's dedicated ClassLoader to deligate to an unloaded type: 

    public static void premain(String arguments, Instrumentation instrumentation) {
  ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
  ClassFileLocator.Compound compound = new ClassFileLocator.Compound(ClassFileLocator.ForClassLoader.of(classLoader), ClassFileLocator.ForClassLoader.ofClassPath());
  TypeDescription delegator = TypePool.Default.of(compound).describe(delegatorClass).resolve();
  new AgentBuilder.Default()
    //.withBinaryLocator(binaryLocatorFor(instrumentation))
    .withListener(DebugListener.getListener())
    .type(is(abstractHttpClientDescription()))
    .transform(new AgentBuilder.Transformer() {
      @Override
      public DynamicType.Builder transform(DynamicType.Builder builder,
          TypeDescription typeDescription) {
        return builder.method(named("execute")
          .and(takesArguments(httpHostDescription(), httpRequestDescription(), httpContextDescription()))
          .and(returns(named("org.apache.http.HttpResponse"))))
          .intercept(MethodDelegation.to(delegator));
      }
  }).installOn(instrumentation);
}

  2. Package the javaagent jar into Spring Boot's uber-jar such that the delegator class can reference classes related to the intecepted classes.

This issue is discussed in greater detail in Byte Buddy's issue tracker.

这篇关于无法使用javaagent为Spring Boot uber jar应用程序检测apache httpclient的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
Java开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆