Java动态加载和卸载.java文件，垃圾回收？ [英] Java loading and unloading .java files dynamically, garbage collection?
问题描述
我正在创建一个将运行很长一段时间的java应用程序，这需要更新的功能而不关闭。我决定提供这个更新的功能，以.java文件（从数据库中拉取为字节数组）的形式加载它们在内存中编译并实例化。如果你有一个更好的方式我所有的耳朵。
我遇到的问题是内存占用增加轻微加载这些脚本的每个周期，当我在人工环境中做一些测试。
注意：这实际上是我第一次做这样的事情或者用java。我在C＃中完成了这样的事情，在加载和卸载.cs文件，还有内存足迹问题，以解决我将它们加载到一个单独的appdomain，当我重新编译的文件，我刚卸载的appdomain，并创建了一个新的一个。
入口点
方法，我用来模拟长时间使用后的内存占用（许多重新编译周期）。我运行这个短暂的时间，它很快就吃了500MB +。
这只有在临时目录中有两个虚拟脚本。
public static void main（String [] args）throws Exception {
for（int i = 0; i <1000; i ++）{
Container [] containers = getScriptContainers（）;
Script [] scripts = compileScripts（containers）;
for（Script s：scripts）s.Begin（）;
Thread.sleep（1000）;
}
}
收集脚本列表>
这是我用来收集脚本文件列表的临时方法。在生产期间，这些将实际上作为字节数组加载一些其他信息，如数据库中的类名。
@Deprecated
private static Container [] getScriptContainers（）throws IOException {
文件root =新文件（C：\\Scripts\\）;
File [] files = root.listFiles（）;
List<容器> containers = new ArrayList<>（）;
for（File f：files）{
String [] tokens = f.getName（）.split（\\。（？= [^ \\。] + $） ）;
if（f.isFile（）&& tokens [1] .equals（java））{
byte [] fileBytes = Files.readAllBytes（Paths.get（f.getAbsolutePath ））;
containers.add（new Container（tokens [0]，fileBytes））;
}
}
return containers.toArray（new Container [0]）;
}
容器类
这是简单的容器类。
public class Container {
private String className;
private byte [] classFile;
public容器（String name，byte [] file）{
className = name;
classFile = file;
}
public String getClassName（）{
return className;
}
public byte [] getClassFile（）{
return classFile;
}
}
编译脚本
这是编译.java文件并将其实例化为Script对象的实际方法。
private static Script [] compileScripts（Container [] containers）throws InstantiationException，IllegalAccessException，ClassNotFoundException {
List& ClassFile> sourceScripts = new ArrayList<>（）;
for（Container c：containers）
sourceScripts.add（new ClassFile（c.getClassName（），c.getClassFile（）））;
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler（）;
JavaFileManager manager = new MemoryFileManager（compiler.getStandardFileManager（null，null，null））;
compiler.getTask（null，manager，null，null，null，sourceScripts）.call（）;
List<脚本> compiledScripts = new ArrayList<>（）;
for（Container c：containers）
compiledScripts.add（（Script）manager.getClassLoader（null）.loadClass（c.getClassName（））.newInstance（））;
return（Script []）compiledScripts.toArray（new Script [0]）;
}
MemoryFileManager类
这是我为编译器创建的自定义 JavaFileManager 实现，以便我可以将输出存储在内存中，而不是物理的.class文件中。 p>
public class MemoryFileManager extends ForwardingJavaFileManager< JavaFileManager> {
private HashMap< String，ClassFile> classes = new HashMap<>（）;
public MemoryFileManager（StandardJavaFileManager standardManager）{
super（standardManager）;
}
@Override
public ClassLoader getClassLoader（位置位置）{
return new SecureClassLoader（）{
@Override
protected Class< ？ > findClass（String className）throws ClassNotFoundException {
if（classes.containsKey（className））{
byte [] classFile = classes.get（className）.getClassBytes（）;
return super.defineClass（className，classFile，0，classFile.length）;
} else throw new ClassNotFoundException（）;
}
};
}
@Override
public ClassFile getJavaFileForOutput（Location location，String className，Kind kind，FileObject sibling）{
if（classes.containsKey（className））return classes.get（className）;
else {
ClassFile classObject = new ClassFile（className，kind）;
classes.put（className，classObject）;
return classObject;
}
}
}
ClassFile类
这是我的多用途 SimpleJavaFileObject 实现，用于存储源.java文件和编译.class文件在内存中。
public class ClassFile extends SimpleJavaFileObject {
private byte [ ] 资源;
protected final ByteArrayOutputStream compiled = new ByteArrayOutputStream（）;
public ClassFile（String className，byte [] contentBytes）{
super（URI.create（string：///+ className.replace（'。'，'/'） + Kind.SOURCE.extension），Kind.SOURCE）;
source = contentBytes;
}
public ClassFile（String className，CharSequence contentCharSequence）throws UnsupportedEncodingException {
super（URI.create（string：///+ className.replace（'。 ，'/'）+ Kind.SOURCE.extension），Kind.SOURCE）;
source =（（String）contentCharSequence）.getBytes（UTF-8）;
}
public ClassFile（String className，Kind kind）{
super（URI.create（string：///+ className.replace（'。'，' /'）+ kind.extension），kind）;
}
public byte [] getClassBytes（）{
return compiled.toByteArray（）;
}
public byte [] getSourceBytes（）{
return source;
}
@Override
public CharSequence getCharContent（boolean ignoreEncodingErrors）throws UnsupportedEncodingException {
return new String（source，UTF-8）;
}
@Override
public OutputStream openOutputStream（）{
return compiled;
}
}
脚本界面
最后是简单的脚本接口。
public void Begin（）throws Exception;}
在编程方面我还是新的，堆栈一会儿找到一些解决方案，我遇到的小问题，这是我第一次提出一个问题，所以我道歉，如果我包括太多的信息或如果这太长了;
你似乎在使用应用程序的默认类加载器来加载编译的类 - 这使得类不可能被垃圾收集。
所以你必须为您刚编译的课程创建单独的类加载器。这是应用程序服务器做的。
但是，即使您为编译的类使用单独的类加载器，通过垃圾收集，因为类加载器和它加载的所有类都不符合垃圾收集，只要这些类的任何一个单一的实例被引用到任何其他地方（即你的应用程序的其余部分）。
这被称为类加载器泄漏，与应用程序服务器的常见问题，导致重新部署使用更多的内存，最终失败。诊断和修复类加载器泄漏可能非常棘手;该文章有所有的细节。
I am in the process of creating a java application that will be running for long periods of time which requires updated functionality without shutting down. I've decided to provide this updated functionality by loading it in the form of .java files (pulled as a byte array from a database) which are compiled in memory and instantiated. If you have a better way I am all ears.
The problem I have run in to is that memory footprint increases slightly with each cycle of loading these "scripts" when I do some testing in an artificial environment.
Note: This is actually my first time doing something like this or much at all with java. I had accomplished something like this before in C# with loading and unloading .cs files and also had memory footprint issues there... to solve that I loaded them into a separate appdomain and when I recompiled the files I just unloaded that appdomain and created a new one.
Entry point
This is the entry method that I am using to simulate the memory footprint after long periods of use (many recompile cycles). I run this for a short period of time and it quickly eats up 500MB+.
This is only with two dummy scripts in the temporary directory.
public static void main( String[ ] args ) throws Exception {
for ( int i = 0; i < 1000; i++ ) {
Container[ ] containers = getScriptContainers( );
Script[ ] scripts = compileScripts( containers );
for ( Script s : scripts ) s.Begin( );
Thread.sleep( 1000 );
}
}
Collecting a list of scripts (temporary)
This is the temporary method I am using to collect a list of the script files. During production these will actually be loaded as byte arrays with some other information like the class name from a database.
@Deprecated
private static Container[ ] getScriptContainers( ) throws IOException {
File root = new File( "C:\\Scripts\\" );
File[ ] files = root.listFiles( );
List< Container > containers = new ArrayList<>( );
for ( File f : files ) {
String[ ] tokens = f.getName( ).split( "\\.(?=[^\\.]+$)" );
if ( f.isFile( ) && tokens[ 1 ].equals( "java" ) ) {
byte[ ] fileBytes = Files.readAllBytes( Paths.get( f.getAbsolutePath( ) ) );
containers.add( new Container( tokens[ 0 ], fileBytes ) );
}
}
return containers.toArray( new Container[ 0 ] );
}
Container class
This is the simple container class.
public class Container {
private String className;
private byte[ ] classFile;
public Container( String name, byte[ ] file ) {
className = name;
classFile = file;
}
public String getClassName( ) {
return className;
}
public byte[ ] getClassFile( ) {
return classFile;
}
}
Compiling the scripts
This is the actual method that compiles the .java files and instantiates them into Script objects.
private static Script[ ] compileScripts( Container[ ] containers ) throws InstantiationException, IllegalAccessException, ClassNotFoundException {
List< ClassFile > sourceScripts = new ArrayList<>( );
for ( Container c : containers )
sourceScripts.add( new ClassFile( c.getClassName( ), c.getClassFile( ) ) );
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler( );
JavaFileManager manager = new MemoryFileManager( compiler.getStandardFileManager( null, null, null ) );
compiler.getTask( null, manager, null, null, null, sourceScripts ).call( );
List< Script > compiledScripts = new ArrayList<>( );
for ( Container c : containers )
compiledScripts.add( ( Script )manager.getClassLoader( null ).loadClass( c.getClassName( ) ).newInstance( ) );
return ( Script[ ] )compiledScripts.toArray( new Script[ 0 ] );
}
MemoryFileManager class
This is the custom JavaFileManager implementation that I created for the compiler so that I can store the output in memory rather than in physical .class files.
public class MemoryFileManager extends ForwardingJavaFileManager< JavaFileManager > {
private HashMap< String, ClassFile > classes = new HashMap<>( );
public MemoryFileManager( StandardJavaFileManager standardManager ) {
super( standardManager );
}
@Override
public ClassLoader getClassLoader( Location location ) {
return new SecureClassLoader( ) {
@Override
protected Class< ? > findClass( String className ) throws ClassNotFoundException {
if ( classes.containsKey( className ) ) {
byte[ ] classFile = classes.get( className ).getClassBytes( );
return super.defineClass( className, classFile, 0, classFile.length );
} else throw new ClassNotFoundException( );
}
};
}
@Override
public ClassFile getJavaFileForOutput( Location location, String className, Kind kind, FileObject sibling ) {
if ( classes.containsKey( className ) ) return classes.get( className );
else {
ClassFile classObject = new ClassFile( className, kind );
classes.put( className, classObject );
return classObject;
}
}
}
ClassFile class
This is my multi-purpose SimpleJavaFileObject implementation that I use to store the source .java files and the compiled .class files in memory.
public class ClassFile extends SimpleJavaFileObject {
private byte[ ] source;
protected final ByteArrayOutputStream compiled = new ByteArrayOutputStream( );
public ClassFile( String className, byte[ ] contentBytes ) {
super( URI.create( "string:///" + className.replace( '.', '/' ) + Kind.SOURCE.extension ), Kind.SOURCE );
source = contentBytes;
}
public ClassFile( String className, CharSequence contentCharSequence ) throws UnsupportedEncodingException {
super( URI.create( "string:///" + className.replace( '.', '/' ) + Kind.SOURCE.extension ), Kind.SOURCE );
source = ( ( String )contentCharSequence ).getBytes( "UTF-8" );
}
public ClassFile( String className, Kind kind ) {
super( URI.create( "string:///" + className.replace( '.', '/' ) + kind.extension ), kind );
}
public byte[ ] getClassBytes( ) {
return compiled.toByteArray( );
}
public byte[ ] getSourceBytes( ) {
return source;
}
@Override
public CharSequence getCharContent( boolean ignoreEncodingErrors ) throws UnsupportedEncodingException {
return new String( source, "UTF-8" );
}
@Override
public OutputStream openOutputStream( ) {
return compiled;
}
}
Script interface
And lastly the simple Script interface.
public interface Script {
public void Begin( ) throws Exception;
}
I'm still kind of new when it comes to programming and I have used the stack for a while to find some solutions to small problems I have encountered, this is my first time asking a question so I apologize if I have included too much information or if this is too long; I just wanted to make sure I was thorough.
You seem to be using the application's default classloader to load the compiled classes - that makes it impossible for the classes to be garbage collected.
So you have to create a separate classloader for your freshly compiled classes. This is how app servers do it.
However, even if you use a separate classloader for your compiled classes, it can be tricky to get those classes to be picked up by garbage collection, because the classloader and all the classes it loaded are not eligible for garbage collcetion as long as a single instance of any of those classes is referred to anywhere else (i.e. the rest of your application).
This is known as a classloader leak and a common problem with appservers, causing redeployments to use ever more memory and eventually fail. Diagnosing and fixing a classloader leak can be very tricky; the article has all the details.
这篇关于Java动态加载和卸载.java文件，垃圾回收？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
