杰克逊SAX解析器在解析巨大的JSON时引发异常 [英] Jackson SAX parser throws exception when parsing huge JSON
问题描述
我正在尝试用Jackson Jackson解析器实现JSON数组迭代器(请不要问为什么). 我的应用程序应该可以处理大文件(最多5 MiB)，这是一个问题.
I'm trying to implement JSON array iterator with Jackson SAX parser (please, don't ask why). My app should work with huge files (up to 5 MiB), and that's a problem.
这就是我初始化JsonParser并调用迭代器创建的方式. 我创建了InputStream，并使用JSON初始化了它，并将其放置在\ raw文件夹中.
That's how I initialize JsonParser and call iterator creation. I create InputStream initialized with JSON placed in \raw folder.
private JsonArrayIterator getIterator(String needle) throws IOException
{
InputStream inputStream = getApplicationContext().getResources().openRawResource(R.raw.products);
inputStream.mark(-1);
try {
JsonParser jsonParser = createJsonParser(inputStream);
// Some unrelated code
return new JsonArrayIterator(jsonParser);
} catch (IOException e) {
e.printStackTrace();
} finally {
inputStream.close();
}
return null;
}
这是我的迭代器类.
private Object parseNextObject() throws IOException {
// I'm not using ObjectMapper because of reasons
HashMap nextObject = new HashMap();
int objectsCount = 1;
while (objectsCount > 0) {
JsonToken currentToken = currentParser.nextValue();
if(currentToken == JsonToken.START_OBJECT) {
++objectsCount;
}
else if(currentToken == JsonToken.END_OBJECT) {
--objectsCount;
}
else if(currentToken == JsonToken.START_ARRAY) {
String currentName = currentParser.getCurrentName();
ArrayList list = new ArrayList(100);
JsonArrayIterator it = new JsonArrayIterator(currentParser);
while (it.hasNext()) {
list.add(it.next());
}
nextObject.put(currentName, list);
}
else {
// Here exception is throwed
nextObject.put(currentParser.getCurrentName(), currentParser.getText());
}
}
currentParser.nextToken(); // Skip END_OBJECT
return nextObject;
}
似乎很不错……哦，等等.
It seem to work perfe... OH, WAIT.
在一个巨大的文件中，我有3个部分(命名为数组). 它成功地首先解析(一个很小的解析，不到1000个字节).但是我接下来不能解析.
I have 3 sections (named arrays) in some huge file. It successfully parses first (a tiny one, under 1000 bytes). But I can't parse next.
下一个数组具有带有简单对象的嵌套数组(像这样):
Next array has nested array with simple objects (like this):
{
"properties":[
{
"id":"1",
"title":"\u0426\u0432\u0435\u0442",
"values":[
{
"id":"1_2",
"title":"\u0427\u0435\u0440\u043d\u044b\u0439"
},
{
"id":"1_5005",
"title":"\u0417\u0435\u043b\u0435\u043d\u044b\u0439"
},
{
"id":"1_5006",
"title":"\u0421\u0435\u0440\u044b\u0439"
}
]
}
]
}
对于值中的一个对象，我称为nextObject.put. currentParser.getCurrentName()成功运行并返回正确的字符串，但是currentParser.getText()失败.这不是JSON问题:可以完美地映射到iOS.这不是对象或迭代器创建的问题:我可以删除解析器引发异常的地方，但是它将在同一位置失败.
For one object in values I call nextObject.put. currentParser.getCurrentName() is runs successfully and returns correct string, but currentParser.getText() fails. It's not a JSON problem: it perfectly being mapped on iOS. It's not an object or iterator creation problem: I can delete that where parser throws exception, but it will fail in the same place.
这是堆栈跟踪:
05-15 21:57:28.617: ERROR/AndroidRuntime(14758): FATAL EXCEPTION: main
java.lang.NullPointerException: asset
at android.content.res.AssetManager.readAsset(Native Method)
at android.content.res.AssetManager.access$700(AssetManager.java:36)
at android.content.res.AssetManager$AssetInputStream.read(AssetManager.java:576)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.loadMore(UTF8StreamJsonParser.java:174)
at com.fasterxml.jackson.core.base.ParserBase.loadMoreGuaranteed(ParserBase.java:425)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:1930)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:1911)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:276)
at ru.studiomobile.JsonArrayIterator.parseNextObject(JsonArrayIterator.java:57)
at ru.studiomobile.JsonArrayIterator.next(JsonArrayIterator.java:73)
at ru.studiomobile.JsonArrayIterator.parseNextObject(JsonArrayIterator.java:47)
at ru.studiomobile.JsonArrayIterator.next(JsonArrayIterator.java:73)
at ru.studiomobile.MainActivity$2.onClick(MainActivity.java:81)
at android.view.View.performClick(View.java:3127)
at android.view.View$PerformClick.run(View.java:12025)
at android.os.Handler.handleCallback(Handler.java:587)
at android.os.Handler.dispatchMessage(Handler.java:92)
at android.os.Looper.loop(Looper.java:132)
at android.app.ActivityThread.main(ActivityThread.java:4126)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:491)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:844)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:602)
at dalvik.system.NativeStart.main(Native Method)
我注意到有一个叫做Utf8StreamJsonParser的东西.它有一个名为_inputEnd的字段等于4000(为什么是4000?).当其他字段_inputPtr变大时，它将引发异常.我该如何处理?我尝试使用具有预定义块大小的BufferedInputStream而不是InputStream，但是它没有效果.
I noticed that there are something called Utf8StreamJsonParser. It has a field named _inputEnd equal 4000 (why 4000?). When other field, _inputPtr, become bigger it throw an exception. How do I can handle it? I tried to use BufferedInputStream with predefined block size instead of InputStream but it had no effect.
有关某些行的信息
47: list.add(it.next());
73: return parseNextObject();
75: e.printStackTrace();
没什么特别的.
推荐答案
这是一个明显的问题:我忘了在创建解析器后关闭流...
It was an obvious problem: I forgot that I close stream after creating parser...
这篇关于杰克逊SAX解析器在解析巨大的JSON时引发异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
