Spring Batch：如何处理多行日志文件 [英] Spring Batch: How to process multi-line log files

问题描述

我正在尝试使用Spring Batch将日志文件的内容导入数据库。

I am trying to import the contents of a log file into a database using Spring Batch.

我目前正在使用FlatFileItemReader，但遗憾的是很多日志条目那没有抓住。两个主要问题是：

I am currently using a FlatFileItemReader, but there are unfortunately many log entries that doesn't catch. The two main problems are:

1. 包含多行JSON字符串的行：

1. Lines that contain multi-line JSON Strings:

2012-03-22 11:47:35,307  DEBUG main someMethod(SomeClass.java:56): Do Something(18,true,null,null,null): my.json = '{
    "Foo":"FooValue",
    "Bar":"BarValue",
    ... etc
}'

包含堆栈跟踪的行

Lines that contain stack traces

2012-03-22 11:47:50,596  ERROR main com.meetup.memcached.SockIOPool.createSocket(SockIOPool.java:859): No route to host
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.Net.connect0(Native Method)
        at sun.nio.ch.Net.connect(Net.java:364)
        at sun.nio.ch.Net.connect(Net.java:356)
        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:623)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:92)
        at com.meetup.memcached.SockIOPool$SockIO.getSocket(SockIOPool.java:1703)
        at com.meetup.memcached.SockIOPool$SockIO.<init>(SockIOPool.java:1674)
        at com.meetup.memcached.SockIOPool.createSocket(SockIOPool.java:850)
        at com.meetup.memcached.SockIOPool.populateBuckets(SockIOPool.java:737)
        at com.meetup.memcached.SockIOPool.initialize(SockIOPool.java:695)

基本上，我需要FlatFileItemReader继续读取，直到达到下一个时间戳，同时聚合之前的所有线路。之前是否有任何此类事情已经完成（在Spring Batch中）

Basically, I need the FlatFileItemReader to keep reading until it reaches the next timestamp, while aggregating all the lines before that. Has any such thing been done before (in Spring Batch)

推荐答案

现在在 Spring Batch文档解决了这个用例。

There's now an FAQ in the Spring Batch documentation addressing this use case.

这篇关于Spring Batch：如何处理多行日志文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！