显示 RTP MJPEG [英] Display RTP MJPEG

问题描述

我正在寻找一种使用 JavaFx 显示 RTP JPEG 流的解决方案.我可以从文件中显示 jpeg 并接收 RTP JPEG 流并将其拆分以识别 RFC2435但我不知道如何将我的 JPEG 数组转换为可显示的图像.我不想自己实现 JPEG 解码器.有什么想法吗?

I'm looking for a solution to display a RTP JPEG stream with JavaFx. I can display jpeg from a file and receive RTP JPEG stream and split it to identify all parameters and data as specify in RFC2435 But I don't know how to convert my JPEG arrays to a displayable Image. I dont want to implement a JPEG Decoder by myself. Any idea?

推荐答案

利用 JavaFX 的内置 jpeg 解码器，它应该能够解码 图像构造函数.

Leverage JavaFX's built-in jpeg decoder, which should be able to decode the jpeg images in the Image constructor.

class MJPEGViewer extends ImageView {
  MJPEGViewer() {
    // setup a thread which processes the input stream.
    // the processing thread invokes onNewData for each new frame.
  }

  private void onNewData(byte[] jpegData) {
    imageView.set(
      new Image(
        new ByteArrayInputStream(jpegData);
      )
    );
  }
}

jpegData 是一个字节数组，假定包含从 RTP 流中提取的帧的 JFIF 数据.

The jpegData is a byte array presumed to contain the JFIF data for a frame extracted from the RTP stream.

可执行示例

这是一个 mjpeg 电影播放器​​，可以播放来自:http://inst.eecs.berkeley.edu/~ee122/sp06/ProgAsgns/movie.Mjpeg

This is an mjpeg movie player that plays the movie from: http://inst.eecs.berkeley.edu/~ee122/sp06/ProgAsgns/movie.Mjpeg

基于来自编程的视频流类作业 5:使用 RTSP 和 RTP 流式传输视频(我希望这不是您的家庭作业课).

Based on the video stream class from Programming Assignment 5: Streaming Video with RTSP and RTP (I do hope this is not your homework assignment class).

根据视频流类描述，它是专有的 MJPEG 格式"，因此您需要根据 RFC2435 对符合标准的格式进行自己的解码.

According to the video stream class description, it is a "proprietary MJPEG format", so you will need to do your own decoding of your standards compliant format according RFC2435.

播放器可以工作，但在正确解码我尚未调查的 JPEG 时确实存在问题.专有 MJPEG 格式"示例影片中的 JPEG 未正确编码，或者 JavaFX JPEG 编解码器在解码帧时出错.可见伪影是您可以看到图像，但图像没有正确着色(具有粉红色阴影).它可能是 RT-14647 的一个实例JPEG 图像显示不正确，因为视频中的粉红色阴影在错误中显示的错误解码的 JPEG 中看起来相同.在下面的示例代码呈现的视频的屏幕截图中，您可以清楚地看到粉红色阴影.该错误仅影响一些 JPEG 图像(我与 JavaFX 一起使用的绝大多数 JPEG 图像都显示良好).因此，您只需要尝试使用您的视频流，看看 JavaFX jpeg 解码器是否为您正确解码了 jpeg 图像.

The player works, but does have an issue correctly decoding the JPEGs which I have not investigated. Either the JPEGs in the "proprietary MJPEG format" sample movie are not correctly encoded, or the JavaFX JPEG codec has errors decoding frames. The visible artifact is that you can see the image, but the image is not correctly colored (has a pink shade). It is likely an instance of RT-14647 Incorrect display of JPEG images as the pink shade in the video looks the same in the incorrectly decoded JPEGs shown in the bug. You can clearly see the pink shade in the screenshot of the video rendered by the sample code below. The bug only effects some JPEG images (the great majority of JPEG image I have used with JavaFX have displayed fine). So you will just need to try with your video stream to see if the JavaFX jpeg decoder correctly decodes the jpeg images for you.

与其每次都替换 imageview 中的 Image，使用 WritableImage 并直接更新它的像素缓冲区，但蛮力替换图像方法对我来说似乎工作正常.

Rather than replacing the Image in the imageview each time, it is probably more efficient to use a WritableImage and update it's pixel buffer directly, but the brute force replace image method seemed to work OK for me.

import javafx.animation.*;
import javafx.application.Application;
import javafx.event.*;
import javafx.geometry.Pos;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.image.*;
import javafx.scene.layout.*;
import javafx.stage.Stage;
import javafx.util.Duration;

import java.io.*;
import java.util.Arrays;

public class MjpegPlayer extends Application {
  public static void main(String[] args) { Application.launch(MjpegPlayer.class); }

  // ADJUST THIS LOCATION TO SET THE LOCATION OF YOUR MOVIE FILE!!
  private static final String MOVIE_FILE = "/Users/lilyshard/dev/playfx/src/fruits/movie.Mjpeg";

  private VideoStream vs;

  @Override public void start(Stage stage) throws Exception {
    vs = new VideoStream(MOVIE_FILE);

    final ImageView viewer   = new ImageView();
    final Timeline  timeline = createTimeline(viewer);

    VBox layout = new VBox(20);
    layout.setStyle("-fx-background-color: cornsilk;");
    layout.setAlignment(Pos.CENTER);
    layout.getChildren().setAll(
      viewer,
      createControls(timeline)
    );

    stage.setScene(new Scene(layout, 400, 400));
    stage.show();

    timeline.play();
  }

  private Timeline createTimeline(final ImageView viewer) {
    final Timeline timeline = new Timeline();
    final byte[] buf = new byte[15000];

    timeline.getKeyFrames().setAll(
      new KeyFrame(Duration.ZERO, new EventHandler<ActionEvent>() {
        @Override public void handle(ActionEvent event) {
          try {
            int len = vs.getnextframe(buf);
            if (len == -1) {
              timeline.stop();
              return;
            }
            viewer.setImage(
              new Image(
                new ByteArrayInputStream(
                  Arrays.copyOf(buf, len)
                )
              )
            );
          } catch (Exception e) {
            e.printStackTrace();
          }
        }
      }),
      new KeyFrame(Duration.seconds(1.0/24))
    );
    timeline.setCycleCount(Timeline.INDEFINITE);

    return timeline;
  }

  private HBox createControls(final Timeline timeline) {
    Button play = new Button("Play");
    play.setOnAction(new EventHandler<ActionEvent>() {
      @Override
      public void handle(ActionEvent event) {
        timeline.play();
      }
    });

    Button pause = new Button("Pause");
    pause.setOnAction(new EventHandler<ActionEvent>() {
      @Override
      public void handle(ActionEvent event) {
        timeline.pause();
      }
    });

    Button restart = new Button("Restart");
    restart.setOnAction(new EventHandler<ActionEvent>() {
      @Override
      public void handle(ActionEvent event) {
        try {
          timeline.stop();
          vs = new VideoStream(MOVIE_FILE);
          timeline.playFromStart();
        } catch (Exception e) {
          e.printStackTrace();
        }
      }
    });

    HBox controls = new HBox(10);
    controls.setAlignment(Pos.CENTER);
    controls.getChildren().setAll(
      play,
      pause,
      restart
    );
    return controls;
  }
}

class VideoStream {

  FileInputStream fis; //video file
  int frame_nb; //current frame nb

  public VideoStream(String filename) throws Exception{

    //init variables
    fis = new FileInputStream(filename);
    frame_nb = 0;
  }

  public int getnextframe(byte[] frame) throws Exception
  {
    int length = 0;
    String length_string;
    byte[] frame_length = new byte[5];

    //read current frame length
    fis.read(frame_length,0,5);

    //transform frame_length to integer
    length_string = new String(frame_length);
    try {
      length = Integer.parseInt(length_string);
    } catch (Exception e) {
      return -1;
    }

    return(fis.read(frame,0,length));
  }
}

更新

我尝试在 Windows 7 上使用 Java 8u20 早期访问版本 11 再次运行该程序，并且视频播放良好，没有任何粉红色调，所以我猜现在已经在以后的 Java 版本中修复了导致该问题的原因.

I tried running this program again using Java 8u20 early access build 11 on a Windows 7, and the video played back fine without any pink tinge, so I guess that whatever was causing that issue has now been fixed in a later Java build.

这篇关于显示 RTP MJPEG的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！