在语音进行过程中突出显示文本 [英] Highlighting the Text while Speech is Progressing

问题描述

我正在开发一个应用程序，其中的textview由String和两个按钮组成.当我单击语音"按钮时，文本将转换为语音.但是我想在语音运行时突出显示该单词.

I'm developing an App in which I've textview consists of String and two buttons. When I click the speak button, the text gets converted to speech. But I want to Highlight the word while speech is running.

请检查以下链接上的我的应用"屏幕截图.

Please check the My app screenshot on this below link.

这是我的文字到语音初始化:

This is My text to speech initialization:

textToSpeech = new TextToSpeech(this, new TextToSpeech.OnInitListener() {

        @Override
        public void onInit(int status) {

            if (status == TextToSpeech.SUCCESS) {
                result = textToSpeech.setLanguage(Locale.ENGLISH);
                textToSpeech.setOnUtteranceProgressListener(new UtteranceProgressListener() {
                    @Override
                    public void onStart(String utteranceId) {
                        Log.d(utteranceId, "TTS start");}

                    @Override
                    public void onDone(String utteranceId) {
                        Log.d(utteranceId, "TTS done");}

                    @Override
                    public void onError(String utteranceId) {
             });
            } else {
                Toast.makeText(getApplicationContext(), "Feature is not Available", Toast.LENGTH_SHORT).show();
            }
        }
    });

以及其他代码:

private void speak() {
 if (result == TextToSpeech.LANG_MISSING_DATA || result == TextToSpeech.LANG_NOT_SUPPORTED) {
        Toast.makeText(getApplicationContext(), "Feature is not Available", Toast.LENGTH_SHORT).show();
    } else {
        textToSpeech.setPitch(1f);
        textToSpeech.setSpeechRate(0.8f);
        HashMap<String, String> params = new HashMap<>();
        params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, "utteranceId");
        textToSpeech.speak(getString(R.string.storytxt), TextToSpeech.QUEUE_FLUSH, params);

    }
}

@Override
protected void onDestroy() {
    super.onDestroy();
    if (textToSpeech != null) {
        textToSpeech.shutdown();
    }
}

直到这里我都没遇到任何问题.现在，我要突出显示文本.我不知道该怎么做.我到处搜寻都仍然没有线索.

Till here I didn't get any problem. Now I want to highlight the text. I don't know how to do it.I've searched everywhere still got no lead on this.

我将字符串存储在String.xml中.

I stored the string in String.xml.

推荐答案

对于Android API 26及更高版本以及支持onRangeStart(在本例中为Google TTS)的TTS引擎:

For Android API 26 and above AND a TTS engine that supports onRangeStart (in this case, Google TTS):

public class MainActivity extends AppCompatActivity implements TextToSpeech.OnInitListener {

    TextToSpeech tts;

    String sentence = "The Quick Brown Fox Jumps Over The Lazy Dog.";

    TextView textView;

    @Override
    protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        textView = findViewById(R.id.textView);
        textView.setText(sentence);
        tts = new TextToSpeech(this, this);

    }

    // TextToSpeech.OnInitListener (for our purposes, the "main method" of this activity)
    public void onInit(int status) {

        tts.setOnUtteranceProgressListener(new UtteranceProgressListener() {

            @Override
            public void onStart(String utteranceId) {
                Log.i("XXX", "utterance started");
            }

            @Override
            public void onDone(String utteranceId) {
                Log.i("XXX", "utterance done");
            }

            @Override
            public void onError(String utteranceId) {
                Log.i("XXX", "utterance error");
            }

            @Override
            public void onRangeStart(String utteranceId,
                                     final int start,
                                     final int end,
                                     int frame) {
                Log.i("XXX", "onRangeStart() ... utteranceId: " + utteranceId + ", start: " + start
                        + ", end: " + end + ", frame: " + frame);

                // onRangeStart (and all UtteranceProgressListener callbacks) do not run on main thread
                // ... so we explicitly manipulate views on the main thread:
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {

                        Spannable textWithHighlights = new SpannableString(sentence);
                        textWithHighlights.setSpan(new ForegroundColorSpan(Color.YELLOW), start, end, Spanned.SPAN_INCLUSIVE_INCLUSIVE);
                        textView.setText(textWithHighlights);

                    }
                });

            }

        });

    }

    public void startClicked(View ignored) {

        tts.speak(sentence, TextToSpeech.QUEUE_FLUSH, null, "doesn't matter yet");

    }

}

//--------------------------------------------- ----------------------

// -------------------------------------------------------------------

Android API 25及以下版本:

Android API 25 and below:

从理论上讲，实现这一目标的最直观的方法是:

In theory, the most intuitive way of accomplish this would be to:

1)将字符串分成段

2)检测何时/正在说出每一句话

2) Detect when each piece has been/is being spoken

3) 突出显示相应内容

但是，不幸的是，当使用实时生成语音输出的Android TextToSpeech类时，您能够精确检测到进度(使用UtteranceProgressListener)的最小语音单位是发声(无论您决定发送到TTS的任何字符串)-不一定是单词.

However, unfortunately, when using the Android TextToSpeech class where the speech output is generated in real-time, the smallest unit of speech that you are able to precisely detect the progress of (using UtteranceProgressListener) is an utterance (whatever string you decided to send to the TTS) -- not necessarily a word.

没有一种机制可以让您简单地发送多字字符串作为发声，然后以某种方式准确地检测出何时说出每个字.

There is no mechanism whereby you can simply send a multi-word string as an utterance, and then somehow detect exactly when each word has been spoken.

因此，为了(轻松)按顺序突出显示每个单词，您将必须:

Therefore, in order to (easily) highlight each word in order, you would have to either:

A)将每个单词作为单个发音单独发送到TTS(但这会导致发音不连贯)，或者

A) Send each word to the TTS individually as a single utterance (but this will cause disjointed pronunciation), or

B)而是突出显示逐句，将每个句子作为发声发送(最简单的方法，但不是您想要的行为).

B) Highlight sentence-by-sentence instead, sending each sentence as an utterance (easiest method, but not your desired behaviour).

如果您真的坚持要实现逐字突出显示效果，那么我能想到的唯一方法(使用Android TextToSpeech)是使用句子大小的语音，而不是使用speak()，而是使用synthesizeToFile ()...，然后使用某种媒体播放器或声音播放器播放语音...以某种方式根据第n个字相对于总音频文件长度的位置近似估算高亮显示的时间.因此，例如，如果句子的长度为10个单词，而文件的完成率是30％，则您将突出显示第4个单词.这将是困难且不精确的，但在理论上是可能的.

If you really insist on achieving a word-by-word highlighting effect, the only way I can think of (using Android TextToSpeech) is to use sentence-size utterances, but instead of using speak(), use synthesizeToFile()... and then use a media player or sound player of some sort to play the speech back... somehow approximating the timing of the highlights in terms of where the nth word lies relative to the total audio file length. So, for example, if the sentence is 10 words long, and the file is 30% complete, then you would highlight the 4th word. This would be difficult and inexact, but theoretically possible.

显然，已经有一些应用程序和游戏可以执行此操作...说唱歌手Parappa或卡拉OK应用程序，但我认为，他们的操作方式是通过预先录制/静态音频文件(其标记在触发亮点的确切时间.如果您的文本内容总是一样，并且只使用一种语言，那么您也可以这样做.

There are obviously apps and games that already exist that do this... games like Parappa the Rapper, or karaoke apps, but I think the way they do it is by having pre-recorded/static audio files with markers encoded at exact times that trigger the highlights. If your text content is always going to be the same, and only in one language, then you could also do this.

但是，如果说出的文字是用户输入的，或者直到运行时才需要TTS才能知道，那么我不知道任何直接的解决方案.

However, if the spoken text is user-entered or unknown until runtime, requiring a TTS, then I don't know of any straight-forward solution.

如果您决定采用这些更严格的方法之一，那么我建议您相应地发布一个新问题.

If you decide on one of these more narrowed-down approaches, then I would suggest posting a new question accordingly.

这篇关于在语音进行过程中突出显示文本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！