用户可以在瀑布式步骤中在文本提示上发送附件和消息吗? [英] Can a user send attachment along-with message on Text prompt in a waterfall step?

问题描述

假设我们在瀑布对话框中执行以下步骤:

Lets say we have following steps in our waterfall dialog:

self.add_dialog(TextPrompt(TextPrompt.__name__))
    self.add_dialog(
        WaterfallDialog(
            WaterfallDialog.__name__,
            [
                self.project_step,
                self.name_step,
                self.confirm_step,
                self.final_step,
            ],
        )
    )

async def project_step(
    self, step_context: WaterfallStepContext
) -> DialogTurnResult:
    """
    If a project name has not been provided, prompt for one.
    :param step_context:
    :return DialogTurnResult:
    """
    confluence_details = step_context.options

    if confluence_details.project is None:
        message_text = key.query_project_confluence_text.value + "?"
        prompt_message = MessageFactory.text(
            message_text, message_text, InputHints.expecting_input
        )
        return await step_context.prompt(
            TextPrompt.__name__, PromptOptions(prompt=prompt_message)
        )
    return await step_context.next(confluence_details.project)

如果用户在提示时将附件和文本一起发送到机器人. 是否可以在 step_context.result 中获得两者.

If a user send an attachment along-with text to the bot at the prompt. Is it possible to get both in step_context.result.

在 on_message_activity 中，我可以使用 TurnContext.activity.attachments 进行附件检查，但是如何在后续步骤中使用Waterfall step_context和Text消息来接收附件?

In on_message_activity i could check using TurnContext.activity.attachments for attachment but how do i receive the same using Waterfall step_context and the Text message as well in subsequent step?

请求正文如下:

{
    "text":"Hello there",
    "type":"message",
    "from":{
        "id":"xyz"
    },
    "attachments":{
        "contentType":"audio/wav",
        "name":"BabyElephantWalk60.wav",
        "contentUrl":"data:audio/wav;base64,UklGRvAEAgBXQVZFZm10IBAA
    }
}

客户端，即iOS应用将使用直线Api https://directline.botframework.com/v3/directline/conversations/EdWGs8IdmjNIy5j2E93EHW-a/activities 发送活动.

Client side i.e iOS App will be using directline Api https://directline.botframework.com/v3/directline/conversations/EdWGs8IdmjNIy5j2E93EHW-a/activities to send the Activity.

iOS应用程序正在使用语音工具包.

iOS application is using speech kit.

在提示符下，无论用户说什么，消息及其音频文件都将通过上面提供的请求正文中的直接路线发送给bot.并且，这将使用麦克风按钮完成.

On prompt, Whatever user speaks,the message along with audio file of it is to be sent to bot over directline in the request body as provided above.And,this will be done using mic button.

有可能这样做吗?

推荐答案

当它实际上更像是一个客户问题时，您似乎将其视为机器人问题.您的漫游器只能响应收到的活动，因此，如果客户端从不发送包含音频和文本的活动，则您的漫游器将无法处理包含音频和文本的活动.由于您使用的是自己的Direct Line客户端，因此由您自己决定允许客户端发送这样的活动.由于音频文件通常很大，因此我建议上传文件，而不要在附件中放置数据URL.

You seem to be treating this like a bot question when really it's more of a client question. Your bot can only respond to the activities it receives, so there won't be any way for your bot to handle an activity with both audio and text if the client never sends an activity with both audio and text. Since you're using your own Direct Line client, it's up to you to allow your client to send such an activity. Since audio files are normally very large, I recommend uploading the file rather than putting a data URL in the attachment.

通常，用户会在不同的回合中将附件和文本作为单独的活动发送.该机器人将通过跟踪处于其状态的对话框来处理这些单独回合中的数据，并且它很可能是瀑布对话框.听起来您不想这样做，因为文本和附件在您的情况下确实是相同的数据.

Ordinarily, the user would send an attachment and text as separate activities on separate turns. The bot would handle the data on those separate turns by keeping track of a dialog in its state, and it would probably be a waterfall dialog. It sounds like you don't want to do this because the text and the attachment are really the same data in your case.

在bot端，您可以根据需要在任何对话框之外访问活动的文本和附件.您还可以在任何提示后直接在瀑布步骤内访问活动，因为步骤上下文包含包含活动的转弯上下文.

On the bot side, you can access the text and attachments of the activity outside of any dialog if you like. You also have direct access to the activity inside a waterfall step following any prompt, because the step context contains the turn context which contains the activity.

text = step_context.context.activity.text
attachments = step_context.context.activity.attachments

您可以使用文本提示或附件提示来执行此操作.如果您希望能够访问step_context.result中的那些内容，则可以做出自己的提示，以将整个活动置于结果中.您可以使用 ActivityPrompt 作为基类，因为它是为此目的而创建的.

You can do this with a text prompt or an attachment prompt. If you want to be able to access those things in step_context.result, you could make your own prompt that puts the whole activity in the result. You can use ActivityPrompt as a base class since it was made for that purpose.

除了发送单独的活动之外，在同一活动中发送文本和音频的另一种方法是仅发送音频，然后让机器人使用

Besides sending separate activities, another alternative to sending the text and the audio in the same activity would be to send just the audio and have the bot convert the audio into text using Cognitive Speech Services. This probably wouldn't be ideal because then your client wouldn't be able to display the text since it's not doing the conversion on its end. I'm assuming you are having the having the user provide audio input with the microphone and then converting it into text rather than having the user enter text and then converting it into audio.

直线语音是内置的方式认知语音服务，以便您的客户端和漫游器都可以访问文本.根据您的需求，您可以考虑研究网络聊天语音.

Direct Line Speech is a builtin way of leveraging Cognitive Speech Services so that both your client and bot can access the text. And depending on your needs, you might consider looking into Web Chat speech.

这篇关于用户可以在瀑布式步骤中在文本提示上发送附件和消息吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！