我如何分页输出spark-shell [英] How can I page output spark-shell

问题描述

如何分页输出到spark-shell控制台的输出?例如，当我运行以下命令从会话中列出已定义的术语时，我经常会得到一长串的输出，这些输出超过了终端中的行数.

How do I page output written to the spark-shell console? For example, when I run the following command to list the defined terms from my session I often get a long list of output that exceeds the number of rows in my terminal.

$ intp.definedTerms.foreach {println(_)}

在 bash 外壳程序中，我将使用 less 来分页命令或程序的输出.有没有类似于 less 的分页功能可用于spark-shell?

In a bash shell I would use less to page output from a command or program. Is there paging functionality, similar to less, available to spark-shell?

谢谢.

推荐答案

spark-shell没有，但是最后我链接到有关RDD上Spark的pipe()动作的信息，让您将输出分叉到外部程序.

spark-shell doesn't but at the end I link to info about Spark's pipe() action on RDDs that let's you fork output to external programs.

窗口滚动?

您没有说自己所处的环境吗?

You don't say which environment you are in?

例如，如果您在Ubuntu或几乎所有的窗口系统中都位于Unity中，那么终端窗口的滚动功能是否可以满足您的需求?

If for example you are in Unity in Ubuntu, or almost any windowing system, would the scrolling function of a terminal window satisfy your needs?

您可以在调用spark-shell之前编辑.bashrc并修改该滚动设置.

You can edit .bashrc and modify that scrolling setting before calling spark-shell.

还有一些有用的基于GUI的方法来影响窗口滚动: https://askubuntu.com/questions/385901/how-to-see-more-lines-in-the-terminal

There are also some useful GUI-based ways to affect window scrolling: https://askubuntu.com/questions/385901/how-to-see-more-lines-in-the-terminal

这是另一个页面，其中包含有关编辑.bashrc的更多建议，同样，有关您的环境的更多信息也将有所帮助. https://askubuntu.com/questions/51122/setting-gnome-从bashrc内获取终端窗口大小

Here's another page with more suggestions on editing .bashrc, again more info on your environment would be helpful. https://askubuntu.com/questions/51122/setting-gnome-terminal-window-size-from-within-bashrc

Spark中RDDS上的PIPE ACTIONS

PIPE ACTIONS on RDDS in Spark

此外，没有您的特定代码，很难知道这是否适用，有一种方法可以将对RDD的操作输出传递给外部程序.请参见 http://spark.apache.org/docs/latest/programming-guide.html#transformations ，摘录如下:

Further, without your specific code it's hard to know if this is applicable, there's a way to pipe the output of actions on RDDs to external programs. See http://spark.apache.org/docs/latest/programming-guide.html#transformations, here's an excerpt:

通过shell命令将RDD的每个分区放入管道，例如一个Perl或bash脚本.将RDD元素写入进程的stdin和输出到其stdout的行将作为字符串的RDD返回.

Pipe each partition of the RDD through a shell command, e.g. a Perl or bash script. RDD elements are written to the process's stdin and lines output to its stdout are returned as an RDD of strings.

这篇关于我如何分页输出spark-shell的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！