设计抄袭检测引擎在Ruby中的噪声滤波器 [英] Designing a noise filter for Plagiarism Detection Engine in Ruby

问题描述

我一直在抄袭检测引擎的基础上，学术文章的后面MOSS（测量软件相似的）

I have been working on an Implementation of a Plagiarism Detection Engine based on the academic paper behind MOSS(Measure of Software Similarity)

链接 MOSS

有关对于像C / C ++ / Java的语言设计一个噪声滤波器，我有一些需要作出决定。

For designing a noise filter for a language like C/C++/Java, I have some decisions to make.

是关键字相关的检测抄袭还是应该被删除？ 在同一种语言的源文件被绑定到共享同一组关键字。本文并不对如何处理与他们讨论。

Are keywords relevant for detecting plagiarism or they should be removed? Source files in same language are bound to share the same set of keywords. The paper does not discuss on how to deal with them.

如何处理标识？ 更换所有关键字使用单个字符V制作相匹配的独立的变量名是有道理的。

How to deal with identifiers? Replacing all keywords with a single character 'V' making matches independent of variable name makes sense.

做什么用的包导入和库包含？

What to do with package imports and library includes?

空白字符，Commments和标点符号都被明确地剥离。

Whitespaces, Commments and punctuations are to be stripped definitely.

我做的所有的操作后不知道，源文件将只是一群V和其他一些乱码的文字。

I am wondering after doing all the operations, the source file will be just a bunch of 'V' and some other garbled text.

应该噪声滤波器执行什么操作？

What operations should the noise filter perform?

这是最好的办法见解和意见处理噪音？

Insights and Opinions on the best way to deal with noise ?

推荐答案

对于单功能：编译它们，然后比较所产生的汇编code或对象。 对于一个完整的程序：做上述所有功能，并创建一个模糊搜索，找到回片段中的已知功能和碎片数据库

For single functions: compile them, and compare the resulting assembler code or objects. For a whole program: do the above for all the functions and create a fuzzy search to find back the fragments in a database of known functions and fragments.

因此​​，基本上，你需要建立一个编译器，发射其输入的封为重新presentation它，类似于P- code，但preferably人类可读的。

So basically, you need to build a compiler, which emits a canonised representation of its input it, similar to P-code, but preferably human readable.

一些片段比其他人更有特色，片段

Some fragments are more characteristic than others, the fragment

for (i=0; i < 12345; i++) {
  array[i] = 54321;
  }

可能会出现某种形式的每一个程序。它是100％的功能相同

Will probably occur in some form in every program. It is 100% functional identical to

j=0;
while ( j < 12345) {
  foobar[j++] = 54321;
  }

和编译器可能会产生相同的code。

, and a compiler would probably produce identical code.

中可以有可变的名字，数字常数，地址常量，任何差异。但关键字的骨架（ - > {比较，循环，前pressions，作业，调用函数}）将是相同的。所以说：不落的关键词，它们是一个程序的脚手架。

There can be differences in variable-names, numerical constants, address constants, anything. But the "skeleton" of keywords (-> {comparisons, loops, expressions, assignments, function calls}) will be the same. So: don't drop the keywords, they are the scaffolding of a program.

这篇关于设计抄袭检测引擎在Ruby中的噪声滤波器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！