自动为网页创建测试用例? [英] Automatically create test cases for web page?
问题描述
如果某人有一个网页，测试网站是否存在用户交互错误的常用方法是手动创建每个测试用例并使用 selenium.
If someone has a webpage, the usual way of testing the web site for user interaction bugs is to create each test case by hand and use selenium.
是否有自动创建这些测试用例的工具?那么，如果我的网页发生了更改，新的测试用例会自动创建吗?
Is there a tool to create these testcases automatically? So if I have a webpage that gets altered, new test cases get created automatically?
推荐答案
我认为 自动增长测试用例 更符合您的要求.更具体地说，我将尝试介绍基础知识，如果您有兴趣，请仔细查看 进化测试
I think Growing Test Cases Automatically is more of what your asking. To be more specific I'll try to introduce basics and if you're interested take a closer look at Evolutionary Testing
通常我们会遇到一组标准的限制条件，例如不断变化的被测系统 (SUT) 的功能、有限的时间框架、缺乏合适的测试工具等等……然而，随着技术的发展，还有另一种类型的挑战出现解决方案进一步发展——系统复杂性增加.
Usually there is a standard set of constraints we meet like changing functionality of the system under test (SUT), limited timeframe, lack of appropriate test tools and the list goes on… Yet there is another type of challenge which arises as technological solutions progress further – increase of system complexity.
虽然典型的限制可以通过不同的技术和管理方法解决，但在系统复杂性的情况下，我们面临着定义用于评估和验证系统行为的直接分析方法的能力限制.复杂系统由多个通常是异构的组件组成，这些组件在协同工作时会放大彼此的统计和行为偏差，从而导致系统以不属于其初始设计的方式运行.更糟糕的是，复杂系统在相同机制的帮助下也会增加对环境的敏感性.
While the typical constraints are solvable through different technical and management approaches, in the case of system complexity we are facing the limit of our capability of defining a straight-forward analytical method for assessing and validating system behavior. Complex system consist of multiple, often heterogeneous components which when working together amplify each other’s statistical and behavioral deviations, resulting in a system which acts in ways that were not part of its initial design. To make matter worse, complex systems increase sensitivity to their environment as well with the help of the same mechanism.
测试复杂系统的选项
我们如何测试每次运行测试场景时行为都不同的系统?我们如何重现一个需要花费数天和数百万才能恢复的问题，但只是在大约已知的条件下不时发生?
How can we test a system which behaves differently each time we run a test scenario? How can we reproduce a problem which costs days and millions to recover from, but happens only from time to time under conditions which are known just approximately?
我想关注的一个可能的解决方案是接受我们缺乏的知识，并通过使用进化测试来处理我们所拥有的一切.在这种情况下，进化测试可以被视为黑盒测试的一种变体，因为我们正在处理将输入输入到 SUT 并评估其输出，而不关注其内部结构.这里的细线是，我们正在组织这个自动测试用例生成和大规模执行的过程，作为一个模仿自然进化的迭代优化过程.
One possible solution which I want to focus on is to embrace our lack of knowledge and work with whatever we have by using evolutionary testing. In this context the evolutionary testing can be viewed as a variant of black-box testing, because we are working with feeding input into and evaluating output from a SUT without focusing on its internal structure. The fine line here is that we are organizing this process of automatic test case generation and execution on a massive scale as an iterative optimization process which mimics the natural evolution.
进化测试
元素:
• Population – 一组测试用例执行，参与优化过程
• Population – set of test case executions, participating into the optimization process
• 世代 – 人口的一部分，参与给定的迭代
• Generation – the part of the Population, involved into given iteration
• Individual – 单个测试用例执行及其结果，来自 Population 的一个元素
• Individual – single test case execution and its results, an element from the Population
• 基因组 - 所有测试用例的统一定义，描述人口的模型
• Genome – unified definition of all test cases, model describing the Population
• 基因型 – 单个测试用例实例、描述个体的模型、基因组实例
• Genotype – a single test case instance, a model describing an Individual, instance of the Genome
• 重组 – 将一种或多种基因型转化为新的基因型
• Recombination – transformation of one or more Genotypes into a new Genotype
• 突变 – 基因型的随机变化
• Mutation – random change in a Genotype
• 适应度函数 – 形式化的标准，表示个体对优化目标的适用性
• Fitness Function – formalized criterion, expressing the suitability of the Individual against the goal of the optimization
我们如何创建这些元素?
How we create these elements?
• 实验目标的定义(选择标准)——设置优化过程的方向并与 SUT 的行为相关.在执行的测试用例实验期间涉及 SUT 状态或环境的某些特征.例子:o SUT 应该使用错误代码完成测试用例执行"o 测试用例应该驱动 SUT 通过 SUT 逻辑结构中最多数量的分支"o 在测试用例执行期间，SUT 所在房间的环境温度不应超过 40 ºC"o 系统上的 CPU 利用率，在测试用例执行期间运行 SUT 的位置应超过 80%"SUT 及其环境的任何可测量参数都可以用于目标陈述.了解测试输入和目标本身之间的关系不是强制性的.这提供了覆盖直接从需求派生的目标的可能性，而不是基于一些后期的需求派生，如业务、架构或技术模型.• 被测系统相关输入和输出的定义——SUT 输入和输出的识别，以及与实验目标相关的环境参数.• 实验基因组的正式定义——将汇总的测试用例集编码为参数化模型(通常是数据结构)，表达相关的 SUT 输入数据、环境参数和动作序列.该定义还需要符合应用于基因组实例的两个主要操作——重组和突变.这两个操作的机制可以针对基因组中存在的数据或动作类型进行预定义，或者具有自定义定义• 选择标准的正式定义(适应度函数)——一种评估机制，它采用测试用例执行(个体)产生的 SUT 输出或环境参数并计算一个数字(适应度)，表示该特定个体与实验目标的接近程度.
• Definition of the experiment goal (selection criteria) – sets the direction of the optimization process and is related to the behavior of the SUT. Involves certain characteristics of SUT state or environment during the performed test case experiments. Examples: o "SUT should complete the test case execution with an error code" o "The test case should drive the SUT through the largest number of branches in SUT’s logical structure" o "Ambient temperature in the room where SUT is situated should not exceed 40 ºC during test case execution" o "CPU utilization on the system, where SUT runs should exceed 80% during test case execution" Any measurable parameters of SUT and its environment could be used in a goal statement. Knowledge of the relation between the test input and the goal itself is not obligatory. This gives a possibility to cover goals which are derived directly from requirements, rather than based on some late requirement derivative like business, architectural or technical model. • Definition of the relevant inputs and outputs of the tested system – identification of SUT inputs and outputs, as well as environment parameters, relevant to the experiment goal. • Formal definition of the experiment genome – encoding the summarized set of test cases into a parameterized model (usually a data structure), expressing relevant SUT input data, environment parameters and action sequences. This definition also needs to comply with the two major operations applied over genome instances – recombination and mutation. The mechanism for those two operations can be predefined for the type of data or action present in the genome or have custom definitions • Formal definition of the selection criteria (fitness function) – an evaluation mechanism which takes SUT output or environment parameters resulting from a test case execution (Individual) and calculates a number (Fitness), signifying how close is this particular Individual to the experiment goal.
流程如何运作?
- 我们使用基因组来创建一代随机基因型(测试用例实例).
- 我们执行测试用例(基因型)生成结果(个人)
- 我们使用健身函数根据我们的目标评估每个执行结果(个人)
- 我们只从给定世代中选择适合度高于给定阈值(前 10%、高于平均水平等)的个人.
- 我们使用选定的个体通过应用重组和突变来产生一个新的完整的生成集
- 我们重复这个过程，返回第 2 步迭代过程通常通过设置与评估的一代适应度相关的条件来停止.例如:• 如果自上次迭代以来，top Fitness 的变化不超过 0.1%• 如果一代中的顶部和底部适应度之间的差异小于 0.3%那么可能是时候停下来了.
优点和缺点
优势:
• 我们可以利用有限的知识来处理 SUT 和面向目标的测试定义
• We can work with limited knowledge for the SUT and goal-oriented test definitions
• 我们使用测试用例模型(基因组)，这使我们能够毫不费力地大量生产大量测试用例(基因型)
• We use a test case model (Genome) which allows us to mass-produce a large number of test cases (Genotypes) with little effort
• 我们可以在第一次迭代中播种"测试用例(基因型)，而不是随机生成它们，以加快优化过程.
• We can "seed" test cases (Genotypes) in the first iteration instead of generating them at random in order to speed up the optimization process.
• 我们可以并行运行测试用例以加快进程
• We could run test cases in parallel in order to speed up the process
• 我们可以找到多种满足我们测试目标的解决方案
• We could find multiple solutions which meet our test goal
• 如果优化过程收敛，我们可以保证接下来的每一代都是我们测试目标的更好的近似解.这意味着即使我们需要在达到最佳 Fitness 之前停止，我们仍然会拥有比我们开始时更好的测试用例.
• If the optimization process in convergent we have a guarantee that each following Generation is a better approximate solution of our test goal. This means that even if we need to stop before we have reached optimal Fitness we will still have better test cases than the one we started with.
• 我们可以实现非常复杂、难以重现的测试场景的重放，这些场景模拟现实生活，并且远远超出任何其他自动化或手动测试技术的范围.
• We can achieve replay of very complex, hard to reproduce test scenarios which mimic real life and which are far beyond the reach of any other automated or manual testing technique.
缺点:
• 为进化测试实施定义必要元素的过程非常重要，并且需要特定的知识.
• The process of defining the necessary elements for evolutionary test implementation is non-trivial and requires specific knowledge.
• 实施这种自动化方法既费时又费资源，只有在合理的情况下才应采用.
• Implementing such automation approach is time- and resource-consuming and should be employed only when it is justifiable.
• 优化过程的收敛取决于适应度函数的平滑度.如果它的定义导致区域不连续或小/无梯度，那么我们可以预期收敛缓慢或没有收敛
• The convergence of the optimization process depends on the smoothness of the Fitness Function. If its definition results in a zones of discontinuity or small/no gradient then we can expect slow or no convergence
更新:
我还建议您查看 遗传算法 和这篇关于 测试数据生成可以为您提供方法和指导.
I also recommend you to look at Genetic algorithms and this article about Test data generation can give you approaches and guidelines.
这篇关于自动为网页创建测试用例?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
