Package org.nuxeo.runtime.test.runner

Class RandomBug

java.lang.Object
org.nuxeo.runtime.test.runner.RandomBug
public class RandomBug extends Object
Define execution rules for an annotated random bug.

Principle is to increase consistency on tests which have a random behavior. Such test is a headache because:

  • some developers may ask to ignore a random test since it's not reliable and produces useless noise most of the time,
  • however, the test may still be useful in continuous integration for checking the non-random part of code it covers,
  • and, after all, there's a random bug which should be fixed!

Compared to the @Ignore JUnit annotation, the advantage is to provide different behaviors for different use cases. The wanted behavior depending on whereas:

  • we are working on something else and don't want being bothered by an unreliable test,
  • we are working on the covered code and want to be warned in case of regression,
  • we are working on the random bug and want to reproduce it.

That means that a random bug cannot be ignored. But must attempt to reproduce or hide its random aspect, depending on its execution context. For instance: 

 
 import org.nuxeo.runtime.test.runner.FeaturesRunner;
 import org.nuxeo.runtime.test.RandomBugRule;

 @RunWith(FeaturesRunner.class)
 public class TestSample {
     public static final String NXP99999 = "Some comment or description";

     @Test
     @RandomBug.Repeat(issue = NXP99999, onFailure=5, onSuccess=50)
     public void testWhichFailsSometimes() throws Exception {
         assertTrue(java.lang.Math.random() > 0.2);
     }
 }

In the above example, the test fails sometimes. With the RandomBug.Repeat annotation, it will be repeated in case of failure up to 5 times until success. This is the default RandomBug.Mode.RELAX mode. In order to reproduce the bug, use the RandomBug.Mode.STRICT mode. It will be repeated in case of success up to 50 times until failure. In RandomBug.Mode.BYPASS mode, the test is ignored.

You may also repeat a whole suite in the same way by annotating the class itself. You may want also want to skip some tests, then you can annotate them and set RandomBug.Repeat.bypass() to true.

Since:
5.9.5
See Also: