The refacola resource based tests are a special variant of JUnit tests. Instead of writing Java code for each new test case, a sample project is created, and for each test a property file is written. The property file contains all information on how to treat the sample project and what results are expected.
de.feu.ps.refacola.restest
The basic idea is to create test cases based on folders (which are assumed to be mini-projects) instead of test methods. For that reason, a different test suite is to be created, which does not collect all test methods but all mini-project folders and some test properties files instead.
Note: This functionality is implemented in de.feu.ps.tests.javatests, but usually you do not have to think about that.
For each mini-project, or for each properties file found in a mini-project folder, a test case is created and added to the test suite. This is then similar to "normal" JUnit tests in Eclipse.
All you need is to create a new plugin, let's say my.refacola.resourcetests, and add a single Java file in that project. This Java files need to have a static method which gets invoked in order to assemble the test suite. Within this method, you have to configure your overall test set up, that is where to find the mini-projects and which main test class to use. This is pretty simple and canonical. Here is an example:
public class MyRefacolaResourceBasedTest extends RefacolaResourceBasedTest { public static junit.framework.Test suite() throws Exception { TestSuiteBuilder testSuiteBuilder = new TestSuiteBuilder("resources", "My Tests", "mytests", MyRefacolaResourceBasedTest.class); TestSuite testSuite = testSuiteBuilder.createSuite(); return testSuite; } public MyRefacolaResourceBasedTest(String i_testBasePath) throws Exception { super(i_testBasePath); } }The following figure illustrates how to configure the test suite and the overall structure of the test project. That's all Java code you have to write!
A test project looks as the two projects in Figure 1 (Simple and Eiffel). You need some sample (program files), these files are usually placed in a folder "in". The expected output can then be placed in a folder "expected", which will enable you to compare the created results (which will be added to a folder "out" within your project folder) with the expected results.
Note: Writing is not supported yet, so at the moment neither files in expected nor in out are read or written.
The test is set up via a properties file. For each test, you have to create a new properties file. In order to store local information (which are not expected to be submitted to the source code repository), a local properties file with the same name but a different extension is read additionally and overwrites settings defined in the original file:de.feu.ps.refacola.restest
, you will find a mini-project called _ResourceTestTemplate
, its properties file contains all necessary information. Here is a table of the properties which can be defined in order to configure your test. If no default value is specified, the property is mandatory.
Property | Default | Decription |
---|---|---|
General settings | ||
ignore | false | If true, test is ignored. This is a flag usually used to disable tests in your local set up. |
order | 1000 | Order in which this test is to be executed. E.g., for first starting with quick tests and run larger tests then. |
Classes | ||
abstractRefactoringClass | mandatory | Fully qualified class name of refactoring. Do not forget to add the plugin, in which the class is defined, to the project dependencies. |
programInfoProviderClass | mandatory | Fully qualified class name of program info provider. Do not forget to add the plugin, in which the class is defined, to the project dependencies. |
constraintGeneratorClass | de.feu.ps.refacola.api. MinimizedConstraint SetGenerator | |
solver | CHOCOV2 | CHOCOV2 | CHOCOV1 | CREAM |
Refactoring Selection | ||
testProgramFile | mandatory | Mini-project relative path to the program (information) which are to be refactored. |
forcedValue.* | mandatory | new values, mandatory for all forced changes. Pattern: forcedValue.forceChangeName=externalID,newValue Example: forcedValue.x=1,ANY |
Refacola configuration | ||
conf.disableConstantEvaluation | false | disable constant evaluation in constraintset generation |
conf.timeOut | -1 (ignore) | timeout of for whole refactoring |
conf.solverTimeOut | -1 (ignore) | timeout of for solver only |
conf.maxSolutions | -1 (ignore) | max. solutions created by solver |
Expectations | ||
expected.success | not defined (ignore) | Boolean, expected result of refactoring. If not defined, this value is ignored. |
expected.constantlySolved | not defined (ignore) | Boolean, refactoring is expected to be constantly solvable, in this case solver is not activated. This value is ignored if conf.disableConstantEvaluation is set to false. If not defined, this value is ignored. |
expected.constantlyInconsistent | not defined (ignore) | Boolean, refactoring is expected to be impossible, in this case solver is not activated. This value is ignored if conf.disableConstantEvaluation is set to false. |
expected.noSolutions | -1 (ignore) | Integer, number of expected solutions. If the refactoring can be constantly evaluated, a single solution with the forced changes is created. If -1, this value is ignored and not tested. |
expected.noProgramElements | -1 | Integer, number of expected program elements. If -1, this value is ignored and not tested. |
expected.noProgramElements | -1 | Integer, number of expected facts. If -1, this value is ignored and not tested. |
expected.noConstraintsConstant | -1 | Integer, number of expected constant constraints. If -1, this value is ignored and not tested. |
expected.noConstraintsVariable | -1 | Integer, number of expected variable constraint. If -1, this value is ignored and not tested. |
expected.noConstraintsTotal | -1 | Integer, number of expected total constraint (const+var). If -1, this value is ignored and not tested. |
Output Generation | ||
changeset.write | false | Boolean value, if true, the change set is written to testfolder/out/test.properties |
factbase.write | false | if true, the output file is created in testfolder/out |
factbase.write.useDataAsName | true | if true, IProgramElement.getData() is used to create the name of an element |
factbase.write.usePropertyAsName | false | Boolean value, if true, a property value is used as name |
factbase.write.nameProperty | "identifier" | String value, name of property used to retrieve element name, if usePropertyAsName is true |
Special modes | ||
checkConsistency | false | Boolean value, if true, no refactoring
is actually performed. Instead, the program (represented by its elements and
facts) is checked against the rule set used by the refactoring. The
constraint system is expected to be solved, that is
solvable , expectedSolutions , and
expectedSolverConstraints are ignored. Also,
forcedValue.* is ignored, as no changes are applied.
constraintGeneratorClass is ignored as well, since the
CheckConstraintSetGenerator is always used in this mode.
|
as x
), or a computed name based on the kind and property name of the value to be changed. The
name is created by using the following pattern: LanguageNameKindName_PropertyName. All names start with an upper case letter. EiffelClass_Identifier
.solved | solvable | solutions | refacola constraints | Comment |
---|---|---|---|---|
true | true | 0 | 0 | All constraints could be constantly evaluated to true, no variable constraints were generated. Thus, the solver (which actually never gets activated) returns no solutions. |
false | false | 0 | 1 | One constraint could be constantly evaluated to false. No variable constraints were generated. Thus, the solver (which actually never gets activated) returns no solutions. |
false | true | 0..n | 1..n | No constraints could be constantly evaluated to false, some variable constraints were created and the solver found 0..n solutions. |
Besides the numbers, the actual changes are to be compared with expected changes. For that purpose, the test can automatically read a file with change sets and compare these expected changes with the actual ones.
The expected changes are loaded from a file found in a subfolder "exp" of your resource based test, the name of the file must equal the name of the test file with extension ".change". The following tree structure shows an example:
+resources + Simple +in ... +exp test.changes test.propertiesThe changes file contains a list of change sets, each change set is stored in a single line. A change set itself contains changes. The following grammar is used:
ChangeSet ::= Change ( ";" Change )* "\n" Change ::= <externalProgramID> ";" <propertyName> ";" <valueAsString>Other ";" are masked with the backslash. The change sets and the changes are sorted before comparison to avoid ordering problems. However, for manual comparison it is recommended to already sort the changes (which can be simply achieved by forcing a test failure, which will print out the actual results in a sorted manner). The following snippet shows a sample change set for a fact base test:
ConstructorA;identifier;X ConstructorA;identifier;X;B;identifier;A;ConstructorB;identifier;A ConstructorA;identifier;X;B;identifier;X;ConstructorB;identifier;XThis can be read more user friendly as:
ConstructorA.identifier=X ConstructorA.identifier=X; B.identifier=A; ConstructorB.identifier=A ConstructorA.identifier=X; B.identifier=X; ConstructorB.identifier=XThe current format is simply easily parsed.
If a comparison fails, it will print out the (first) difference, e.g.
Change set differ on line 3: Class B..identifier: U != XThe first value is the expected one, while the second one is the actual value.
################################################################################ # Test properties to be shared via SVN, local settings # should be defined in test.local.properties (with svn:ignore set). # ################################################################################ ################################################################################ # Test Suite Settings # # known properties: ignore, order ################################################################################ # ------------------------------------------------------------------------------ # ignore.Short template:=true that test in specific test class (simple name) or # ignore=true for ignore that project in all test classes # ------------------------------------------------------------------------------ #ignore=true # ------------------------------------------------------------------------------ # order in which this test is to be executed, default: 1000 # ------------------------------------------------------------------------------ # order=1000 ################################################################################ # Test Specific Settings # # Properties can be defined for all tests or for a specific # test. # E.g., a property "timeout=1000" is used for all tests, while # "MySpeedyTest.timeout=10" is only used in test MySpeedTest. # # Properties are test specific, there are no generic settings defined. ################################################################################ # ------------------------------------------------------------------------------ # general settings: # ------------------------------------------------------------------------------ # programInfoProviderClass, class name of program info provider, mandatory programInfoProviderClass=de.feu.ps.refacola.factbase.FactBaseReader # additional parameters (properties of the provider class) programInfoProviderClass.factory=de.feu.ps.refacola.lang.enums.EnumsFactory # constraintGeneratorClass, class name of constraint generator, # default: de.feu.ps.refacola.api.MinimizedConstraintSetGenerator constraintGeneratorClass=de.feu.ps.refacola.api.MinimizedConstraintSetGenerator # solver = CHOCOV2 | CHOCOV1 | CREAM, default: CHOCOV2 solver=CHOCOV2 # ------------------------------------------------------------------------------ # refactoring selection # ------------------------------------------------------------------------------ # abstractRefactoringClass, class name of refactoring, mandatory abstractRefactoringClass=de.feu.ps.refacola.lang.enums.refactorings.RefacolaChangeAccessRefactoring # testProgramFile, file name of test program testProgramFile=in/simple.factbase # ------------------------------------------------------------------------------ # forced values, mandatory for all forced changes # the name of the change is either the explicitly defined name, # or a computed name from kind's language, kind and property type, # e.g. EiffelClass_Identifier (from _ ). # The kind must be the kind as defined in the refactoring, and # not the actual kind (which may be a subkind). # ------------------------------------------------------------------------------ forcedValue.x=B,drei # ------------------------------------------------------------------------------ # configuration # ------------------------------------------------------------------------------ # conf.disableConstantEvaluation: boolean, disable constant evaluation in # constraintset generation # default: false conf.disableConstantEvaluation = false # conf.timeOut: int, timeout of for whole refactoring # default: -1 (ignore) conf.timeOut=-1 # conf.solverTimeOut: int, timeout of for solver only # default: -1 (ignore) conf.solverTimeOut=-1 # conf.maxSolutions: int, max. solutions created by solver # default: -1 (ignore) conf.maxSolutions=-1 # ------------------------------------------------------------------------------ # expected results # ------------------------------------------------------------------------------ # expected.success: boolean, expected result of refactoring # default: if not defined, this value is ignored expected.success=true # expected.constantlySolved: boolean, refactoring is expected to be # constantly solvable, in this case solver is not activated. This # value is ignored if conf.disableConstantEvaluation is set to false. # default: if not defined, this value is ignored expected.constantlySolved=false # expected.constantlyInconsistent: boolean, refactoring is expected to be # impossible, in this case solver is not activated. This # value is ignored if conf.disableConstantEvaluation is set to false. # default: if not defined, this value is ignored expected.constantlyInconsistent=false # expected.noSolutions: int, expected number of solutions # default: -1 (ignore) expected.noSolutions=2 # ------------------------------------------------------------------------------ # expected statistic data # ------------------------------------------------------------------------------ # expected.noProgramElements: int, expected number of program elements, # default: -1 (ignore) expected.noProgramElements=2 # expected.noProgramFacts: int, expected number of facts, # default: -1 (ignore) expected.noProgramFacts=1 # expected.noConstraintsConstant: int, expected number of constant constraints, # default: -1 (ingore) expected.noConstraintsConstant=0 # expected.noConstraintsVariable: int, expected number of variable constraints # default: -1 (ignore) expected.noConstraintsVariable=1 # expected.noConstraintsTotal: int, expected total number of constraints # default: -1 (ignore) expected.noConstraintsTotal=-1 # ------------------------------------------------------------------------------ # output # ------------------------------------------------------------------------------ # changeset.write: boolean, write change set to out folder. If change set is # written, and a change set is found in exp, the changes are checked. # default: false changeset.write = true # factbase.write: boolean, write fact base to out folder # default: false factbase.write = true # factbase.write.useDataAsName: boolean, use AST data for element name # default: true factbase.write.useDataAsName = true # factbase.write.usePropertyAsName: boolean, use a property value (if defined) # for element name, the property is defined via factbase.write.nameProperty # default: false factbase.write.usePropertyAsName = true # factbase.write.nameProperty: String # default: identifier factbase.write.nameProperty=identifier
# ------------------------------------------------------------------------------ # general settings: # ------------------------------------------------------------------------------ programInfoProviderClass=de.feu.ps.refacola.factbase.FactBaseReader programInfoProviderClass.factory=de.feu.ps.refacola.lang.abc_lang.Abc_langFactory # ------------------------------------------------------------------------------ # refactoring selection and forced values # ------------------------------------------------------------------------------ abstractRefactoringClass=de.feu.ps.refacola.lang.abc_lang.refactorings.RefacolaChange_b_cRefactoring testProgramFile=in/in.factbase forcedValue.x=B1,C2 # ------------------------------------------------------------------------------ # expected results # ------------------------------------------------------------------------------ expected.success=true expected.noSolutions=2 # ------------------------------------------------------------------------------ # output # ------------------------------------------------------------------------------ changeset.write = true #factbase.write = true
Note: At the moment, only some text files are read using the Eiffel reader. Still, it is even possible to use the JDT within the tests. In that case, the overall set up doesn't much differ from the description above. Of course, the Java project has to be configured somehow. Then the suite has to be run as Eclipse Plugin test. This has already been done in the congen project, the congen tests (congen.parsertest) are resource based as well, although they do not use the current Refacola API.
log.de.feu.ps.refacola.api.refactorings=SEVERE log.de.feu.ps.refacola.api.refactorings.AbstractRefactoring=FINEST log.de.feu.ps.refacola.api=INFOspecifies the level for three loggers. These lines are similar to definining the levels in a logger properties file without the prefix "log.", that is
de.feu.ps.refacola.api.refactorings=SEVERE de.feu.ps.refacola.api.refactorings.AbstractRefactoring=FINEST de.feu.ps.refacola.api=INFOThe following levels are defined:
A logger is automatically defined when its level is set. Although the name of a logger can be any name, usually the class name is used. Loggers are defined hierarchically, that is, the level of logger
de.feu.ps.refacola.api.refactorings
is used by a logger named
de.feu.ps.refacola.api.refactorings.AbstractRefactoring
as long as the level for the later is not explicitly defined somewhere else.
A common practice to use JDK logging is illustrated by the following code snippet:
public abstract class AbstractRefactoring implements IRefactoring { /** * Logger for this class */ private static final Logger log = Logger.getLogger(AbstractRefactoring.class.getName()); ... protected void runRefactoring() { ... if (log.isLoggable(Level.INFO)) { log.info("numberOfsolutions=" + numberOfsolutions); //$NON-NLS-1$ } }
As you probably want to configure logging for debugging purposes, the best location to configure logging is not in the test properties file but in the local properties file. Do not forget to add svn:ignore to your local properties files, if this is not already configured in the test project.
Further readings: