When a programming task has been completed by a candidate a report is generated. There is a lot of information we collect, so here is an explanation of what everything means.
Below is what a report will look like.
Automatic Evaluation
This consists of two parts: the Functionality score and the Code quality score.
The Functionality score (90%) takes into account the verification tests. It will show how well the code is working, the higher the score the better the functionality.
Code quality is 10% of the overall score with any TalentScore task (It will be individually set if you create your own). It will score the code before the candidate starts and after the candidate has finished against an outside source code analyzer. It finds common programming flaws like unused variables, empty catch blocks, unnecessary object creation, and so forth.
The best way to think of these two tests is to compare them to learning a language. The functionality score is whether a person is able to communicate what they want, so if a sentence gets the desired result. The code quality score is the grammar, a sentence might get the desired result but it may not be grammatically correct.
I want red ice cream - it is functional (mostly) but the quality is bad.
Hello, I would like the strawberry ice cream, please. - it is functional and the quality is good
Build Status
This section will tell you if the candidate has managed to build a working piece of code. In this example, we can see that they haven't. If desired, it is possible to go in further to see their build log and the exact reason why it failed. The button is located in the bottom right-hand corner of the box.
Verification tests
At TalentScore we have two types of tests, candidate tests that are visible during the test, and verification tests that are hidden from the candidate and are only visible to the recruiter. They will test the edge cases and try to find out if the candidate is a problem solver. A basic example is the calculator task that we have, here is the task:
Build a calculator that can divide, multiply, subtract, and add. The candidate tests are as follows:
- Can divide
- Can Add
- Can Subtract
- Can Multiply
The verification test that we have is:
- The candidate has written a rule to say you cannot divide by zero.
This will help you to find a candidate who isn't just focused on the output but on the code as a whole.
Violations
Violations encompass a wide range of mistakes or errors that can be made during the programming task. The report will give the original number of violations and violations in the piece of code after it has been altered by the candidate. There are many examples of violations and they will all be weighted differently depending on the severity. Here are some examples:
- Not naming variables correctly
- White spaces after a piece of code
- The method is too long
- The class has too many fields
- Spelling mistakes
Rules Compliance
Rules compliance is an infographic, which is produced from the violations and the weight of the severity of the violations. If a candidate was to solve all the violations it would get a score on 100%. If the candidate increased the violations in the code, this is displayed as well.