This article explains the various evaluation methods by which subjective questions can be evaluated.
Subjective questions provide you with three methods of evaluation, which are as follows:
1. AI evaluation
AI evaluation uses Hackerearth's proprietary AI model to evaluate answers automatically. It will compare the base answer you provided to the candidate's submission. You can use the AI evaluation method to answer essay questions or questions with long text answers.
You also have the option to view the difference between the actual answer and the candidate's submission. For example, you are comparing the two answers after enabling View difference.
You can also provide a solution template for candidates, which can work as a starting point for them to write the answer.
Note: Turn this feature on only for text-based answers and not for numerical answers or answers where the language is not English. You can use the Keyword evaluation method to evaluate such answers.
2. Keyword evaluation
Subjective questions can also be evaluated using the keywords method. A Keyword can be any of the following:
- Text string: This can include spaces, special characters, and alphanumeric characters. It must have at least one non-whitespace character.
- Number: This can include an integer or decimal.
You can use keyword evaluation to evaluate questions related to data analytics (MS Excel), mathematical numerical, or fill-in-the-blank questions.
Keywords will be trimmed of leading and trailing whitespaces once an admin adds them. You can use Keywords with a maximum length of 30 characters. At least 1 Keyword is mandatory for the Keyword evaluation method. You can add more than one Keyword up to a maximum of 15 keywords. Each option mandatorily has a corresponding score. At least one keyword option score must be equal to the maximum score of the question.
For every keyword option, the subsequent condition needs to be fulfilled:
- Organize Keyword Options in descending order based on their scores.
- Verify whether the keyword is present in the candidate's response at least once. This verification is case-insensitive.
- If the keyword is found, allocate the associated keyword score as the question's score.
If the keyword is not found, the steps get repeated with the next highest-scoring keyword.
Example: Text string as a keyword
Here is an example where the added keyword is a text string.
Example problem statement
Once the question is evaluated, and if the answer contains the exact keyword, the candidate will receive full marks assigned to that keyword.
Keyword evaluation
Example: Number as a keyword
Here is an example where the added keyword is a number.
If the answer contains the exact keyword, the score assigned to the keyword is allocated to the candidate’s answer.
Note: Partial matches will not be given a score.
For example, only the entire keyword will be the right answer to the given question.
The score will not be provided as the answer is 12, but the submission is 1. Similarly, with string answers, only the answer with the entire keyword will be the right submission. For example, if the expected keyword is mango, and the candidate gave the submission as man, the candidate will not receive any score.
3. Manual evaluation
Subjective questions can also be evaluated manually, which means you can give your own answer in the base answer, but it is not mandatory to write the base answer in this type of evaluation. You can manually check the candidate’s submission and use the base answer as a reference.
The base answer will also be present in the candidate’s report to make the comparison easy.
These three evaluation methods can be used to evaluate a subjective question.