How to Analyze Applicant Results
So your Campaign is live, and you want to start analyzing your Results!
Talent Llama has incredible reporting capabilities (and, if you're integrated with an ATS, we can even push candidate results over, so your ATS remains your source of truth!).
Let's start with a quick overview video, and then we'll dive in below.
Note: Applicant names and identifying details are hidden to prevent bias. Account Admins can enable name visibility in Account Settings if needed, however
In this Article:
Overview
Let's start with the Overview. This is the page that opens first when you click on an active Campaign.
Interview Funnel
This is an all-time view of interviews being run by Talent Llama.
Applicants shows the total number of applicants for that role.
Invitations Sent shows the total interviews sent. It may be less than Applicants if there are any candidates waiting in the queue to be sent (check the Applicants tab and filter by 'Email Invite Status'.
Interviews Started shows how many interviews are currently in progress or were abandoned.
Interviews Completed shows the total number of finished interviews (including those who didn't meet minimum requirements but still 'finished' the interview).
Key Metrics
This is your Campaign Key Metrics for all-time, shown as averages.
Completion Time shows how long, on average, it's taking candidates to complete the whole interview.
Invite to Start shows how long, on average, it takes to start an interview from the time the email is sent, until the link is clicked.
Cheating Probability shows the average probability of cheating across everyone (remember, if one person is significantly higher, it would drive this average up).
Experience Rating shows the average sentiment of applicants based on their form feedback post interview (the submission is not required, so the amount of responses might be lower than the rest).
Completions and Latest Results
Completions is a simple visual for seeing how many interviews were sent and completed on any given day for the last 30 days. Remember, you might have sent 8 invites and had 3 completions on one day, however completions may have flowed over from previous days, as above example shows, the Invite to Start is just below 2 days.
Latest Results is a scrollable view for the latest interview results, you can click on any line here and go straight to that candidates results, or click View All in the top right of that block to see them all.
Results
By clicking on the Results tab on the right hand side of your Campaign, you'll be taken to your Overall Statistics and Evaluations page.
Overall Statistics
The Overall Statistics will give you a breakdown of all Topics you built into your Campaign. By clicking the dropdown arrow on the top right of any score, you'll reveal more information, including Minimum, Maximum and Median scores for each Topic. This gives you a birds-eye view into how each section is scoring.
Evaluations
This screen is a high-level view of all your applicants. This screen shows the Overall Score (out of 10) which is calculated based on several inputs, most commonly how applicants respond to the Topics you've set up in the Campaign Outline.
This score is relative, so if the majority of your candidates are sitting around a 5, it's not necessarily a bad thing, it could be that certain components weren't answered - but this could also be because of industry, or role type. (IE, you shouldn't expect an entry level role to score 10's across the board but it's possible!).
By clicking into any line (for example, second from the bottom) you'll be taken to that applicants specific evaluation.
You can see in the example below,, this applicant scored 8.7/10, and is in the top 5% of all applicants.
This page will breakdown two key results:
Overall - This shows how the applicant performed overall, including Positives and Potential Concerns. You'll also see a blurb directly next to the larger 8.7 score, which is a simplified and combined outcome for the interview as a whole.
Language - This shows how the candidate performed in their overall fluency of English. If you have multiple languages activated, and an applicant is taking their interviews in Spanish for example, it would talk to their overall fluency of Spanish.
Minimum Qualifications
This section simply confirms if the applicant met the Minimum Qualifications you set in your Campaign Outline. Green ticks show a candidate met those requirements, and;
Red ticks show if a applicant did not meet those requirements. (In this case, the interview would politely end, and the applicant would score a 0).
Topic Evaluations
Further down the page, you'll see a deep-dive into each topic. Here, you'll see a number of things. Each Topic has a blurb about how the applicant performed, where they ranked, and what Talent Llama's score was (for example, below you'll see Support Ticket Triage as 8/10, whilst Customer Feedback Management was 9.50/10)
Interview Experience Rating and Interview Integrity
Towards the bottom of the Results page, you'll see the applicants' Experience Rating (and any comments they left), and the Interview Integrity block.
Interview Integrity shows you the candidates IP address as well as a rough location based on the IP. It also shows you a Cheating Probability score, which is calculated on a number of things, most importantly was the candidate navigating away from the screen to other windows or apps on their computer.
Transcript, Next Steps, and Applicant Details
Back at the top of the page, you'll notice tabs for Transcript, Next Steps and Applicant.
Transcript
The Transcript is, as it sounds, a complete record of the entire from start to finish. Not only that, but you can see how long it took the applicant to begin the answer right above each blue bubble.
Note: If your Admin has turned off Bias Prevention, you can also listen to the answers your applicant gave. A Listen button will appear below their answer.
Next Steps
Clicking on Next Steps, you'll see suggestions that you could provide to your hiring manager, or use yourself in the next stage interview. These suggestions offer a question to ask, and why this has been recommended.
Applicant
Applicant details show specifics, only if Bias Prevention is off. Otherwise, and in most cases, you won't see anything in this tab.
Now you're ready to go! Let us know if you need help by reaching out to support@talentllama.com