Internal Release Notes
4.14.0 Our latest feature releases, product improvements, and bug fixes
New Features
Content Marketing Simulation
- The final marketing simulation is here for a content marketing manager!
- Includes a task where candidates will review a competitor article and then create their action plan to get something similar live ASAP. This task will let them showcase their observation and critical thinking skills and put it all into practice within a realistic timeframe.
- The simulation is available to all paid customers
- The simulation consists of 9 tasks;
- Task 1 – Strategic Thinking – Google Doc
- Candidate sees a task asking them to finalise their content marketing strategy for a new product to present to the marketing team at the end of the week – link in email to relevant KB article
- Task 2 – Setting Objectives – Text
- Candidate then sees an email thanking them for the strategy and asking them to identify 3 short term goals for the strategy. Candidate must respond in the email
- Task 3 – Content Calendar – Google Doc
- Candidate sees another follow up email asking them to update the content calendar in support of the new strategies
- Task 4 – Copywriting – Google Doc
- Candidate is asked in an email by their manager to optimize an old under performing article that gets a lot of traffic due to some strong keywords
- Task 5 – Design Briefing – Text
- Candidate then sees a prompt to create a brief for updating images in the old article and work collaboratively with the design team to execute
- Task 6 – Verbal Communication – Audio
- Candidate sees an email back from a design team member asking them give them a quick call to clarify a specific part of the brief
- Task 7 – Deadlines and Delegation – Spreadsheet
- Candidate sees an email from the CMO asking for help to meet a crazy deadline and must allocate a series of tasks to the right team members and brief them properly to ensure they meet the deadline
- Task 8 – Deadlines and Delegation – Video
- Candidate must then brief the team on the tasks they’ve been asked to do via a video call
- Task 9 – Creative Writing – Text
- Candidate sees an email from the CEO outlining a request to create a short content piece that explains the companies unique position in market for an investor
- Task 1 – Strategic Thinking – Google Doc
- The simulation can be accessed via the simulation question type in the Create editor
- An assessment has been added to the public library which contains the simulation – Content Marketing Manager
- GTM notes are here
Screening Questions
- Screening/eligibility questions are here and available to all plan types!
- Screening questions sit at the beginning of an assessment and are mandatory for candidates to complete before they can advance to the assessment.
- Customers can create their own or choose from 8 preconfigured options in suggested
- Screening questions have no skill group and are either Single or Multiple Choice type questions with a `pass` or `fail` allocated to them.
- Candidates that do not pass screening questions don’t count towards completion caps and aren’t visible in Select.
- Candidates who successfully pass the questions can then start the real assessment. Unsuccessful candidates that fail will not progress to the assessment.
- An assessment’s screening section has no timer, meaning assessments with timers won’t start until a candidate successfully passes the screening component.
- Extention request rules remain the same if the candidate started and extended we show the place from when the candidate stopped
- GTM notes here
Custom Assessment Flow
- All Enterprize customers can now request and manage their custom assessment credits in app.
- Customers with existing credits will be able to see how many they have and initiate a request for an assessment
- Customers without credits will be able to request them via sales from the app.
- In order to support this flow the `Create Assessment` header button has been renamed `New Assessment`
- The custom assessment flow is now visible to all account users and can be initiated by clicking on `New Assessment` and following the Request Custom Assessment flow.
- They will be asked to upload their JD and asked a series of simple questions about their role including the key focus and main challenges of the role.
- Once the request has been submitted an email will be sent to support/CMS admins with the details of the request.
- Custom assessment requests are then managed in the CMS
- A new header tab has been added in the CMS account titled `Assessment Requests`
- Clicking on a request will open a card with the details of the request
- CMS admins can mark the request as `done` once complete
- Requests can be filtered by status
Improvements
Segment Events
- Added event ‘A-stripe-payment-succeeded’ for tracking company revenue. This can replace the Stripe integration in the #wins-stripe channel to reduce the failed charges noise
Re-Order Skill Groups `Customer Request`
- Customers can now reorder skill groups giving them control over which questions come in what order.
- Reordering occurs for the `settings` page and is accessed the same as question reordering by moving the skill group up or down the list
- If a customer reorders during a live hiring process the changes will only be seen by candidates that start after the changes are made
Candidate Cards `Customer Request`
- Status history has been added to the candidate card that logs the time and date that the status was changed for the candidate.
- This will help customers determine things like fairly granting a deadline extension request or if a candidate has previously been messaged
Deadline Extension Request Reason `Customer Request`
- Candidate’s can now add the reason for requesting a deadline extension
- This will allow customers to more accurately access the extension request
- Customers can see the reason as part of the extension request and on the candidate card
Assessment Validation
- The ability to invite candidates will be locked if there’s no at least 1 valid question in the assessment
- Questions that are AI-graded will be considered invalid if there’s no sample answer
Billing Page Updates
- The candidate packs table has been merged into a tab on the account details/billing page making it easier to navigate when there’s been a long purchase history
- The order of candidate pack purchases has been updated so that most recent is now visible first
Library Search
- Updated library search so that all filters reset when a new search is made from the search bar
Bug Fixes
- DT-4909 questionCount returns wrong value on Candidate’s dashboard in case there is a group with non-gradable questions only
- DT-6236 Wide image makes the assessment application page unreadable on mobile
- DT-6392 FE – Prevent Candidate from selecting several options in the Single Choice question when clicking on them at the same time
- DT-6445 “Preview” and “Copy preview link” are not enabled immediately after adding questions from Saved
- DT-6446 Failed Candidate gets 4 rejections email after submitting Excel assessment with auto-reject
- DT-6479 FE – “Request more time” and “Time extension requested” buttons are hidden for candidate on mobile if we don’t allow to start assessment on mobile
New Features
Candidate Pricing
- New purchasing option of candidate credit packs is live!
- Initial options range from 10 candidates up to 500 with 5 and 1000 being added at a later date to test price anchoring
- Users can purchase multiple credit packs in the same account
- Each pack has a 12 month expiry date
- Those with the earliest expiry date will be used first for candidate completions
- Platform usage with the packs includes;
- 10 active assessments
- Max 1000 invites per day
- Unlimited Invites
- Unlimited Assessments
- Standard Assessment Library
- Standard Assessment Builder
- AI Lite
- CSV Exports
- Company Branding
- Email Support
- Standard Terms
- Standard Integrations (Zapier, Jobvite, Jobadder, Greenhouse, Lever)
- Customers can view their candidate pack expiry dates in their Account Details/Billing Details page
- Locked candidates will now sit at the top of an assessment making them easier to see
- Customers on Pay&Go can switch to these candidate packs in app while customers on older Plans will need to be manually removed from their existing plan and then they will be able to purchase.
AI Health Lite – V1
- AI Health V1 only includes the updated status bar on Invite and Select that shows a static Optimized status
- In this release this modal is not clickable
- The following plans will now see this by default
- Free Trial
- Pay & Go
- Candidate Packs
- Premium
- Premium+
- Appsumo
- Old credit card plans and all Enterprise customers will remain with the full version of AI Health by default
- The V2 release (which is next) will contain the grading functionality as follows’
- 3 statuses only
- Optimized – no action is required
- grading required – at least 100 candidates have completed and some grading can be done
- review and score – new compulsory manual scoring of some AI questions (presentations, document uploads and google sheets)
- 3 statuses only
Improvements
Plagiarism V2
- An updated version of plagiarism is available at an additional cost to all customers.
- This enhanced version of plagiarism includes ChatGPT and web-based content matching to determine if someone has copied the text from these external sources.
- Customers can request this additional feature through support/sales and it can be enabled via the admin panel in their account.
- The feature uses the existing plagiarism flag with an additional note outlining the source of the original content eg. https://www.wikipedia.org/
Create Assessment Builder
- The new Create Assessment Builder uses our own skills taxonomy for top roles across key industries
- Users will select the category of their role type and can then search our database for a matching role. Skills are then automatically matched to the role and the base of an assessment generated.
- Users can create their own questions, use their saved bank or use one of our suggested questions. Each skill now has a series of quality questions that can be selected from.
- The name of the assessment will no longer populate a skill group within the assessment
- Users that select other will be put through our old manual workflow where they must select their own skills and assessment name.
Candidate Feedback
- New Segment events have been added to track usage of the settings and to properly monitor and track the sending
MFA
- Updated the validity period to be 10 minutes for the verification code to give the email enough time to reach the customers inbox
Comeet Integration
- Updated the Comeet integration approach: We replaced the “Retrieve an Evaluation” webhook with the “Submit an Evaluation” endpoint to enable real-time status updates for assessments. This transition reduces server load caused by frequent requests to the old webhook.
Onboarding
- Removed the Project Manager Simulation card from onboarding (assessment library step)
Premium Plans
- Updated Premium plans registration link behaviour so that existing free trial users can use either to access the new plans
Segment
- Removed old Wootric from code and segment
- Added segment event AL-scores-exported to track downloads of manual scores from AI Health
Appsumo
- Daily invite limit now set to 1000 candidates
Bug Fixes
New Features
Candidate Feedback
Feedback for Candidates is here and available to all plan types including those on Free Trials and Appsumos.
Customers can enable feedback sending via the `Settings` of their assessments. By default it is disabled.
- Feedback can be sent to candidates at the close of the assessment or at a specified time after they complete (the minimum time allowed is 3 days to allow AI calculation to settle and the maximum is 30 days)
- Assessment admins are also able to trigger the feedback individually to candidates via their candidate report card
Customers can preview and edit the default email template used to send feedback to candidates
A sample PDF of the feedback can be viewed by clicking on the preview link (this is a generic sample unrelated to candidates in their assessment)
- Customers can remove the candidates rank from the feedback in the settings
- A feedback sent icon will appear next to the candidate once feedback has been actioned
More info here in the release notes
New Low Touch Plans
We have 2 new ‘low touch’ plans for sales to use, Premium/Premium+, for customers that don’t have the budget for an enterprise deal.
These plans are only available via the low touch landing page. Customers must register via this page in order to access the plans in app.
The plans are:
Premium | Premium+ |
$5000/yr | $9000/yr |
Up to 10 hires 2 assessments5 users1000 completions | Up to 20 hires 5 assessment5 users2000 completions |
Payment is via credit card
- Customer on these plans will not have access to enterprise features like a dedicated account manager and personalised onboarding. Instead they will be given an in app onboarding experience via Appcues and email
- Because the accounts have a company level completion cap (which is not something the product can support today) an alert will be sent to CS when one of these accounts is nearing it’s completion cap.
Improvements
Saved Questions
Customers can now save questions they like from the public library when they are previewing an assessment
“Saved” link will be disabled for customers on free and pay&go plans
Deprecated Features (Internal)
- Remove ATS FLOW code 👋 (more to come)
- Removing the old SmartRecruiters v1 integration from data base, code + Admin panel now that all customers are on V2
API
Added an endpoint for un-rejecting candidates via API
Enableld un-rejecting candidates in app if they were invited via Vervoe API
Added webhook URL to Vervoe API logs when sending the reports
- Also, added the field notifyEmployerUrl to the event user_audition_report_task_created
Deadline Extensions
In response to a problem register request `Problem: City of SF and other government customers would like to know if a candidate has previously had a deadline extension as this could unfairly give them an advantage over other candidates` We have added an assessment ‘Extension Received’ label to the candidate row on the Invite page + Candidate Report Card
Updates deadline banner text so it says `Assessment Settings` instead of `Create Settings`
Anti-Cheating
Added anti-cheating filters to the Select list so it’s easy to see or hide those that have been flagged
Team Members
When managing a team members assessment permissions it’s now clear the count of both open and closed assessments they have access to.
Show Create + Invite tabs from the assessment page for Team Viewers
Save the history of Team Member rejections and re-invitations to the Admin panel
Free Trial
Free Trial assessment deactivation/reactivation rules change so an assessment must have a completed candidate before it’s unable to be deactivated
Free Trials will now expire after 7 days instead of the current 90 days. Accounts created before the release will stay at 90 days.
Free Trials will now be able to create assessments via the builder
Closed Assessments
The ‘Tag/Untag’ button will now be disabled instead of hidden for closed assessments
Enabled Create and Settings for assessment Admins
Questions can be saved from a Closed/Archived assessment
Compliance
Removing candidate phone property from the events: `G-candidate-phone-changed`
`A-vervoe-admin-signedin-candidate-BE`
`A-candidate-registered`
`A-candidate-signedin-BE`
+ Identify call
Segment
Added ‘icims’ integration property to candidate’s events to track integration usage
Grading Updates
All time a Candidate spends on a question before they close the browser tab with the assessment will be recorded
Select Updates
Added a tooltip on hover with the full skill name in Sort on Select page to make it easier to see what’s being deselected
The custom candidate sorting setting of Select/Invite + All Candidates/Hired lists will now save even with a page refresh or closing/opening of the assessment
The heading of the column with the sort selected will now highlight making it easier to understand what the list has been sorted on
Hired
Added in additional information to the grading scale of the hired survey to help hiring manages understand where to place a candidate on the scale
Permissions
Disable the quick time deadline extension request list for Team Viewers
Image Updates
Updated Twitter (X) icon inside the employer app
Updated Oracle icon on the Integrations page
Bug Fixes
DT-6151 FE – Add loader when importing csv file on Invite page
DT-6209 There is wrong link on button “skill groups” on editing question
DT-6254 Activity feed didn’t refresh in the task 6 of Account Manager Sales Sim until reopening tab
DT-6332 Logged in Employer is redirected to the login page after clicking on Reset password link from the email, if their refresh token is expired
DT-6363 [Internal] Full name and Email titles are cut on text fields on Invite tab
DT-6381 FE – ‘TT-eeoc-enabled’ event is not triggered
DT-6384 Team member can see wrong image for several seconds on registration page
New Features
SEO Simulation
- SEO (Search Engine Marketer) Simulation is here! This features 9 tasks across a range of skills like data analysis, copywriting, strategic thinking, and SEO
- Task 1: Data Analysis – Look at reports for site traffic to understand patterns, volume, and sources and record answers
- Task 2: SEO – Use tools to check the setup of the page to ensure the right technical capabilities are in place for it to rank on search engines
- Task 3: SEO – Candidate Is then prompted to review the site structure and create a sitemap noting if core pages are missing or if there’s an internal link structure.
- Task 4: Strategic Thinking – Candidate is then prompted to respond in an email with notes about the site map and suggestions for missing pages and improvements
- Task 5: Written Communication – Candidate must review a specific page of the microsite using dev tools and make then make suggestions back in a Google doc with a predefined template. Candidate must also use knowledge base articles to understand business ICP
- Task 6: Verbal Communication – Candidate must then brief the content writer about the changes they would like to see to these headings in a short audio message
- Task 7: Copywriting – Candidate sees the next task in an email from their manager asking them to compose copy for a specific landing page including H1, H2, and paragraph text
- Task 8: Off-Page SEO – Candidate sees an email from a senior stakeholder asking for recommendations for off-page SEO
- Task 9: Written Communication –
- Candidate then sees a response from the senior stakeholder asking for them to start link-building outreach. Candidate needs to compose an outreach email asking a publication for a backlink
- Candidates will be tasked with reviewing a real live microsite to determine its SEO health and make recommendations for areas of improvement. They’ll also be able to showcase their data analysis skills using Amplitude reporting to make observations about site traffic.
- Candidates will be given 1 hour to complete the simulation
- Available to all Enterprise customers and via the new onboarding
- Can be added via the library `SEO Marketing Simulation` or via Create as part of the Simulation question type
- Simulation is not mobile-compatible
GTM notes here
MFA
- MFA has now been implemented.
- For existing accounts, it’s off by default and can be enabled by Admins/Owners via the team member settings page
- For new account, this setting is enabled by default
- Our MFA implementation requires the user to log in using a 6-digit code that is emailed to them (this is the same approach for account validation on new sign-ups).
- The code expires after 5 minutes
- More info here
Improvements
Cyber Simulations
- The Cyber Simulations will be removed from the app now that technical roles are not a focus. Uptake has been limited and there’s a substantial cost to support the simulations.
- They’re being “powered down” so they are able to be released into the app again when required.
- There’s one customer with some completions on a now deactivated assessment that has been notified. Past results will be saved on candidate cards.
Lever
- Added functionality to refresh the authorization token for lever to avoid disconnection issues
- Added an info banner to the customer app to show employers that Lever is unauthorized and needs to be reauthorized or candidates won’t be invited.
Greenhouse
- Functionality updated so that if a candidate is reinvited they are not sent the invite links
SmartRecruiters
- We’ve updated the trigger for the event `ai.score.changed` so that that update will only send if the score changes by more than 2%
- All SR users are now on the V2 integrations and V1 will be deprecated and removed from the UI
Assessment Regeneration
- Team Members permissions are now copied for regenerated assessments
Deleted Assessment Behaviour
- If a customer previously knew the URL of a deleted assessment they could still access it now we show a “not found” page
- Previously candidates from deleted assessments weren’t shown on the Hired page this has been updated to now include them
API
- Updated the timeframe starting point for invite-with-reapply-timeframe endpoint so that it is now calculated from the time of invite and not the time of completion
Permissions
- Enabled the closed assessment page for Team Viewers who previously could not see it
Copy Assessment Preview Button
- After feedback that it was confusing, we’ve updated the assessment preview link copy from Link to Share to Copy Preview Link
Bug Fixes
- DT-5965 Info question isn’t marked as answered for Candidate for the first time
- DT-6064 Unnecessary rounding of assessment/question score
- DT-6271 BE/FE – Employer is able to copy unpublished assessment from the Public Library.
- DT-6284 Auto-progress setting isn’t saved on UI until page refresh
- DT-6292 “score”: 0 in “List of candidates” Vervoe API endpoint if candidate’s answer doesn’t have score
- DT-6293 Nothing is exported as a question text in the csv file and in the CandidateDetailsQuestionObject for Marketing Simulation
- DT-6309 Wrong questions order for Candidate on assessment with group randomization
- DT-6310 [Internal] Blank page and console error for candidate on Task 5 (Review email + Email) of Growth Marketing simulation
- DT-6338 Zendesk is shown in the attachments from Growth Marketing (Task7), SEO Marketing (Task 6, Task9)
New Features
Close Assessment Update
- View only’ mode for closed assessments or new closing behaviour – ability to see all candidate and content data but not make changes to anything that could impact a candidates
Assessment Re-generation
- On the click of the ‘Re-generate’ button the existing assessment will be moved to an archived state and a new version will be ready to use that holds the original settings and assessment slug.
Improvements
CSAT Score
CSAT scores and candidate feedback has been added to Candidate Report Cards (internal, public, integration)
SmartRecruiters
Added the setting to SR integration for managing sending the Candidate Integration Link
Vervoe API
Added endpoint for API to get candidate details same as details from the CSV export per assessment
Candidate Experience
DT-6042 If assessment application link is broken, the ‘Not found’ page is shown for the Candidate
AI Health Modal
AI Health modal has been updated to full screen size
Assessment Page
Updated the ‘Copy assessment preview link’ button from the assessment header
CMS
Added ‘Instructions to candidates’ field to CMS assessment settings
Admin Panel
Removed deprecated the ‘Hire per Month’ setting from company level in Admin panel
Internal Improvements
Added logs of employers’ logins
Bug Fixes
DT-6162 Employer needs to reopen Excel/Google Spreadsheet auto-graded question in order to get updated data about sheets, column and row ranges
DT-6196 Fixed the issue with Question/Answer/Score expot to CSV if question was deleted
DT-6235 Fixed the text of question preview from the right side question bar if the assessment is timed in candidate app
New Features
Growth Marketing Simulation
- The first of 3 marketing related simulations to be released this quarter
- Contains 7 tasks – 5 of which are AI graded
- Task 1: Strategic Thinking – Review company data in the knowledge base and then create the profile of low-value customers that will be used to identify them later in a chatbot flow
- Task 2: Marketing Funnel – Create a workflow using a builder that aligns a specific user to the correct journey based on their company details
- Task 3: Communication – Create another message option to AB test to welcome the user and direct them to the right space
- Task 4: Reporting – Update a forecasting spreadsheet with the relevant conversion numbers for the workflow they created
- Task 5: Analysis – The candidate must use a data dashboard to assess how the new workflow is going and communicate their findings back to stakeholders
- Task 6: Lead Gen – Candidate must identify areas of opportunity to promote using a new budget
- Task 7: Lead Gen – Candidate should identify appropriate channels for each campaign based on spend and ROI
- Available to all Enterprise customers and via the new onboarding
- Can be added via the library `Growth Marketing Simulation` or via Create as part of the Simulation question type
- This is a timed simulation with candidates being given 1 hour to complete
- Simulation is not mobile-compatible
GTM notes here
Transcripts
- Added audio and video transcripts to Candidate report cards
Improvements
Company Branding
Improved the experience of adding your company logo and increased the size of the logo in the candidate experience from 36 px to ~44px
Not Mobile Compatible
Added a notification and icon to create making it easy to see which questions are not mobile compatible
CSV Uploads
‘Something went wrong’ will now display instead of the endless loader for timeout errors on Invite and CSV uploads
Candidate Experience
Refreshing a page in the candidate app previously sent candidates back to the first question now they’ll remain on the question they were on.
Jobvite
A custom field has been added in Jobvite for showing errors that can occur when inviting a candidate eg. trial expired, account inactive, assessment inactive
Excel/Spreadsheets
DT-6187 Remove single expected result for test cases in Excel/Spreadsheet auto-graded questions
Data
[internal] Assessment will be deleted from the Data Base when it’s deleted by a customer on the front end if there are no candidates invited to this assessment
Bug Fixes
DT-6234 Turn off setting “eeocEnabled” in CMS account assessments and public assessments
DT-6241 [Internal] There is no transcription for Sales simulation question type in all Report Card
DT-5964 Old candidate’s data (before reapply) is exported to csv file after candidate has reapplied
DT-5965 Info question isn’t marked as answered for Candidate for the first time
DT-6035 [Internal] Blank screen and console error when Employer tries to create Sales Sim question
DT-6036 There are no scores by skills for Sales Simulation and QA Simulation questions on the Agency Card after the Guest graded the candidate
DT-6093 500 error on adding Sales Simulation question type on Create page
DT-6094 BE – Assessment brand is copied from another company when assessment is duplicated via Admin
DT-6193 FE/BE – Sales/QA Sim question preview contains irrelevant text from the right side question bar
New Features
QA Simulation
- QA simulation is here 🎉 This simulation contains 5 tasks;
- Info task (User Story) – not graded
- Test Case – autograded using MC
- Test Run – autograded using multiple MC single choice
- Edit Bug – autograded using multiple MC
- Create Bug – graded via AI using text questions
- Candidate should be able to see walkthroughs when starting the QA simulation for the first time that highlights each of the workspace areas including task panel, issues panel and response area
- Candidates will be tasked with reviewing a live product that they will need to complete testing on in order to answer the simulation questions
- The product is based on a calendly type booking tool called ACMi and has 2 core functions
- A user can create an event that others can book
- A user can book an event that someone has created
- Candidates have 1 hour to complete the simulation
- Simulation cannot be completed on a mobile device
- More information here
Improvements
Billing Page
- Update to show number of remaining open assessment slots on the Billing page to avoid confusion
Excel and Spreadsheet autograding
- Adding actual and correct values per test case to candidate report cards for Spreadsheet auto-graded and Excel questions to make it easier to see scores that need to be amended
- Customers can now add multiple correct values per test case for Spreadsheet auto-graded and Excel questions meaning they can add variations of the correct answer to be graded as correct
Internal Admin Updates
- [Internal] Adding a log of candidate rejection/unrejection to the admin panel including the following details
- candidate’s ID
- assessment ID (or slug)
- date of the rejection/unrejection
- employer’s ID who reject/unreject this candidate
Candidate Time Taken
- `timeTaken` on the candidate card won’t exceed the time for timed assessments regardless of the scenario:
- If assessment has timer:
- We’ll hide “QUESTION COMPLETED IN X” on the question level and only display the total time taken
- If candidate completes assessment for the first time without an extension but the time taken is greater than the timer (due to multiple video or audio attempts or multiple tabs) then we will only display the maximum time of the timer
- Eg candidate took 1 hr and 10 min due to working on questions in multiple tabs but the assessment has a 1 hr time limit we will only display 1 hr for this candidate’s time to complete
- If assessment has timer:
Onboarding
- “Calculating score” has been removed candidate cards when the Cyber Sim is selected in the new onboarding flow
EEOC
- EEOC is now off or disabled by default and you will need to enable it to use the feature
Bug Fixes
New Features
New Onboarding Flow
- The onboarding flow will exist for all new users that register on the free trial and will showcase the candidate experience of several simulations
- The simulations to begin with are sales, customer service, cyber security and accounting. Marketing and QA will be added once released.
- Users will be able to experience a simulation and then see their score drop into select with a range of other “fake” candidates
- They will be able to view their candidate report card and play around with the filter and sort functions of Select
- If a user closes the app before completing or reloads the page they’ll be returned to the last step they were at
- Those on mobile devices will see a notification prompting them to come back on a desktop device. If they continue on a mobile then they’ll be dropped into the existing free trial without the onboarding as simulations don’t work on mobile
- The free trial is being updated to include the use of simulations so that users can select one after the onboarding walkthrough is complete
Invite Resend and Assessment Permissions
- Add a team member to relevant assessments and assign permissions when you invite them. Owners and admins can manage assessment permissions on invites removing multiple steps to invite and allocate a team member to each assessment individually.
- Individual permissions can be managed from this global view also.
- Resending invites to team members that haven’t accepted is also now available via the invite panel
Unreject Candidates
- Unreject is now available via the app! This will help customers undo accidental rejections quickly. For candidates that have been rejected the `rejected` option on the drop-down modal will be replaced with `unreject`.
- Once a candidate is unrejected their status will return to their last status prior to rejection and their score will be reestablished. They will also be given a rank in the select view.
- Only admins or assessment editors can unreject a candidate
- Candidates invited via integration cannot be unrejected as the endpoints don’t exist in the third-party tools to support this
Improvements
[This Release] Close Assessment by Deadline
-
Candidate experience of closing an assessment by deadline has been updated so that candidates won’t be able to continue or submit the assessment once the deadline has passed - Candidates will be able to request more time to complete the assessment that the employer can review
[This Release] Onboarding
- Once the onboarding starts there’s now no option to go back a step
- Onboarding data will be deleted as soon as the Free Trial expires
- New events added for skipping the onboarding OB-skipped-onboarding, OB-skipped-during-candidate-experience, and OB-skipped-select-results
[This Release] Segment Events
- Sending candidate events from the employer side so we can have completions back in Amplitude
- Removed the old woootric_nps event from the employer app
[This Release] API Updates
- We won’t create a report if there is no webhook URL for a company with Vervoe API integration.
Seek Candidate List
- Added the ability to import a candidate list from SEEK for inviting candidates to an assessment
- Feature must be toggled on in admin first
Assessment Insights Icon
- Assessment Insights icon has been highlighted on the closed card to show that it can still be accessed without reopening
Segment
- DT-3622 Recall identify/group call with plan upgrade, company deactivation and account/company settings update
- Google Analytics clientID has been added for the backend account created event to assist the marketing team with reporting in GA
AI Optimize
- We’ve renamed the optimize statuses so that Low – remains, Average is removed and replaced by Medium, Medium becomes High and High becomes Optimized. Medium has also been recoloured to green so now all statuses above low appear to be good.
- New lines or line breaks in the text for Team Scoring Instructions and Correct Answer Example fields will now display correctly instead of the text just displaying in a block which is difficult to read.
- an info icon and tooltip have been added on hover over Avg Team Score text. The tooltip links to this help article to assist customers in understanding how to use the functionality
CSV Exports
- Question content and score have been added to the CSV file as additional columns per question
- The score column will only contain manual or autograded scores and not AI graded scores
- Rank has been added to the CSV file. A toggle has also been added to the export modal as an opt in given that this feature slows the download speed considerably. (can take about 5min to generate now)
Sales Simulations
- Simulation questions will now be shown in the dropdown for custom score sort
- The video preview of the Sales Simulation has been added
- The list in the accordion under the sales simulation skill on the candidate report card is now open bey default instead of closed
- The count of total tasks in a simulation has now been added to the task panel so candidates can see how many tasks they have to go.
Google Sheets
- Updating the view on the candidate report card to be view link instead of preview which will show all rows numbers and context information making it easier for users to grade a specific row or column
Login Page
- Login page is being rolled back to the old design that doesn’t reference the free trial benefits
Laptop/Desktop Wording
- Updated the restrictions warning for candidates with questions that can’t be completed on a mobile so that the text includes both desktop and laptop
Integrations
- Candidates can’t be invited via any integration when an the assessment has an expired deadline or the assessment is inactive
- Improved sending score updates to Greenhouse to give priority to the newest score updates, so that old ones with errors don’t slow down the queue.
Candidate Cards
- Assessments with autograded questions won’t get an AI score for those questions to speed up grading
- Added BE vs FE comparison of time candidate spent on question so that if the BE notices that time spent on a question is too big, it saves it’s own time to DB based
Bug Fixes
- [This Release] Excel test cases are now copied when the assessment is duplicated
- DT-5261 There is 400 error when Candidate upload certain PDF file as question answer
- DT-5406 FE – Long text in MC/SC question choices isn’t split into the rows
- DT-5566 MC/SC question is shown without answer choices on the assessment preview if it has at least one invalid choice
- DT-5672 Issues with text overlapping the audio/video attached to the choice in MC/SC questions on assessment preview
- DT-5680 BE/FE – Employer should be able to save the question from Suggested tab
- DT-5738 Redundant requests were sent after deleting spreadsheet auto-graded and excel question type from assessment
- DT-5778 FE – Score isn’t shown for some auto-graded questions on Compare modal
- DT-5825 [FE] Employer continues searching by searchterm after it was removed from the search in Assessment Library
- DT-5836 Button “Replace file” is not visible after page refresh in File upload type of question
- DT-5837 [BE] Wrong candidate source numbers on statistics shown on Assessment Insights page
- DT-5840 Answer with sign “%” in cell doesn’t score as correct one in Excel
- DT-5847 [Candidate App] If the prospect has a long name in Contacts, the label “Survey Complete” overlap the name
- DT-5848 Something went wrong and 404 status if Company has no existing branding in the assessment and tries to add a new one on the Settings page
- DT-5851 [FE] Extra quotes signs are sent to BE when exporting csv from Insights page
- DT-5852 [FE] GST isn’t recalculated on UI after applying discount coupon
- DT-5869 Team Scores List isn’t shown for assessments with Sales Simulation
- DT-5876 Fix the company removing
- DT-5879 “Clear all filters” button doesn’t actually remove search term from the request on Assessment Library
- DT-5894 Candidate was able to submit several choices for Single Choice question
- DT-5916 It’s impossible to see the Preview of the Excel document in the Private Library and Suggested tab
New Features
Sales Simulations
- Version 1 of the Sales Simulations is here!
- New Sales Simulation question type with 3 simulation workflows
- Sales Development Representative
- Account Executive
- Account Manager
- Each simulation contains a series of interlinked tasks that are based on existing question types like text, MC, video, and audio
- The candidate will be presented with a workspace containing a list of contacts, a series of knowledge base articles that relate to the product they are selling, and a set of up to 10 tasks they must complete
- Candidates cannot skip tasks or go back and change a response to a task once submitted – like in real life!
- A walkthrough will guide the candidate around the interface prior to the start of the simulation
- Candidate tasks include things like cold calling prospects, handling objections in the sales process, and taking notes during a call
- Each task will be shown individually on the candidate’s report card so employers can review audio, video and text responses
- Each task is associated with a different skill
- Prospect names will randomize for each candidate as an anti-cheating measure
- V1 will not support active learning via the AI health modal or the compare view
- Simulations are not mobile compatible
- Questions are not localized (Spanish version unavailable)
- 3 Assessments in the public library contain the simulations
- Sales Development Representative Simulation
- Account Executive Simulation
- Account Manager Simulation
Improvements
AI Health
- Employers can now export a CSV of all the manual scores from team members via the Optimize modal. The export will contain all of the manual grades from the assessment as well as a column that shows if a grade has been flagged. @jess requested
Candidate Experience
- Candidate experience has been updated to that they are redirected to the application page if they are logged out
Billing
- If a company is upgraded via Admin the ‘Cancel’ button will be removed from the Billing page as it currently does nothing
- Update info about candidate limits on the Billing page to specify it’s per assessment/time period
Assessment Branding
- Email previews in app will now reflect the correct branding that has been selected for the assessment and not the default company brand
Bug Fixes
- DT-5406 FE – Long text in MC/SC question choices isn’t split into the rows
- DT-5513 Fix setting/checkbox “Randomize question order for each candidate” for users without QUESTION_RANDOMIZATION feature from Skill Group Settings
- DT-5610 BE – Removed team member remains in “reply-to” field in custom invite/reminder/rejection email
- DT-5730 Category name is clickable on public assessment card
New Features
Excel Question Type
- Our latest Excel question type allows customers to natively embed MS Excel into any assessment.
- They can do this via the Create editor where they are prompted to create from scratch or upload their excel file (max 10MB).
- While we natively have Google Spreadsheets as a question type we haven’t supported the same functionality for MS Excel. This update will now allow users to select between Google Spreadsheets and MS Excel to test functionality and role-relevant skills correctly. Embedded MS Excel also will enable us to expand into other MS Office applications supported by the API.
- The Excel question type is available to all customers across all plan types. The question type mimics the functionality of the Autogradable Spreadsheet question type; autogradable, with at least one correct answer sample or test case.
- Some important considerations to note are customers are able to amend scores after grading to account for answers that are technically correct but marked incorrect due to formatting (i.e. 10 vs 10.00) and cannot be completed on a mobile device.
- The Excel question type doesn’t currently work in an iframe but this is being updated.
- Once this is updated, we will update the spreadsheet questions to the Excel question type for these assessments: Financial Accountant, Finance Manager, Payroll Administrator & Pricing Analyst.
Improvements
Anti Cheating
- Candidates are no longer able to copy the text or content from a question. This will make it more difficult for candidates to copy and paste content into tools like Chat GPT.
Segment
- SmartRecruiters V2 event added S-smartrecruiters-v2-enabled
Candidate Cards
- The number of candidates being shared has been removed from the header on public candidate report cards to avoid confusion (David request)
Statistics Bar and Assessment Card
- The colours of the percentages on the statistics bar, small assessment card, Select/Invite statistics, and Insights have been updated to remove the colour bands. They are now all grey which will resolve the issue of low completions looking bad when it’s actually expected behaviour. (Problem register fix)
Assessment Library
- By default assessments in the Public Library are now sorted by popular so assessments with good data sets should appear first in the results (Mel request)
Bug Fixes
- DT-5701 It’s unable to copy assessment preview link, share candidate card on Safari browser
- DT-5705 Unchecking checkbox in filters doesn’t remove filter in Assessment library
- DT-5721 linuxSimulationUseCaseResults aren’t shown on old candidate’s cards
- DT-5723 [CMS] There is 500 error when publishing assessment from CMS
- DT-5733 timeSpent includes only time Candidate spent on question before clicking “Download file” button
DT-5742 BE – Employer doesn’t see Candidate’s answer on report card
New Features
Cyber Security Simulations
- Our latest simulation question type `Cyber Security` contains 6 use cases;
- check-sudo
- failed-logins
- update-firewall
- web-content
- web-redirect
- Ssh-guard
- Each use case is completed within a live terminal that’s embedded within a question. Candidates will see the question down the left hand side and a network diagram if it’s applicable to the question. Multiple windows can be opened at once via the tab system in the terminal panel.
- Each use case is completely autograded.
- Simulations can only be completed on a desktop.
- A Spanish version is available.
- Customers can add the simulations to their own assessments by selecting the `Simulations` question type and then selecting the type of simulation eg Cyber, Customer Service.
- Simulations are not editable by customers however new use cases can be built so we welcome requests.
- X assessments have been added to the library for a range of different roles that use the simulations
- The candidate card will show the breakdown of grading for each simulation including what they should have got and whether or not they passed that stage
Please note: Extending a deadline for this simulation will wipe any data the candidate has already input. A warning has been added to the deadline extension modal for this
Question Weighting
- New weighting per question type has been added as in above image
- This means questions that require more input and therefore give more context for a candidate’s abilities are now worth more eg. Simulations and questions that require less effort like MC are worth less
- This will create better variation in candidate scores and should promote better candidates to the top of the list
- Example of the change:
Improvements
Update Assessment Library
- Assessment library has had a refresh to make it easy to filter assessments by category, skill, level, or question type.
- Assessments can also be sorted via relevancy including date, popularity, and alphabetically.
Candidate Source
All integrations have been added as a candidate source
This is visible in the invite list, as a filter in select and on the candidate card (as above)
AI Health
Audio and video questions now have transcripts available in AI Health grading
- The field is collapsable and closed by default
Segment
Lever and Oracle have been added to segment S-oracle-enabled S-lever-enabled
Security
Vervoe Engineering team can log a user out (candidate + employer) from all sessions/devices on request
Added a 404 page for Shared Assessment preview page if Company was deleted
API
- Geolocation and plagiarism flags can now be accessed via the candidate assessment report
- Added ability for an API user to use a candidates previous assessment score if they reapply within a certain timeframe
CMS
Skill level label has been added to small assessment card in CMS
CMS Admin can manage Popular Skills list for public assessments from the CMS admin account
CMS Admin can add/remove tags/quick searches to assessments for public library
CMS Admin should be able to manage Assessments Tags/Quick Searches from the CMS account
Remove new/old tags from small cms assessment card as they are no longer required
Question Types
Google files are removed from shared assessment preview when a company account is deleted
UI Changes
Update assessment banners text about limits to remove unnecessary upgrade messaging
Updated Vervoe favicon icon
Reorganize the main navigation header to remove multiple CTAs
Updated so Assessment Branding shows on the internal assessment preview instead of the company branding
Update steps of Setup Guide; instead of create assessment it’s now choose assessment
Bug Fixes
DT-5302 There is wrong Candidate answer on Candidate’s report card
DT-5403 BE – Plagiarism detection in question settings is always turned off for question that was added to/from Saved tab, Suggested tab, Public Assessment
DT-5481 Empty city name causes the marking candidate with geolocation fraud detection flag
DT-5549 When Employer chooses “Start fresh” on opening Assessment, there is still statistic about candidates from the previous original Assessment
DT-5551 There is no “Question 1” in title when Employer opens Google doc file
DT-5554 Candidate with inactive userAuditions are selected for reminders to be sent
DT-5555 Text of questions isn’t trimmed in the list on reordering questions in Assessment
DT-5560 [Comeet] Candidate invited via Comeet has “assessment link” as a source
DT-5564 “graded by” tooltip on Select and Shortlist has wrong padding
DT-5582 Expired candidate has “applied” status if they applied after the expiration
DT-5584 There is 500 Error when Employer tries to add emoji to the assessment slug
DT-5585 There is 400 Error on Assessment preview if Assessment is empty/doesn’t have any valid question
DT-5605 [Internal] There is endless loader on the list of Assessments on My Assessments page
DT-5606 [Internal] There is 500 Error on adding Assessment from Expert library to My Assessments
DT-5607 [Internal] There is blank page when Employer adds saved question to the Assessment
DT-5613 [Internal] Applied filters or sorting are reset after page refresh or switching between pages
New Features
Assessment Settings
- All assessment settings including assessment, skill groups, invitation, deadlines and candidate communications have now been merged into the one `Settings` page.
- This update will make it easier for employers to use Vervoe by removing some of the noise in assessments. By default we recommend certain settings in our assessments. By placing the ability to edit these in the creation process we inadvertently make it more difficult for employers trying to navigate the platform. We also find that a lot of settings end up being incorrectly used which can impact candidates and ultimately the hiring decision for the employer.
- Removing the settings from each stage has also allowed us to remove the settings sidebar from each stage giving the user a larger workspace.
- As a part of this change we’ve also said goodbye to a breadcrumb structure, that helps users navigate the steps of an assessment and hello to our new chevrons. Designed to be bolder and indicate the stages of an assessment more clearly. We’ve also moved assessment insights into the flow so it’s easier for users to find and review.
- The 2 options for inviting a candidate (email and link) have been merged into the one page instead of the separate individual pages which streamlines the invite process.
A separate tab has been added to the invite stage which tracks all of the invited candidates through the different stages (invited, started, completed, etc)
Invite links now list the individual breakdown of applied started and completed candidates to help employers identify top candidate sources.
Assessment Slug
Employers can now edit the slug of an assessment for consistency.
Please note this feature is not available to AppSumos.There are several requirements for the slug to be valid:
The slug should be unique (across all assessments not just the customers account)
If they are using a name that comes back as taken they can append numbers to it eg. user-experience-designer-34
Min 2 characters Max 100
Only English letters+numbers
Should not contain special characters
Old links will not update to the new slug however candidates will still be able to use them
API
The API has been expanded to include an entity/package endpoint. This endpoint will allow our API users to invite multiple different brands to the one assessment.
For example: Seek would like to sell Vervoe to its customer base. Currently in order for the customers to have their own branding per assessment Seek would need to create an account per customer and write to the API for each account which is unscalable. This improvement means that Seek can use the same account but pass multiple different customers into the same assessment. Each customer will see their own branding and have their own completion caps. But from Seeks perspective it’s all managed within the one account which will also help with optimising AI grading for a set assessment.
- Additional company logos can be added via the account settings in Vervoe (see Account Settings – Multiple Brands update below)
- This will be added to our public API docs shortly
Account Settings – Multiple Brands
- Employers are now able to customize the branding per assessment in their account.
A new Branding Details section has been added to account settings where multiple logos can be uploaded and colors set.
Assessment branding can be managed for the assessment level within the new settings area. The drop down allows for the assessment branding to be updated easily within the workflow.
Improvements
Candidate Experience
Candidate experience has been updated to include more prominent employer branding including a company color matched header.
The profile page has been removed to stop the unnecessary collection of additional PII information outside of what is required to facilitate the assessment (name and email)
The “Run Tests” button is now hidden on code challenges if there are no example test cases to run to avoid confusion
Skill Groups – Migration
All assessments now must have a skill group. Old assessments that were missing skill groups have been updated to include one default group that is the same as the name of the assessment. This can be edited in the skill group settings
From now on employers will be unable to remove all of the skill groups in their assessment. The last one left will not be able to be deleted. It can however be updated.
Comms have gone out to customers notifying them of this change
Saved Questions
A flash message now appears when a question is saved to the bank to confirm the action has happened.
Assessment Preview
As part one of our assessment preview updates the employer will now see the candidate experience view of an assessment when they click preview instead of the long list of questions
Future updates will make the interactive questions previewable
A preview link now exists that can be copied and shared outside of the Vervoe app. This can be used for private library assessments only.
Email Deliverability
Employer and candidate emails have been split for sending now in an effort to increase email deliverability
Email Customization
“Hi [first name],” is now an editable part of the email body for Invitation/Reminder/Rejection Emails
Magic link email copy for candidates has been merged for all use cases
Google Docs
Employer’s can no longer edit file/google files from the locked question preview
Deadline
The time selector format for deadline by date is now in 12hr time with am and pm clearly marked
Internal
Updated React version
Removed Ziggeo duplicates
Ziggeo tokens are now unique
Added logs to admin panel when for when they change company plan and company settings
Segment
L-tt-preview-shared added for tracking assessment sharing
DT-5422 Added multiple segment events for Assessment/Company Branding
Registration Form Validation
Debugged employer registration form + updated verification email send
ATS Updates
Updated the way scores are sent to Greenhouse to prioritize newest scores first
Lever added to the plan description list on the Subscription/Account Details page
Assessment Insights
Candidate Completions chart default state has been updated to ‘Last 7 days’ instead of all time.
CS Sim
Employers can now view an interactive preview of the CS Simulator question type from their own assessment. The preview will allow them to move through the simulation without having to answer questions.
Test to Self
Test to self candidates will now be clearly labeled in Select and the Candidate Report Card
CMS (Internal)
When an Admin edits a Question and clicks ‘Settings’ the settings area will expose and focus.
Admin’s are now redirected back to the category folder page from the assessment page
Candidate Card Improvements
The status of an assessment (open, closed) can now be seen and edited from a Candidate Report Card on the All candidates page
DT-5210 FE – Design Fixes
Deprecated Features
Final Feedback Question has been removed from the assessment settings and can no longer be added to an assessment. This has been replaced with the CSAT modal.
Advance scoring setting/Text score has been removed for all customers now. Affected customers have been contacted and all text based scores have been reverted to percentages.
Ask candidates to complete the profile questions assessment setting has been removed along with the ability for a candidate to see these fields in their candidate dashboard.
Bug Fixes
DT-5461 Change redirect link after posting Assessment link on Facebook
DT-3856 FE – Only 1 space is shown in assessment name that has multiple spaces in a row that leads to confusing search results
DT-3877 FE – Number of candidates in filters isn’t updated after bulk actions applied until Select/Invite page refresh
DT-4722 New team members of existing companies with name/slug that contains “vervoe” cannot register
DT-4909 questionCount returns wrong value on Candidate’s dashboard in case there is a group with non-gradable questions only
DT-4917 FE – Selected category/skill in public library remains when the employer opens another page and back to the library page
DT-5246 FE – Fix colour of button for CS simulation preview window
DT-5260 “Assessment completed in” time is less than the real value
DT-5286 FE- Number of Candidates in Hired tab isn’t updated after hiring until page refresh
DT-5296 Checkboxes and radio buttons look stretched on Assessment Preview in Safari browser
DT-5301 There is no “required” label on preview of Assessment in Assessment library
DT-5338 There is no validation on numbers and special characters on reset password
DT-5357 It’s unable to invite candidate, 500 error was shown- if Company timezone is Europe/Kyiv
DT-5388 Audio and video player are not the same width as the block for choice on question modal in Compare mode
DT-5402 EEOC is always turned off for candidates after reapply
DT-5404 “Days Remaining” is displayed if the company have no subscription days on the Billing page from Account details
DT-5410 The left slots are not aligned with the right block on the Plan Types page
DT-5429 [VervoeAPI] Unnecessary field “premiumContent” in the response body of move folder request
DT-5441 [Admin] 500 error when setting unlimited completions without increment period in the company settings
DT-5447 There is wrong question details when Employer turns back after group with no questions or after deleted template
DT-5400 FE – Change the color of button “Talk to sales” on hover
New Features
Autograded Google Docs Questions [Internal Release]
- Please note: This first release is a preparation stage, where we start collecting the data for scoring and once it’s read, the docs will be AI scored
- Google Docs question type can now be AI-graded
- The changes a candidate makes to a document can now be automatically graded using the AI
- There’s a command that runs every 2 minutes, retrieving the text from a google document, and calculating the difference between the original document and candidate’s document.
- Presentations are not currently included in this
Duplicate All Assessment Settings
- This feature is currently only available on the Enterprize plan
- By default now when an assessment is duplicated all settings and custom email content will also carry across
- This includes the assessment deadline, randomization, and assessment settings like mandatory equipment check, and candidate profile settings
- Autoprogress settings are not copied
Assessment Started
- Candidates that have viewed questions within the assessment but have not started are now marked with a green tick on the invite page.
Improvements
Optimize (AI Health)
- Optimize has been removed as a stage within the assessment and will instead be accessible from the Select stage.
- Each assessment will have an AI health score of either low, average, medium or high depending on the status of their grading.
- The grading modal has been upgraded to include an estimated score range which should assist employers in grading answers more efficiently
- More data is being surfaced like the average team score which links to a list of all manual grades given to candidate responses for this question
- Employers can manually mark one of these manual grades as being inconsistent which can be reviewed periodically and removed if required
- A new question insights panel will show the employer which models have been activated per question
Assessment Library
- ‘All’ option has been added to the category dropdown for small screens (from 960px)
- Assessment Description/Summary field has been increased to 1000 characters
Invite Settings
- A ‘Saved’ indicator now appears when editing fields from the Invite Settings tab
Candidate Cards
- Redirect to 404 page if User opens Shared card of deleted Candidate
Disable candidate sharing when an assessment has been removed
CSV
- Added scores columns to exported hiring data CSV to include predicted overall score, predicted skill scores, actual overall score, and actual skill scores
- Updated CSV file to include all candidate statuses (Invited, Applied, In progress, Completed (Scored/AI scored/Calculating AI score), Expired, More time requested, Rejected, Hired) + date change
Segment Events
- Added segment events for tracking guest user actions on the free trial
- Updated invite TeamMember form (for stopping HubSpot from loading on invite pages)
API
- Send sms and update candidate’s number when “reapply: true” and “sendInvitationSms: true” are set for for the VervoeAPI
Admin
- Ability to delete a candidate account in the admin panel has been added
- Vervoe team can set a notification to be alerted when an enterprise account is close to reaching a candidate completion cap
Hired Flow
- Employer can now assign a future Hire Date to delay the sending of the post hire survey
Emails
- Emails about completion and daily digest won’t send to Integration Team Member’s as they’re not real accounts
Typing Test
- Preselected copy is added to the question introduction when the employer creates a Typing Test from scratch `Please complete the typing test to showcase your typing proficiency.`
Bug Fixes
- DT-5213 Wrong Question icon on assessment creation left sidebar
- DT-5207 Company wasn’t registered but User gets email with confirmation code
- DT-5206 There is no flash message about new Folder creation
- DT-5154 Integration Team Member is removed with other team members after upgrade of Free Trial company
- DT-5134 Horizontal video is out of border page on Apply to job page on mobile platform
- DT-5126 The “Extend” button from the 3dot menu and block with Shortlist, Hire, Reject buttons are active for inactive assessments on the All Candidates page
- DT-5118 Candidate list isn’t loaded on All-Candidates page for big companies – 504 error
- DT-5117 “Help” Zendesk button appears in iFrame after Candidate has opened Code challenge/ Google docs file
- DT-5091 Wrong number of questions displayed on email invitation letter if Skill Group Randomization is enabled
- DT-5033 Assessment preview tooltip looks bad sometimes
- DT-4915 Search term in private library remains when the employer opens public library page
- DT-4909 questionCount returns wrong value on Candidate’s dashboard
- DT-4881 There is wrong text in flash message when Employer deletes Assessment Folder
- DT-4880 There is empty text field on editing Folder name if previous editing was canceled
- DT-4798 “Question completed in” time is bigger than the real value
- DT-4354 Issues with rich text editor
Improvements
Autograded Spreadsheets
- Customers can now manually override an autograded spreadsheet to mark an answer as correct or incorrect
API Updates
- New endpoint added to API for extending candidate deadline by candidateAssessmentUuid
- New endpoint added to API for recalling the candidate URL magic link by candidateAssessmentUuid
Public Library
- Added the ability to reorder categories for the public library in the database
Social Invites
- Changed the redirect on a Facebook share link to the Assessment Invite page after posting instead of the now nonexistent job page
Bug Fixes
- Fixed the 400 error on the AU instance when a Candidate uploads a certain PDF file as a question answer
- Candidates receiving multiple reminder emails had been resolved
New Features
Typing Test
- Our new typing test question type lets customers measure candidates typing skills, speed and accuracy.
- It can be added to an assessment from within the “Create” editor. The test comes with the preconfigured copy for both English and Spanish.
- The test is locked at 2 minutes to complete and once the candidate begins their first keystroke, the 2-minute timer will begin.
- As the candidate types, the incorrect text will highlight in red and candidates will get two attempts.
Candidate Share Cards
- Candidate share cards have been revamped to allow sharing and collaborating on shortlisted candidates even more seamless, whether or not stakeholders have Vervoe access.
- Candidate cards are shareable report cards that let you deep-dive into a candidate’s skills, attributes, and responses.
- Customers can share one or multiple assessments the candidate has taken, and one or more candidates to compare.
- Customers can share results while hiding the candidates names to reduce bias. This update also includes the candidate integration card for employers using Vervoe via an ATS.
- This update also includes the candidate integration card for employers using Vervoe via an ATS.
Close Assessment By Date
- Customers can now choose to set a date to close an assessment by
- By selecting a date, the number of days feature will automatically disable, which means all candidates will have until the date to complete the assessment regardless of the time they were invited.
- Customers can still choose to set a time limit within which the assessment must be completed for each candidate
- When the assessment deadline is reached the job links will automatically disable and new candidates will not be able to be invited to the assessment
- Candidates in the middle of completing the assessment will only have the ability to request more time
Improvements
Internal CMS
- ‘Suggested’ questions and buttons are being added back into the create stage of assessments
- Suggested content will now pull from already published assessments and new questions published via the CMS
- ‘duplicate detection’ feature will now run in the CMS question library to flag multiples of the same content
- “Instructions to Candidate” field won’t copy from a private assessment to the CMS
Account Settings
- Company timezone is now based on the country the customer registers from by default instead of UTC
Segment Events
- Segment events for tracking auto-progress/auto-reject features have been added
- `TT-autoreject-enabled`
- `TT-autoprogress-enabled`
- `TT-autoprogress-link-added`
- `TT-autoprogress-link-removed`
Bug Fixes
Candidate Statistics
- Deleting a candidate will reset the statistic counts
- questionCount was returning the wrong value on the Candidate’s dashboard due to non gradable questions
- Investigating why “Question completed in” time is bigger than the real value in some instances
Email Copy
- Email invitation for the linked assessment in Auto-progress stage now directs to the invite of the second assessment
Account Settings
- DT-3718 Google Maps don’t work when selecting location on Employer’s side
Job Links
- DT-1007 Sharing job link via Facebook doesn’t work
New Features
Anti-Cheating Plagiarism
- Plagiarism detection lets you track candidates that have the same answer by comparing all candidate answers within a question across the platform
- If 2 or more answers are the same the feature will flag the candidate in Select
- Only Text answers are included in this feature
- The feature can be turned on in the question settings of Create however by default it is off
- eg. You may wish to keep the feature off if you know the text response will be the same for each candidate like asking someone to correct the spelling or grammar of a question
- Plagiarism is available on all plan types
Improvements
Daily Email Updates
- A list of extension requests has been added to the daily digest email
API
- Add the ability to tag and untag candidates via the API.
- This was a request from Walmart who wish to review candidates in Vervoe based on their location. Currently, when candidates apply they add their location in TalkPush. This change will enable that input to be passed as a tag from TalkPush to Vervoe.
CMS Internal
- Admins can now publish an assessment without adding it to a category. This was required to support the switch over of old assessment content into the new library
Assessment Library
- Learning and Development category has been added to the hardcoded assessments at the top of the public library
- Adaptability, Coding, Attention to Detail, Critical Thinking and Empathy have been added to the list of popular skills
Candidate Emails
- The words interview and assessment have been removed from candidates’ emails
Segment
- Segment event ‘A-user-disabled/enabled’ added on BE for tracking and managing users’ accounts from the Admin panel
Internal Admin Updates
- The company owner can now reassign ownership of an account
- Admin panel now has the functionality to mark candidates as hired and add hiring data + via uploading CSV file
- History of activating/deactivating private assessments
- Transfer candidate feedback data from Wootric to Vervoe data warehouse (CSAT)
Hubspot Invite
- Stop HubSpot from Loading on invite pages
Bug Fixes
CS Simulation
- The scroll was missing in the Customer details block in the CS simulation question
CSAT
- CSAT flow now starts to redirect after the candidate submits or skips the feedback option for PageUP/API candidates
- CSAT modal is now not shown for API candidates invited with parameter enableFeedback:false
- CSAT feedback can now be exported via CSV from the feedback modal
Statistics
- “Question completed in” time is bigger than the real value for some candidates
Segment
- Wrong ‘company_planToHire’ property was sending to segment
Quick Invite/Invite
- Modal about successful invitation now closes after Quick invite
- Country code for phone field has been added on Quick invite modal
- The data entered in the field “Full Name “, “Email” resets when switching from the email invitation page to another
CMS Internal
- Questions in the set are now shown in the new order after reordering
- Admin can’t publish invalid questions
- Unsynchronized assessment status shows immediately after changing randomization settings
- Question state and set synchronization status update immediately after the question is removed from the set
- Unsynchronized status shows immediately after questions/skills reordering
Assessment Workflow
- “Hire” button is disabled if Integration team members don’t have permission to hire a Candidate
- A limit of 5,000 characters has been enforced in the Score instruction field when editing a question
- Сandidate’s with long names are now truncated on the Report card
- Google Maps now works when selecting the location on the Employer’s side
- Facebook job links now work on the invite tab
- Align headers through the employer’s app
Admin
- Error that stopped Admins from creating new assessment category has been fixed
New Features
New Starter Plus Plan
- The new plan will act as a more proportionate upgrade incentive for Starter customers
- Subscription:
- Yearly: US $2148/year ($179/month)
- Monthly US $229/month
- Assessment Slots: 3
- Users: 4
- SMS: 150/month
- Can be purchased directly from the pricing page or from within the app on the subscription page
Enterprise Completion Caps
- Some Enterprise customers have completion caps per assessment this change will now enable them to see in app when they are nearing the end of their quota
Improvements
Internal CMS
- CS Simulator can now be managed in admin by the product team making it easier to make amendments or create new content for it.
- Questions, Sets, and Assessments can now be published/unpublished and Synchronized to the public library from the CMS
- All questions from the Airtable and create builder have now be merged into the CMS
CS Simulator
- Vervoe content admins can now add notes/skills to the tickets in the CS Simulator meaning the report card will now reflect a skill breakdown per question in the simulator
- Vervoe admins can now create/edit/remove/duplicate/reorder knowledgebase articles and tickets for CS Simulator questions in the CMS account
Yearly Only Pro Plan
- Professional Monthly subscription payment options have been grandfathered no new users will be able to select this payment option
- The Professional plan can only be purchased yearly at an increased cost of US $8,000
Segment Events
- ‘A-account-confirmed-BE’ segment event is now triggered for multi-type users
- Added a property to measure the total days on a plan type (totalDaysOnPlan) to the ‘A-plan-upgrade’ segment event
Bug Fixes
SMS Invite
- Blank screen and console error when Employer with no SMS feature (Free Trial and AppSumo) enters candidate’s name and email in Quick Invite
Assessment Library
- DT-4780 Search in Public Library doesn’t show the assessment if search term doesn’t contain the suffix at the end of the assessment’s name
- The assessment page layout is partly broken sometimes after the Employer’s login
- DT-4760 [Internal] 500 error when Employer tried to download file with name that contains non ASCII characters from preview of public assessment
New Features
Improvements
Account Settings
- Truncate long names/emails with 3 dots + add a tooltip on hover with full name/email on the account Team page
- In order to reduce the visual noise in a users account we’ve removed the following banners from PLG customers and the Free cohort. Notifications around completion limits still remain
- The remaining banners mentioned above are now dismissable by clicking the X at the top right
My Assessments
- Updated assessment cards makes it easier to see the progress and vital stats of the assessment without clicking into it. View open/ closed statuses and quick invite/ completion stats at a glance with assessment cards enhancements on the Vervoe’s home screen.
Appcues
- The Appcues notification icon in app has changed from the heart icon to a bell
Hired
- The “Would you like to record the hired info?” is now hidden from the Candidate Report card if the hiring date or manager have already been set
- Previously if a candidate has been marked as hired in 2 assessments the total number of hires incorrectly counts this candidate twice. This has been updated so the same candidate will only be counted once.
Segment
- TT-candidates-filtered has been updated to include the filters for completed, hired, NEW invited and applied
- ‘TT-candidates-batch-actioned’ and `G-message-sent’ updated to include a content property that will capture the message sent to a candidate
Assessment Insights
- The Question by Question Insights table has been updated to include all started candidates (not only completed)
- The information tooltip for the Question by Question table has been updated to make it clearer that the count includes candidates that have not completed the assessment. `Question insights are calculated even if assessment is unfinished`
Filters
- Once a user has applied a filter to Select/Shortlist/Invite/All candidates/Hired pages it will now remain with page refresh or switches pages. For example filtering by skill group on Select and then viewing a candidate report card and closing it will place the user back in the skill group filtered select view instead of resetting.
CSAT
- Note: The following relates to new CSAT data. The old data will be migrated from Wootric to the data warehouse in a future release.
- The candidate CSAT survey is now integrated as a part of the Vervoe assessment. This allows us to show the CSAT score as a part of assessment insights for all customers including Australia Post. Previously this was not possible.
- All candidates who complete and submit an assessment are now able to see the CSAT survey
- The final feedback question that is currently enabled for all customers by default is now disabled by default so that candidates are not asked to complete feedback twice
- Customers on AF are now able to see the CSAT score from the assessment Insights page
- Customers can also access the candidate comments for the CSAT survey by clicking on the grey number next to the CSAT score, and the score breakdown and comments will appear.
Statistics, Filters, and Invite
- Invite statistic block has had a refresh and is universal across both invite and select.
- Invited, applied, completed and hired filters have been added to the Invite page
- The tag filters from Invite and Select/Shortlist contain only the tag options that are used in the assessment
Internal
- CS Sim has been added to the hero images in the public library
- Customers’ name is now sent to Stripe by default with subscription creation
- The history of accounts changing plans has now been saved to the database and made available in the admin panel (similar to the candidate statuses) and include:
- Event date (date of changing plan)
- New plan name
- Plan ID
Candidate Experience
- Replaced “interview” with “assessment” in the Candidate app. Click here to see all updates
CSV Export
- Moved the ‘Tags’ column to last place in exported CSV file from Select + Invite to assist Walmart with their reporting
SMS
- Added prefilled country code and validation of candidate phone number
- SMS Quota count now counts down and not up to avoid confusion
Bug Fixes
Reapply
- Candidates that reapply were showing the incorrect rank in Select
- The count of “All Candidates” filter incorrectly increases after a candidate reapplies via the Vervoe API
- Wrong value in tag filter on Invite and Select pages after a candidate reapplies has been fixed
Optimize
- Blank page and console error when a customer grades the first answer in status “Ready to grade” has been resolved
Hired
- An empty state now appears if there are no candidates in applied filters on the Hired page
- Error message now appears for search results that have the wrong name/email on Hired page
- “Would you like to record the hired information for this candidate?” block was shown for non-hired candidates
- Count of all candidates now won’t change when customers apply filters or search on Hired pages
- Unhired candidates will now return to their last status before hired eg “Completed”/ “Scored”/ “AI scored”
Account Settings
- Long name of custom assessment application link makes horizontal scroll on Invite -> Candidates list
- New team members of existing companies with name/slug that contains “vervoe” can now register
- Notification emails weren’t sent to all employers (candidate’s activity and daily digest)
- Company size of 1 – 10 can now be selected from the account settings page
Statistics
- Wrong values of “in progress” candidates were showing on the statistics block on Invite/Select pages
Create
- Blank spaces can no longer be saved in lieu of a correct answer sample
Improvements
Candidate Email Updates
- We’ve ensured the user experience of the email editor is consistent everywhere it appears.
- You may notice the layout of the actual emails (Invitations & Reminders) are cleaner and we’ve also taken the opportunity to update the default candidate email invitation and reminder copy and subject lines to an even warmer and more personable tone. 🤗
- Most importantly, you can now edit the copy and subject lines for any of these emails and personalize your candidate experience.
- You can now set the from email address to be a no-reply email address (or keep it as your own email address as it was previously)
- This builds on the improvements we released last month to support rich text editing and paragraph styling to create a more unified email experience.
Invite
- Toggle assessment reminder emails on and off altogether to suit your candidate experience.
- SMS assessment invitations are available for Enterprise customers to purchase. Pricing here.
My Assessments
- Updated default order of `open` and `closed` assessments so `open` assessments show first.
- Users can still filter between open and closed assessments or search universally across both.
Segment
- Added paused/cancel status to segment group call and employer identify
- companyCancelled (true/false) → cancelled status (
true
if the company subscription has been cancelled) - companyPaused (true/false) → paused status (
true
if the company subscription is paused at the moment of the event)
Improvements
Code Challenges
- We’re making it much easier for our users to create code challenge questions by providing the ability to generate boilerplate code for all supported programming languages with a couple of clicks.
- A new section has been added to the edit functionality of the question enabling users to generate boilerplate code and create test cases.
- Test cases can be used to autograde a question or as an example answer
- GTM docs here
Hired
- Unhire flow added to the Candidate Report Card
- Unhired added to the 3 dot menu on the full Select candidate list
My Assessments
- Updated design of the Private assessment library for merging Active/Inactive assessment list
- Active/inactive assessments are now in one list with `open` and `closed` labels indicating the status of the assessment
- Users can now filter between open and closed assessments or search universally across both.
Invite
- Users can now format the text on the candidate welcome screen by using a rich text editor in invite
Select
- Trophies will only be awarded to candidates that score over X% per specific skill
- Candidate tags will now be exported as part of the CSV download
- API enabled customers on AF will now see a percentage score per skill on cards
Internal Admin
- The history of team invitations will be tracked in the admin panel including (email + name, invited by team email, invited to role and date of event)
- The history of changing employers email addresses will be captured in the admin panel
Revenue Tracking
- Google Analytics clientID added to all backend events tracking revenue and activation
- Added support for facebook-conversions-api-destination
PLG
- The Vervoe invoice template will be replaced with the Stripe invoice so it’s clear what the cost is and how it’s being calculated
Candidate Experience
- Hiden the Zendesk “Help” icon so it doesn’t block part of the Google doc file when Candidate opens
- [Ziggeo] Removed the second button for video/audio recording in the candidate app
Bug Fixes
- Fixed typos on SSO on sign-in fail modal
- Missing categories in “Refine by Categories” on Skills page have been fixed
- Social share button for Facebook job link now works
New Feature
Self Service Pausing
- Assessment flow customers on `Starter` and `Professional` plans can now pause and unpause their subscription automatically from the billing page of their account settings.
- Accounts and payments can be paused twice per year for up to a total of 6 months (up to 3 months per pause).
- Customers can unpause their subscription at any stage. After 3 months the account will automatically reactivate.
- Paused accounts will limit access so customers can only access the account settings page to reactive their subscription
- The pause option will also be available via the cancellation flow
- Customers that choose to pause will receive email notifications confirming the pause and then 7 days before the pause is due to end.
- Segment events added to track pause/unpause subscription
- Find out more here
SSO
- Customers with SSO activated for their account can sign in with SSO from the login page
- Customers without SSO that click the sign in via SSO button will be redirected to zendesk to enquire about the feature with a sales team member.
- New setting enables/disable the ability to create new team members via SSO
- Find out more here
Improvements
Team Management
- Assign company owner as the owner of assessment if its previous owner has been deleted
Segment
- Added a new property to ‘A-plan-upgrade’ segment event to track the number of days a user spends on a plan type
Bug Fixes
Invite
- Some candidate’s being excluded from the `Source` filter on the Invite page if they’ve applied to the assessment in one company and then been invited by email in other
Improvements
Segment
- Added a new property ‘company_candidateStartedLast7days/30days’ to the employer identify call to track the number of candidates that start an assessment on the Employer side
- Deprecated segment event ‘A-account-confirmed’ (FE) as there’s now a BE version
- Segment event L-search-conducted won’t fire on opening/reloading libraries pages
- Renamed ‘amount’ to ‘value’ properties from ‘A-stripe-payment-succeeded’ segment event
- Update ‘A-plan-upgrade’ segment event to include the total number of days on the plan
API
Improved Public API documentation
Hired
- Users can now unhire candidates from the Hired tab of all candidates, the reason will also be collected.
- Comment field for ‘other’ option will only show if `other` is selected.
- Text updated on other comments field to `Please provide a reason for selecting other`
- Today’s date will be pre-selected when the hire date picker is opened the first time
- Buttons + Vervoe logo should be the company brand color on the post-hire survey page
- Updated the ‘HIRED CANDIDATES’ statistic counting approach from the ‘Hired’ page
- Updated text to add an ‘s to the candidate’s name on the post-hire email and page
- Add tooltip for long names on What date was the candidate hired?
- Comment free text field will expand on post-hired survey page to accommodate long responses
- Added endpoint for hiring candidates via Public API
- Added unhire feature to the Public API
- Added the number of candidates to filter options from ‘All candidates’ page
Assessment Insights
- Optimize Score Distribution request for one skill – Walmart example
- Update names of CSV files (completions/distribution) from Insights
Deactivated Accounts
- Hid ‘Create Assessment’ button from the header for deactivated/expired accounts so they can’t access assessments
Inactive Assessments
- Hid Predicted score -> “Calculating AI score” section for candidates on inactive Assessments
- Confirmation modal disappears when Employer deactivates assessment from the Assessment list
Select
- [Internal] Data in filters will be refreshed without page refresh
- [Internal] Style of amount of candidate doesn’t match if there are no candidates in some positions on filter
Bug Fixes
Hire
- [Internal] If skipping the hired date no date will be added
- [Internal] Blank screen and console error when unhiring the only hired candidate from selected in filter assessment or hiring manager
- [Internal] countHiredCandidates value isn’t decreased after unhiring candidates
- [Internal] 500 Error on unhiring candidates with empty status history
- There is hiring survey modal on deactivating assessment with all rejected completed candidates
Select
- countCompletedNotRejectedCandidates value doesn’t change when Employer rejects Candidate
- BE – Field “skipped” is always false in question-results despite question was skipped
- FE – No overall AI score is shown on big candidate card for rejected candidate
Migration
- Some candidates weren’t migrated from ATS to Assessment flow
Free Trial Expiration
- Some old Employers cannot login if their companies are on Expired Free Trial now
ATS Flow
- [ATS flow] Stage disappears from Candidate’s card after rejecting
Onboarding
- Wrong link to Invite page in Setup Guide
Tagging
- DT-3980 FE – Fix hover on 3-dot menu from tag on Company Account page
Advanced Scoring
- Advanced Scoring: Text score from one group hides when another is added to the other group
New Feature
Hired – Closing the loop
- You can now track the progress of hired candidates in Vervoe. Our closing the loop survey is sent directly to hiring managers 3 months after their new employee starts.
- The survey asks 2 simple questions around the quality of the hire
- The data we capture will be used in company reporting to show time to hire, quality of hire and basic reporting like ROI
- The flow is activated upon a candidate being marked as hired. You will be asked to select or add the name of the hiring manager along with their email and the date the candidate was hired.
- You can click on the new `hired` tab in the top navigation bar at any time to review, change or add these details to candidates marked hired across all assessments.
- The `hired` link in the navigation bar leads to the hired tab in the redesigned all candidates
- Post hire survey will be viewable from the hired tab 120 days after the hired date.
- GTM docs are here
Assessment Insights
- Customers can now review assessment level insights including completions over time, AI score distribution and question by question drop off
- Insights are accessible from any stage of an assessment via the insights icon
- Data from each chart can be downloaded via a CSV file
- Assessment insights GTM docs are here
Feature updates
Integrations
- Updates have been made so that expired or inactive company accounts with integrations do not work. This includes PUP, Greenhouse, API, and Zapier
Assessment Updates
- If you change the question type eg from MC to text the related entities and drafts won’t be deleted meaning if you change the question type back you’ll still see all of the MC options.
Segment
- Rename ‘amount’ to ‘value’ properties from ‘A-stripe-payment-succeeded’ segment event
Bug fixes
Job Flow
- Migration: Candidate, invited to an offline stage wasn’t displayed on All Candidates page
Candidate Experience
- Language settings: Candidate’s saw a blank screen and console error on submit if Google translation was enabled
API/Integrations
- invitationReceived status is duplicated for candidates who were invited via API
New Features
- [Requires approval from JobAdder before being made public to customers ~3 weeks wait time] JobAdder integration added to our growing suite of ATS integrations. Available to all paid plan users and activated instantly via the /integrations page of the account settings. Find out more in the GTM docs here
- Assessment library has been completely redesigned making it easier to connect with relevant content. Search has also been updated so that it’s more robust and occurs on a global level so regardless of any category you may be in you’ll always be searching the entire catalog. Find out more in the GTM docs here
Improvements
Assessment Flow Improvements
- Updated default green color on all buttons/icons in assessment in the employers app inline with our new design color palette
- Hired is now accessible from the candidate report card without needing to click on the extended menu making it more visible to users
- Updated functionality for ‘sort’ drop down in Select mode. Now if only a few questions in a skill group are select the skill group will default to open to make it easy for users to see what has been selected
- Small assessment card statistics have been updated to reflect the same numbers as inside the Select stage. Candidates that reapply are no longer counted on the assessment card.
- ‘Create Assessment’ button has been lifted up to the top navigation bar when in the public library
Permissions
- Team viewers can no longer see the edit button for google docs when reviewing an assessment via the create stage or comparing candidates in Select.
Segment Events
- Updated ‘L-search-conducted’ event to include skill, category or assessment attributes
API
- Endpoint added for removing assessments from folder through public API so that folders in UI match folders in integration
Internal Updates
- Migrated Vervoe API Symfony framework to 4.4 version from unsupported version 3.
- Switched out segment tracking for TEST and DEV environments
- Improve bulk actioning – linked to #QA channel request from Brooke here
Candidate App
- Custom email button text now matches with the button text from Terms and Privacy Policy tip from invitation candidate email
Bug Fixes
Candidate App
- DT-3756 Candidate saw blank screen and console error on submit
- DT-3578 Assessment timer error related to Daylight Saving fixed
Assessment Flow
- DT-3755 Request posts return 404 error on integration card if assessment slug != opportunitySlug
- DT-3749 ‘Close Report card’ button not closing the candidate tab in select has been fixed
- DT-3740 [Internal] Fixed the 500 Error on Active learning in Assessment
- DT-3730 Wrong front end page is passed to the back end when searching in “My assessments” if the previous request hasn’t completed
- DT-2325 [Production] Skills repeat on AB input modal
- DT-3773 [Internal] Skills list looks broken on some assessment cards
- DT-3772 [Internal] There is 400 Error when Employer chooses skill “Time management”
ATS Flow
- DT-3728 [Internal] Assessment card is broken in Private Assessment on ATS Flow
Segment Events
- DT-3703 Investigate G-candidate-hired event → AF/ATS companies mixed up
New Features
Folders
- A new folder system has been added to the top of the My Assessments page to allow customers to organise or group their assessments
- This feature is only available to AF customers
- Folders can hold an infinite number of assessments and their can be an infinite number of folders
- Segment events have been added to track ‘folders’ actions
- Folders do not contain permissions and can be used by any one in an account
- GTM documentation here
- Help article here
Migration
- After migration customers on AF should be able to delete an assessment that had been used on ATS
New Sign In
- Sign Up/Sign In redesigned to include panel for social proof quotes and logos
- Additional duplicate page created for free trials to test messaging
Appcues
- Launchpad widget added
Improvements
Internal
- Command added to update candidate completion limits by company id
- Free Trial accounts are deactivated 3 months after sign up
- Removed old ziggeo recordings of inactive/canceled accounts
- Improved validation of question and answer types
- Re-imported questions to Public Library from Airtable that were deleted during duplicates removal
- Added the ability to block the company by IP address (registration/login/access to all pages)
- Updated brand colour from shared agency/integration cards
- Do not show “Calculating AI score” animation for Candidate who completed the assessment with only info questions
- Replace S to Z in optimizing names states
- Don’t show ‘Calculating AI score’ for rejected candidates from ‘Select/All candidates’ list + Candidate Report card
- Added ‘Saved’ indicator on Auto Reject/Progress requests from Assessment Settings tab
- Fixed Comeet icon
Integrations
- Customers can create several Zaps for the same event on the same assessment
- Add endpoint for rejecting candidates with public API + add ‘rejected’ status
- Added information about assessment folder to ‘Get list of assessments’ endpoint for API
- Added list of skills in assessment to ‘Get list of assessments’ endpoint for API
- Added validation on length of candidate’s name and email when inviting candidate via email, Public API, integrations or when candidate applies via link
- Added segment events when the customer changes company account settings
Candidate Experience
- Change text in Magic link email for Candidate that has no deadline in Assessment
- Updated colour of “confirm email” button from Change Email (To New Email)
- Updated “Vervoe” logo in the application and on the candidate page
Select
- Added ‘Hired’ status to exported CSV file
- Don’t show in status “Scored” for Rejected Candidates
- Improved left side nav panel from ‘Create’ and ‘Invite’ pages
- Candidates open tabs are saved after the page refresh
- Don’t delete related entities and drafts when changing question type. Don’t copy them to other places
- Added info about timer and required question on question preview on Compare modal
- Added a link to Invite Settings to rejection email modal (when rejection email is turned off)
- Updated order of candidates cards in Compare mode
- Hired badge added to the select list
- Applied filters from Shortlist are visible
Bug Fixes
Select
- Fixed text for Group names in Assessment editor
- Candidate’s are no longer greyed out after being moved to offline stage and then applied
- Scroll added for Candidate answer (cod editor/challenge type of question) on Candidate card
- After deleting group with questions in Skill Groups, all questions are moved to the previous group, but after page refresh they are shown in the first group – fixed
- Syntax error fixed on import CSV in case an accented character is right before comma
- Invite, Select, Grading tabs can no longer be opened by URL after assessment was deactivated
- Skill groups have been rearranged on Candidate’s card
- Question content now appears immediately after generating assessment
Events
- The wrong source being sent in the candidates segment event has been updated
Account
- [task reopened] Customers can now open unlocked candidate’s card immediately after plan upgrade
- Decimal price on “Order Summary” modal has been fixed while upgrading plan
API
- Candidate’s score and completion time are included to statistic values despite candidate re-applying via Vervoe API has been updated
Permissions
- Team Member sees on All-Candidates page the Assessment/Job name where they have no access to
Candidate Experience
- Fixed text for email with magic link when Candidate has expired status
- Incorrect error message in Candidate’s profile when they are trying to upload the file with not-supported format has been updated
Question Types
- Blank page and console error when customer selects the last test case and clicks on delete icon has been fixed
Migration
- Job names are not displayed on public/agency Candidate cards on Assessment flow
New Features
The new Vervoe product video is here!
Invite
- Customers will now be able to see a new Invite UI that includes invite via link, email, csv upload, candidate experience settings and an overview of invitations and completions.
- Candidates can be filtered by email and link
- Deadline extension requests are easily seen and managed
- Tags can be added to candidates
- Candidates can be rejected from here
- ‘Extend’ renamed ‘Re-Invite’ from three dot menu from Invite page for candidates that have never opened assessment invitation
- Help article here
Optimize
- The design of the ‘Optimize’ UI has been updated to match the new stages
- Help article here
Select
- Customers will see a completely redesigned results page with rank, score, badges for top performers and trophies for skills
- Candidate report cards and small hover cards have been redesigned to show score per skill break down and question by question responses along with any tags or comments from team members
- Customers can sort candidates by knocking out skills or questions and watching the scores recalculate
- Customers can filter candidates based on tags
- Customers can view multiple candidate cards via a tab system within the select screen
- Candidates can be added to a shortlist
- Candidates can be compared question by question from the shortlist
- Compared candidates or individual candidate cards can be quickly shared with stakeholders
- Question Groups have now been renamed Skill Groups
- Sort is disabled when question randomization + limit/limit is activated for the whole assessment and for assessments with text score
- Help article here
Hire
- Hire flow is accessible via the assessment card when an assessment is deactivated and from within an assessment.
- ‘Hire’ can be selected from the three-dot menu from select overview and shortlist pages along with the candidate report card.
- Hired’ status added to Vervoe API, Zapier and Segment events
- ‘Hired’ filter added to ‘All candidates’ page
- Help article here
Improvements
Internal Admin Updates
- Updating the admin to include a section for published assessments
- Creating the ability to deactivate assessments from Admin
API Improvements
- Added list of skills in assessment to ‘Get list of assessments’ endpoint for API
- Hidden link to “Company account” for Team members on Tag modal
Bug Fixes
Improvements
MC Question Type update
- Multiple Choice question type has been updated and now accepts media
- Media files can be added to answer variant fields of Multiple Choice questions type including code, audio, video, file upload, and images
- Updated design of icons for media in question field
- Update “Multiple Choice” question type can be exported from Airtable
- Customers will see a notification warning them that old MC questions will be made “do not grade”
- Customers can see updated MC candidate answer from the grading card
- Customers can see media files in the answer variant fields of MC question from the grading card
- Existing MC-multiple options (graded) questions should not be sent to suggested questions in TT builder
- Mark existing MC-multiple options (graded) questions invalid in Airtable
Assessment Improvements
- ‘Team Scoring Instructions’ are now displayed on grading card for autograded question types (code challenges, MC, Numbers and spreadsheets)
- Files can now be downloaded from create, grading or the preview
- Use the same rules for “Examples of a correct answer” on grading modal as we have on Create tab
- Set recordingTimeLimit:3 for all questions that have recordingTimeLimit:0 in Public Library service and add validation on import questions into Public Library
Migration Improvements
- Candidate emails sent from ATS flow will cease when an account is moved to AF
Plan Type Improvements
- Don’t send candidate assessment reports that are over limit (locked candidates) to PageUp, Zapier, Greenhouse
- Change upgrade policy by adding fixed number of completion spaces while upgrading
- Don’t send candidate assessment reports that are over limit (locked candidates) to SR, Comeet or the API
Event Improvements
- Skill group names and status added to Segment events
- HubSpot event tracking added
- Mixpanel event tracking added
Bug fixes
File uploads
- Employer sees blank question page after uploading too large file has been fixed with notification message
- Question text disappears while Employer is uploading the file to the question has been fixed with loading state
Plan Fixes
- Incorrect limit number is shown on banner “You’ve almost reached your limit”
- DT-2966 Issues with upgrading subscription via Stripe
Assessment Fixes (Internal not on Production)
- Make text of placeholders 12px for Correct answers in Spreadsheet auto-graded questions
- Score isn’t shown as 0 for existing choices when changing answer type from Multiple Choice to Single Choice
- Wrong assessments are shown when clicking on some category in Assessment Library for the first time
- Value in Search field doesn’t clear if Employer switches from Home page to Categories
- Question text disappears while Employer is uploading the file to the question
- Employer sees blank question page after uploading too large file
- User can change code of answer example on the Grading modal
- User adds code to the answer example input field changes the size
- Text of placeholder is too large for field with points
- There is no error notification if no answer is marked as correct in Multiple-choice option question
- There is empty space in grid with question type icons
Permissions
- Team Viewer receivers 403 error if opens assessment with no invited candidates
Zapier (Deployed but not live to customers yet)
- 500 error on start interview when Zap “Assessment started” is turned on
- start_interview Zapier is sent on every call of start endpoint, even if Canddiate resumes the interview
ATS Flow
- Employer on ATS flow cannot search Candidate inside the job using capital letters
Improvements
Internal Improvements
- Admin should be able to move assessment from one company to another
- Add hash for correct answer examples in questions
- Template UUID is generated while assessment is published from Admin
- Duplicated Assessments will carry default settings for Hide Complete later button (unchecked) and, candidate deadline (4 days) if copied from Admin
- ‘Team Scoring Instructions’ are now displayed on grading card for autograded question types (code challenges, MC, Numbers and spreadsheets)
Plan Changes
- Candidate completion limit of 300 per assessment added to Professional plan
- Candidate completion cap changed from 50 to 30 completions on Free Trial
- Unlimited user seats added to Free Trial
- Segment events about limit on Free Trial confirmed
- Segment events CE-completion-limit-almost-reached and CE-completion-limit-reached updated to reflect completion limits for Professional plan
- Customers with AF on Professional plan will see warnings and banners about candidate completion limit
- Update ‘Upgrade to unlock’ modal from the small locked candidate card for Starter plan
- Update candidate limit information on Subscription page Professional plan tab
- Customers on Free Trial with unlimited user seats will see a notification to ‘Invite a new team member’ via a modal with information about the temporary team seat
- Assessment banners on the Results page for new Free Trial limit
Cancel Flow
- Zendesk should be opened on click of “Contact support” from “Other” option from ‘Cancel my account’ modal
SmartRecruiters
- ‘Complete Assessment order’ call into SR integration configured to finalise integration
OKRs
- Link website and employers app
- Assessment name added to event ‘A-account-created’ when the user comes from the website from specific assessment
- Simplify the onboarding flow from 3 steps to 1
Skill Groups
- Information about skill groups added to segment events to properly track their use
Internal Events and Reporting
- Add segment event ‘A-stripe-payment-failed’ to automate in app dunnings series
- Country name should be full in company_country property instead of the abbreviation eg IN will become India
- Changing the ID of form elements to help with Hubspot reporting
- Group call added to Segment includes all of the ID properties
Backend Improvements for Assessments
- Customers with enabled API feature can disable/enable candidate deadline for submission
- Group Randomization settings and Advanced Scoring settings are copied when Employer publishes assessment
- Assessment deadline won’t copy when customer publishes an Assessment into the library
- Copy template UUID while creating audition with method of autofilling from TT builder
- Save data from the public library about audition skill title (audition name, slug, skill and number of times used)
- Save data about adding, removing and usage of questions in assessments from the public library
- Don’t send Zapier events for inactive assessment
Bugs Squashed
Number Question Type
- Candidate’s answer on Number question type is exported to csv file
- Candidate’s answer and correct answer for Number question type are shown on shared cards (public, agency)
- Some invalid values in correctNumber field weren’t considered as invalid and have been updated
- “Answered” checkmark now disappears from Number question after candidates deleted their answer
Questions
- Duplicated entries removed from Production Airtable
- Question has incorrect status “Answers graded”, despite there is an empty answer to grade
- Google file for Google doc questions and test cases for Spreadsheet auto-graded questions are not visible in copied assessment if copied from admin
Tagging
- Fix deleting companies and candidates if some candidate had a tag
AI
- Some AI scores on Production aren’t updated because of different rounding in MySQL and php
- Old AI score is no longer shown for candidate who complete an interview for the 2nd time (after extending deadline)
- Grade Answer Error in PUP updated
iFrame
- Header sometimes moved to the middle of the page for several seconds when question downloads
Permissions
- Team editor can’t edit auto-progress setting if he has only 1 active available assessment and it is auto-graded has been fixed
- Wrong permission for a former team member who was invited again has been fixed
PLG Flows
- Customer upgrading from Starter to Professional was incorrectly given ATS flow after updating subscription via Stripe
- Fix redirection after sign up from pricing page so user lands on subscription page
- After registration customer was landing on My Assessments page instead of Expert Library page
- Language updated to “Upgrade” instead of “Talk to us” for Professional Plan
- Banner “candidate completion limit has almost been reached” isn’t shown for Employer on Professional plan on Results page
Improvements
Internal Improvements
- Admin should be able to move assessment from one company to another
- Add hash for correct answer examples in questions
- Template UUID is generated while assessment is published from Admin
- Duplicated Assessments will carry default settings for Hide Complete later button (unchecked) and, candidate deadline (4 days) if copied from Admin
- ‘Team Scoring Instructions’ are now displayed on grading card for autograded question types (code challenges, MC, Numbers and spreadsheets)
Plan Changes
- Candidate completion limit of 300 per assessment added to Professional plan
- Candidate completion cap changed from 50 to 30 completions on Free Trial
- Unlimited user seats added to Free Trial
- Segment events about limit on Free Trial confirmed
- Segment events CE-completion-limit-almost-reached and CE-completion-limit-reached updated to reflect completion limits for Professional plan
- Customers with AF on Professional plan will see warnings and banners about candidate completion limit
- Update ‘Upgrade to unlock’ modal from the small locked candidate card for Starter plan
- Update candidate limit information on Subscription page Professional plan tab
- Customers on Free Trial with unlimited user seats will see a notification to ‘Invite a new team member’ via a modal with information about the temporary team seat
- Assessment banners on the Results page for new Free Trial limit
Cancel Flow
- Zendesk should be opened on click of “Contact support” from “Other” option from ‘Cancel my account’ modal
SmartRecruiters
- ‘Complete Assessment order’ call into SR integration configured to finalise integration
OKRs
- Link website and employers app
- Assessment name added to event ‘A-account-created’ when the user comes from the website from specific assessment
- Simplify the onboarding flow from 3 steps to 1
Skill Groups
- Information about skill groups added to segment events to properly track their use
Internal Events and Reporting
- Add segment event ‘A-stripe-payment-failed’ to automate in app dunnings series
- Country name should be full in company_country property instead of the abbreviation eg IN will become India
- Changing the ID of form elements to help with Hubspot reporting
- Group call added to Segment includes all of the ID properties
Backend Improvements for Assessments
- Customers with enabled API feature can disable/enable candidate deadline for submission
- Group Randomization settings and Advanced Scoring settings are copied when Employer publishes assessment
- Assessment deadline won’t copy when customer publishes an Assessment into the library
- Copy template UUID while creating audition with method of autofilling from TT builder
- Save data from the public library about audition skill title (audition name, slug, skill and number of times used)
- Save data about adding, removing and usage of questions in assessments from the public library
- Don’t send Zapier events for inactive assessment
Bugs Squashed
Number Question Type
- Candidate’s answer on Number question type is exported to csv file
- Candidate’s answer and correct answer for Number question type are shown on shared cards (public, agency)
- Some invalid values in correctNumber field weren’t considered as invalid and have been updated
- “Answered” checkmark now disappears from Number question after candidates deleted their answer
Questions
- Duplicated entries removed from Production Airtable
- Question has incorrect status “Answers graded”, despite there is an empty answer to grade
- Google file for Google doc questions and test cases for Spreadsheet auto-graded questions are not visible in copied assessment if copied from admin
Tagging
- Fix deleting companies and candidates if some candidate had a tag
AI
- Some AI scores on Production aren’t updated because of different rounding in MySQL and php
- Old AI score is no longer shown for candidate who complete an interview for the 2nd time (after extending deadline)
- Grade Answer Error in PUP updated
iFrame
- Header sometimes moved to the middle of the page for several seconds when question downloads
Permissions
- Team editor can’t edit auto-progress setting if he has only 1 active available assessment and it is auto-graded has been fixed
- Wrong permission for a former team member who was invited again has been fixed
PLG Flows
- Customer upgrading from Starter to Professional was incorrectly given ATS flow after updating subscription via Stripe
- Fix redirection after sign up from pricing page so user lands on subscription page
- After registration customer was landing on My Assessments page instead of Expert Library page
- Language updated to “Upgrade” instead of “Talk to us” for Professional Plan
- Banner “candidate completion limit has almost been reached” isn’t shown for Employer on Professional plan on Results page
New Features
Anti-cheating
- Questions within skill groups can now be randomized. Users can select individual groups to randomize questions for and the number of questions that get displayed to candidates. For example, a candidate sees 9 questions that have been selected randomly from a total pool of 30 questions.
- Changes to questions groups are now auto-saved.
- See the help article for more information
SmartRecruiters Integration
- SmartRecruiters is shown in the list of integrations for employers with SmartRecruiters
Improvements
Create assessment builder
- Suggested skills in the Create builder have been updated to remove duplicates. Each skill will now only show once.
- Duplicate questions have also been removed from the Suggested option.
- Style fixes have been implemented on the notifications to make them easier to read
- Team members with View only access will no longer see the `invite candidates` button on the results page
- All questions related to a skill will now show in Suggested even if they don’t have a sample answer.
- The questions list in the Create assessment page now has filters for showing different skill groups. This makes it easier to manage assessments with a lot of questions per skill.
Active Learning
- The scoring component has been updated to make all numbers from 1 – 10 visible by default
- Scores now highlight to confirm selection before moving to the next answer.
Assessment Flow
- When you deactivate an assessment you will now remain in the `active` tab and a notification that the Assessment was deactivated will show as confirmation.
- The question groups page was refactored in the BE. The customer sees a new question groups page design that is easier to use and saves automatically.
More helpful error screens for candidates and employers
- Implement 404 page on Candidate’s app — if candidate’s find themselves on a page that doesn’t exist they will see a helpful 404 page that links to support.
- Update ‘down for maintenance’ screen for the employer and candidate — Employers and candidates will see a helpful and on-brand page when the product is down for maintenance
- Update error screen for the deactivated shared link — when a share link is deactivated the candidate sees a helpful error page.
Minor UI fixes to employer app
- Rename ‘Grading Instructions’ from AL to ‘Scoring Instructions’
- Add tooltips on hover of icons on collapsible right navigation panel from Grading page
- Side nav panel should be exposed by default on ‘Create&Edit Assessment’ page
- Users can use keyboard scrolling/navigation to scroll down a long candidate answer on candidate grading card — Bug fix for keyboard scrolling (down and up arrows) on a long answer in candidate card.
Bugs Squashed
Results CSV download
- All data and column names in CSV exports have been realigned so the data and columns match
Saving from assessment library
- If you navigate away from the page after clicking `add` on an assessment you don’t have to wait for the spinner to stop for the assessment to save to your list.
Mobile Platform
- Fix displaying the error on candidate dashboard when there are google questions in the assessment on mobile — allows candidates to do assessments with Google questions on Mobile and gives them the suggestion `we recommend completing this assessment on a desktop` instead of an error.
- Fix displaying the assessment timer in the mobile header — Shows assessment timer instead of the assessment expiration timer if the assessment timer will run out first.
Candidate
- Typo fixed for candidate “submission” — previously said “submittion”
- There is extra scroll in questions list in Candidate interview
- Wrong amount of max score in Text score type after adding not valid info/non-gradable question
- After generating assessment with tt builder Employer gets Multiple choice question with empty answer
- Employer gets several emails about assessment completion of candidate (the same assessment, the same candidate)
- Wrong expiration date of Assessment in letter with magic link for Candidate — candidate assessment reminder email now includes the correct expiration date.
- Wrong “expired” status and expiration time is shown on Candidate’s dashboard after assessment deadline was decreased
- Time in “in progress [time] ago” status resets every time when candidate resumes the interview or refreshes the interview page
- Candidate can apply via job link, which was opened before Assessment/Job was deactivatewe rec