Applicant tracking systems demonstrate well. Every vendor shows a clean pipeline, a tidy candidate profile and a report that looks useful. Feature comparison at that level produces a spreadsheet where everything has a tick and nothing is decided.
These twenty-one questions separate systems. They are ordered by how expensive the wrong answer becomes.
Process fit
1. Can we define our own stages, per role type? Fixed stages force your process into the software model. Ask to see two different pipelines side by side.
2. Can evaluation criteria differ by role? Scoring an engineer and a warehouse supervisor on one form makes the form meaningless — see scorecards.
3. Can we require scores before the debrief? If everyone can see each other scores before submitting, independent evaluation is not enforceable.
4. How does bulk action work at volume? Advancing or rejecting two hundred candidates should take one action. Ask them to demonstrate it, not describe it.
5. Can candidates schedule themselves? Against real interviewer availability, with rescheduling. This is where elapsed time actually goes.
Data and compliance
6. Where is candidate data stored, primary and backup? Frequently different regions, and the backup answer is volunteered least. Why it matters.
7. Which sub-processors touch candidate data, and where? Ask for the list before signing. Speed of response tells you as much as the content.
8. Can support staff read candidate records, from where, and is access logged? Access is a transfer. Scoped and logged is fine; unlimited and unlogged is not.
9. Can retention periods be configured, and does deletion actually run? A retention field that nothing acts on is decoration — the requirement.
10. Can we produce everything held about one candidate, on request? Including interview notes and scores. Subject access requests arrive eventually.
11. Is there an audit trail of who viewed and changed what? The question every regulator eventually asks is not what happened but whether you can show it.
AI and automated evaluation
12. Which features produce a score, ranking or recommendation about a person? A precise list. This determines what falls under high-risk obligations.
13. Can you show why this candidate received this score? The single best question on the list. A description of the model is not an answer.
14. What is excluded from scoring? Name, gender, age, photograph, institution. If nothing is excluded, everything is a potential proxy.
15. Can we export scores and outcomes to test for disparate impact ourselves? If you cannot get the data out, you cannot test it.
16. Are you the provider of these features, and what changes if we customise them? Get it in writing — provider or deployer.
Integrations and distribution
17. Is calendar integration two-way, with real availability? An emailed calendar file is a notification, not an integration.
18. Does one posting action reach our careers page, our boards, and a structured feed? With source recorded on every application that returns. What good looks like.
Commercial and exit
19. What is the pricing model, and what happens when we exceed a limit? Per user, per job, per hire, or tiered — each rewards different behaviour. The comparison.
20. What does implementation actually involve, and who does the work? Ask for a week-one plan and a named owner, not a duration.
21. If we leave, what do we get, in what format, and how long until data is deleted? Including interview notes and evaluation history. Exit cost is rarely priced and always paid.
How to run the evaluation
Send the questions in advance. Written answers are comparable and quotable later. A demo is a performance; a document is a commitment.
Weight them before you see the answers. Decide which five matter most while you are still objective. Otherwise the vendor with the best demo defines the criteria retrospectively.
Get two recruiters to run a real requisition in a trial. Not a guided demo — an actual role, with actual candidates. Most systems fail on daily friction rather than missing features, and friction is invisible in a demo.
Ask a reference customer about the worst month. Every product is pleasant when nothing is wrong. What happens during a hiring surge, or when an integration breaks, is what you are actually buying.
Our own answers are on security and compliance, pricing, and across the product pages.