Correction (12 October 2026): an earlier version of this article showed a chart of “match accuracy” percentages and described the weighting of core skills incorrectly. We could not support those figures, so they have been removed and the article rewritten from how the matcher actually works and from our own evaluation results.
Keyword search treats a job as a bag of words. A search for “Python backend” can return a senior distributed-systems role, a data-entry job that mentions Python scripting, and a sales-engineering role, all ranked by how often the words appear. It also misses near-matches: an engineer who has built systems on Pulsar and RabbitMQ is a reasonable fit for a role that names Kafka, but a literal filter will not connect them.
How Jobspiq scores a match
- Meaning, not words. Candidate profiles and job postings are turned into 768-dimensional vectors with nomic-ai/nomic-embed-text-v1.5 (using the search_query: prefix for candidates and search_document: for jobs). The closeness of the two vectors is the starting score.
- Penalties for the wrong kind of job. When a candidate’s technology stack differs from the job’s and their background does not already cover it, the score is cut by 45% (90% when one side is a corporate or non-engineering track). A domain-compatibility penalty applies when the candidate’s and the job’s domains are far apart.
- Experience fit. Each year a candidate is short of the required experience cuts the score by 20% (capped at 85%). Each year of over-qualification cuts it by 18% (capped at 70%).
- Skills. Each of the job’s core skills the candidate has adds 0.05 to the score (up to +0.15). Each core requirement they are missing subtracts 0.10 (up to −0.30).
- Optional reranking. A cross-encoder (BAAI/bge-reranker-base) can re-score the strongest matches.
Jobs scoring below 0.45 are hidden. Alerts are sent above a threshold that defaults to 0.70 and that each candidate can set anywhere from 0.50 to 0.95.
How we check it
Penalties and thresholds in a matcher are tuned by hand, so we measure them. Each week a scheduled job re-scores a fixed set of hand-labelled candidate–job pairs against the live engine. Each pair was labelled by a person: 3 = should send an alert, 2 = relevant, 1 = borderline, 0 = wrong. The latest run (12 October 2026) scored 117 of the 140 pairs:
- Shown jobs: 90% of jobs the engine would show were labelled at least borderline relevant.
- Alerts: 88% of alert-level matches were labelled relevant or better.
- Missed jobs: 14% of the jobs the engine hid were ones a person had labelled relevant.
- Rank agreement: the Spearman correlation between engine score and human label was 0.60. Over the eight weekly runs since 24 August it has ranged from 0.60 to 0.71.
What this does and does not show
- The sample is small. 117 scored pairs can show a regression, but they are not a precise accuracy figure.
- There is no keyword baseline. We have not run keyword search against the same labelled pairs, so we do not claim a comparison with it.
- It moves. The score changes week to week as the matcher and the job data change, which is why we track it.