Our Methodology

The Legit Score (our 0–100 rating of how safe a platform is to work for, based on published inputs) answers one question: is this platform safe to work for? It combines five inputs. Our content team sets the inputs and weights and publishes any change here.

Until a platform has 10 approved worker reports, its payout input is based on our editors' own test accounts and the platform's published payout terms, and the platform page says so.

The Pay Index (our running record of what AI-training platforms pay per hour, built from every job we track) is built from every job we track, not from surveys or platform marketing. Today it covers 1,504 live roles that state pay, out of 2,174 live roles in total.

What we calculate

• Median pay: the middle value of each role's pay mid-point, in US dollars per hour. Half of roles pay more, half pay less.
• Minimum and maximum: the lowest posted minimum and the highest posted maximum.
• Open roles: live roles on the day, after duplicates are merged.
• Time open: days between when we first and last saw a role at its source.
• Month-on-month change: this month's median against last month's.

Figures are broken down by platform, profession and country.

How we make pay comparable

• Hourly pay is used as posted.
• Yearly pay is divided by 2,080 hours (40 hours × 52 weeks).
• Per-task pay is converted using the platform's stated time per task. If none is stated, we use the median time per task from worker reports.
• Other currencies are converted to US dollars at that day's reference rate.
• We always show the original posted amount next to the converted figure.

Roles that do not state pay are counted as open roles but left out of pay figures. History is never deleted, so trends use every day since we started tracking. See the Pay Index or download the data as CSV.

Where jobs come from

We collect jobs only from approved sources: the platforms' own career pages and hiring systems. We respect each source's terms and robots rules, and every job links to the original posting and names the real employer.

• Tier 1 (live now): Prolific, Meridial (Invisible Technologies), Lilt, Mercor, Welo Global, Mindrift (Toloka), 10x Team, Rise Data Labs, micro1, Anyone AI, Alignerr, RWS TrainAI.
• Tier 2 (being added): HumanSignal, xAI (AI Tutor roles), Innodata, Scale AI Human Frontier Collective, Gramian, g2i, Toloka, Handshake AI, 10a Labs, Remotasks.
• Discovery: a weekly check of cross-company job searches flags new AI-training employers. An editor approves each one before it becomes a source.
• Never used: Indeed, LinkedIn, Glassdoor, ZipRecruiter, SimplyHired, reposting sites and other job boards.

How fresh it is

• Every live job is re-checked against its source at least every 24 hours.
• A job that disappears at the source is marked expired and removed from listings and sitemaps within 24 hours.
• If a source breaks, we flag it within 2 hours and aim to fix it within 48 hours. Its jobs are not shown as verified in the meantime.
• The “Updated” date on a page only moves when its data or content changes.

How we clean it

• The same role posted for many cities or countries is shown once, with a list of locations. For example, one role posted for 20 US cities is one job on our board.
• Titles and company names are cleaned: correct casing (RN, PhD, LLC), no tracking codes, no repeated words.
• Countries and languages are matched to standard names so filters and country pages are accurate.
• Each job is sorted into a profession, job type and pay band. Editors can correct the result.

Which pages we offer to search engines

A job page is offered to search engines only if it has a clear description, pay or a pay range, eligible countries, and is not a copy of another role. Today 1,504 jobs of 2174 pass this bar. Pages below it stay on the site for visitors. A jobs page for a profession, job type or country is offered only once it has at least 5 live jobs.

Anyone who has worked on a platform can report their experience. We ask for the platform, role, pay received, hours offered, whether you have been paid yet (and if so, how many days the first payout took), every screening step you went through, the outcome, your country and the month.

• Every report goes to a moderation queue. An editor checks it is plausible, not a duplicate and free of personal details.
• Only approved reports are used. They are anonymised: we never show names, email addresses or anything that could identify you.
• We show a platform's report figures (such as median days to first payout) once it has at least 10 approved reports.
• Approved reports feed the payout reliability input of the Legit Score.
• We never write, buy or edit reports to change a platform's picture.

So far we hold 0 approved reports across the platforms we review; 0 of 8 platforms have enough to show figures.