PeakTalent Contributor Experience
AI Training & Annotation Platform
Peaktalent AI (Evaratus)
In production
Team
Naman Bhalla ( PM )
Shreyansh S ( Manager )
PeakTalent AI is Scaler's remote contributor platform, built within Evaratus, connecting vetted subject-matter experts with AI evaluation and data annotation projects for frontier labs and enterprise teams. Contributors complete specialized work such as response evaluation, code review, and expert data creation, with approved contributions becoming eligible for payout.
Note: To protect confidential information, all product data and identifiers have been modified. This case study reflects my work during my time at Scaler.
PeakTalent AI is Scaler's remote contributor platform, built within Evaratus, connecting vetted subject-matter experts with AI evaluation and data annotation projects for frontier labs and enterprise teams. Contributors complete specialized work such as response evaluation, code review, and expert data creation, with approved contributions becoming eligible for payout.
Note: To protect confidential information, all product data and identifiers have been modified. This case study reflects my work during my time at Scaler.
AI contributor platforms must balance two competing goals: helping organizations source high-quality human expertise at scale while enabling contributors to discover opportunities, qualify for work, and track compensation. As PeakTalent expanded beyond its initial release, workflows designed for an earlier stage of the product no longer supported its growing scale and complexity.
AI contributor platforms must balance two competing goals: helping organizations source high-quality human expertise at scale while enabling contributors to seamlessly discover opportunities, qualify for work, and track compensation. PeakTalent's initial version successfully validated our operational workflows. As the product matured, growing participation and new capabilities exposed opportunities to rethink experiences that had been designed for its earliest stage.
Rather than continuing to refine individual workflows, the focus shifted toward strengthening the parts of the product contributors relied on most and creating a stronger foundation for future growth.
The original Project Exploration and Payout system.
Internal research, usability sessions, contributor feedback, and ongoing collaboration with product and engineering helped identify where improvements would have the greatest impact.
Internal research, usability sessions, contributor feedback, and ongoing collaboration with product and engineering helped identify where improvements would have the greatest impact.
Project Exploration
The first step in accessing paid work, allowing users to evaluate projects before investing time in qualification.
Project Exploration
The first step in accessing paid work, allowing users to evaluate projects before investing time in qualification.
Contributor Workflow
Projects varied in both interview requirements and difficulty, but these details were only revealed after opening individual project pages. This made opportunities difficult to evaluate upfront, requiring contributors to inspect each project individually before deciding whether it was worth pursuing.
Surfacing these signals directly within the listing enabled more informed decisions at a glance.
Projects varied in both interview requirements and difficulty, but these details were only revealed after opening individual project pages. This made opportunities difficult to evaluate upfront, requiring contributors to inspect each project individually before deciding whether it was worth pursuing.
Surfacing these signals directly within the listing enabled more informed decisions at a glance.
New Project Listing
Users rarely evaluated every opportunity in depth. The redesigned listing was built around this behaviour, helping contributors compare relevant opportunities before exploring them further.
Users rarely evaluated every opportunity in depth. The redesigned listing was built around this behaviour, helping contributors compare relevant opportunities before exploring them further.
New Explore Experience
The same approach extended across the broader discovery experience, including search and filtering.
The same approach extended across the broader discovery experience, including search and filtering.
Search Results
Payouts
Work could now span multiple projects within the same payout cycle, with hundreds - or even thousands - of approved tasks contributing to a single settlement. While earnings accumulated task by task, payouts were ultimately received as one bi-weekly payment.
The original payout history no longer reflected this reality. Understanding what had been paid, when it was paid, and which projects contributed to each settlement became increasingly difficult as activity grew.
We reorganized the history around settlement dates rather than individual tasks, aligning it with how payouts were actually received while keeping project-level breakdowns accessible.
Payouts
Work could now span multiple projects within the same payout cycle, with hundreds - or even thousands - of approved tasks contributing to a single settlement. While earnings accumulated task by task, payouts were ultimately received as one bi-weekly payment.
The original payout history no longer reflected this reality. Understanding what had been paid, when it was paid, and which projects contributed to each settlement became increasingly difficult as activity grew.
We reorganized the history around settlement dates rather than individual tasks, aligning it with how payouts were actually received while keeping project-level breakdowns accessible.
Payouts Section
Understanding a payout shouldn't require downloading a separate settlement statement. The essential details should be immediately understandable, with complete records available when needed. I first explored keeping settlements inside a single history view, but the interface became increasingly difficult to navigate as records accumulated.
Understanding a payout shouldn't require downloading a separate settlement statement. The essential details should be immediately understandable, with complete records available when needed. I first explored keeping settlements inside a single history view, but the interface became increasingly difficult to navigate as records accumulated.
Breaking each payout into dedicated views made individual settlements easier to inspect while keeping the overall history easy to scan.
Breaking each payout into dedicated views made individual settlements easier to inspect while keeping the overall history easy to scan.
Settlement breakdown shown in context.
Completed work could remain under review before becoming eligible for payout, creating a delay between finishing a task and seeing those earnings reflected. The By Project view kept earnings visible as they progressed toward the next settlement.
Completed work could remain under review before becoming eligible for payout, creating a delay between finishing a task and seeing those earnings reflected. The By Project view kept earnings visible as they progressed toward the next settlement.
By project
This also aimed to make earnings easier to reconcile.
This also aimed to make earnings easier to reconcile.
Account Settings
The existing Account experience organized profile information across separate sections. Payout setup remained within Payouts, despite being something contributors typically configured only once and rarely revisited.
Account Settings
The existing Account experience organized profile information across separate sections. Payout setup remained within Payouts, despite being something contributors typically configured only once and rarely revisited.
Moving payout settings into the Account experience alongside the new verification flow created an opportunity to rethink how account-related information was organized.
Moving payout settings into the Account experience alongside the new verification flow created an opportunity to rethink how account-related information was organized.
This also allowed me to establish a more consistent visual language and navigation across these surfaces, making them feel like part of the same product.
This also allowed me to establish a more consistent visual language and navigation across these surfaces, making them feel like part of the same product.
A dedicated review step within the new setup flow allowed financial information to be checked before entering verification, helping prevent avoidable mistakes before submission.
A dedicated review step within the new setup flow allowed financial information to be checked before entering verification, helping prevent avoidable mistakes before submission.
Payout methods were no longer activated immediately. Introducing verification meant contributors now had to wait while their information was reviewed.
Payout methods were no longer activated immediately. Introducing verification meant contributors now had to wait while their information was reviewed.
For payout methods that required external authentication (OAuth), users could resume setup if the flow was interrupted.
For payout methods that required external authentication (OAuth), users could resume setup if the flow was interrupted.
While the long-term vision included multiple payout accounts and a default destination for future settlements, the release focused on a single verified account to keep it simple while leaving room to scale.
While the long-term vision included multiple payout accounts and a default destination for future settlements, the release focused on a single verified account to keep it simple while leaving room to scale.
Looking ahead
Beyond the immediate scope, I explored ideas that could further enhance the contributor experience over time. A lightweight step indicator could show what to expect before beginning, making a multi-stage setup easier to anticipate.
Looking ahead
Beyond the immediate scope, I explored ideas that could further enhance the contributor experience over time. A lightweight step indicator could show what to expect before beginning, making a multi-stage setup easier to anticipate.
Using existing profile information, resumes, and qualification history, AI-assisted matching could help surface relevant projects without relying on manual browsing.
Using existing profile information, resumes, and qualification history, AI-assisted matching could help surface relevant projects without relying on manual browsing.
These explorations were intended to facilitate conversations with the wider team and inform direction.
These explorations were intended to facilitate conversations with the wider team and inform direction.
Designing across interconnected contributor workflows required thinking beyond individual features. To ensure decisions remained consistent across the platform, I relied on a shared set of design principles throughout the project.
Designing across interconnected contributor workflows required thinking beyond individual features. To ensure decisions remained consistent across the platform, I relied on a shared set of design principles throughout the project.
Shaping these core contributor workflows meant continuously exploring ideas, gathering feedback, communicating design decisions, and balancing contributor needs with technical and operational constraints.
As of June 2026, platform remains in beta, with the redesigned experience being rolled out in phases. While long-term impact is still being measured, early qualitative feedback has been positive. Thankyou
Shaping these core contributor workflows meant continuously exploring ideas, gathering feedback, communicating design decisions, and balancing contributor needs with technical and operational constraints.
As of June 2026, platform remains in beta, with the redesigned experience being rolled out in phases. While long-term impact is still being measured, early qualitative feedback has been positive. Thankyou