AI Coding Evaluation And Training Developer
The Work
As an AI Coding Evaluation and Training Developer, you will build and review software used to train and improve AI models. You will combine software development with model evaluation, response ranking, supervised fine-tuning dataset work, and reinforcement learning with human feedback.
You will apply technical judgment and consistent standards to coding tasks and broader user queries. The work also includes writing clear rationales, reviewing code and documentation, and giving useful feedback that supports model and product improvements.
- Design, develop, and maintain readable, reusable, efficient code for AI model training and optimization.
- Benchmark model performance and evaluate and rank model responses against defined criteria.
- Create clear explanations and rationales for evaluation decisions.
- Create and maintain task-specific datasets for supervised fine-tuning.
- Support reinforcement learning with human feedback workflows and reward-model refinement.
- Develop evaluation strategies that consider user needs, technical accuracy, and ethical alignment.
- Create and improve responses used to train models.
- Review code and documentation, provide constructive feedback, and contribute to product improvements.
What It Pays And Takes
The role is open worldwide and is structured as part-time contract work. The listing does not specify a pay rate.
You should be comfortable building software and explaining technical decisions in clear English. A bachelor's or master's degree in engineering or computer science is preferred, but equivalent experience may also qualify.
- Hours: 20 or more hours per week.
- Location: Worldwide.
- Language: Excellent spoken and written English.
- Work type: Contractor and part-time.
- Experience level: Entry level.
- Required programming: Strong Python and JavaScript or TypeScript skills, including familiarity with ES6 conventions.
- Application development: Experience building modular, scalable web applications with attention to readability, security, stability, and testing.
- Frameworks and tools: Familiarity with Node.js or NestJS and React, Angular, or Vue.js, plus Docker knowledge.
- Code quality: Ability to write clean, organized, correct, and clearly annotated code.
- Preferred education: Bachelor's or master's degree in engineering or computer science, or equivalent experience.
- Helpful background: Software quality assurance, test planning, test-driven development, LLM prompting, code review, model evaluation, supervised fine-tuning, or RLHF.
How It Works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI Training Work
AI training is the human work behind many AI systems, including writing and rating model responses, evaluating code, and preparing examples for models to learn from. People with strong technical skills are paid to apply careful judgment and help make model outputs more accurate, useful, and reliable.
