Senior Architect, Model Performance and Post-Training
Job Description:
- Develop meticulous frameworks to measure and analyze coding model performance across code generation, debugging, code completion, repository-level reasoning, and multi-step software engineering tasks
- Develop and optimize post-training pipelines for coding models, including supervised fine-tuning (SFT), preference optimization, reinforcement learning (RL), and outcome-based or process-based reward modeling
- Analyze benchmark results, training dynamics, failure modes, and model behaviors to find opportunities for improvement
- Improve model performance on sophisticated coding tasks through targeted training interventions, reward design, curriculum development, and iterative error analysis
- Build scalable, reproducible infrastructure for training experiments, evaluation pipelines, and model performance analysis
- Collaborate with research scientists, infrastructure engineers, and product teams to turn model insights into production-ready capabilities
Requirements:
- Bachelor's degree (or equivalent experience) in Computer Science or AI
- 12+ years of meaningful work experience
- Excellent programming skills in Python
- Strong understanding of large language models, transformer architectures, and training and evaluation of generative models
- Foundational understanding of reinforcement learning algorithms and implementation, including MDPs, policies, and reward design
- Deep understanding of software engineering fundamentals, code generation, automated testing, and program correctness
- Proven track record to lead model development life cycles, including dataset versioning, experiment tracking, and evaluation pipelines
- Experience training or post-training state-of-the-art coding models or models designed for software engineering agents
- Experience with execution-based rewards, verifiable rewards, reward hacking mitigation, and automated code evaluation
- Familiarity with coding benchmarks and evaluation environments such as SWE-bench
Benefits:
- Highly competitive salaries
- Comprehensive benefits package
- Equity
- Benefits for you and your family
