Senior Software Engineer, AI Evaluation Infrastructure

Description: Build infrastructure for evaluating coding agents and language models, including benchmark datasets and automated task quality checks. Own features across the stack, from data models to user interfaces, ensuring reliability and performance. Requirements: 5+ years of experience in production backend systems, with ownership of at least one service. Strong proficiency in Python, experience with Django, PostgreSQL, and task queues like Celery. Hands-on experience in LLM evaluation and understanding of benchmark task construction and scoring. Ability to assess task quality and make sound design tradeoffs. Ownership mentality, capable of independently completing tasks from request to deployment. Familiarity with TypeScript and React for frontend changes. Clear communication skills in writing pull request descriptions and design notes. Benefits: Work at the forefront of AI, contributing to advanced models and datasets. Opportunity to showcase work at top AI conferences. Apply AI innovations to real-world business challenges. Collaborate with experienced colleagues from leading tech companies. Enjoy the speed and impact of a startup environment.