AI & Machine Learning Generative Systems Hybrid · San Francisco, CA Full-Time
$180,000 – $230,000 USD · 0.15% – 0.3% Equity · Posted on Aug 15, 2026
Ensure Theme Framer models maintain the highest benchmarked accuracy in the industry by building automated sandbox testing frameworks.
What You Will Do
- › Build automated sandbox runners that execute and verify generated code across 10,000+ unit test suites.
- › Create synthetic benchmark datasets for Next.js, Tailwind, and full-stack API generation.
- › Identify model failure modes and partner with research teams on dataset filtering and RL feedback.
What We Are Looking For
- › 4+ years experience in Python, automated testing frameworks, and LLM evaluation tooling.
- › Strong knowledge of continuous integration (CI) and containerized execution.
Bonus Qualifications
- › Experience with HumanEval, SWE-bench, or custom code evaluation suites.
Primary Technology Stack
Python Docker PyTest PostgreSQL FastAPI GitHub Actions
✓ 100% Employer-Covered Health & Dental
✓ Meaningful Equity Ownership
✓ Remote-First Setup Stipend ($2,500)
✓ Flexible Time Off & Mental Health Days
✓ Annual Company Engineering Retreats
✓ $2,000 / Year Continuous Learning Budget
Shape The Future Of Autonomous Software
Explore all open roles across compiler infrastructure, generative AI, and design systems.
#J-18808-Ljbffr