Preference Model

Preference Model is a superintelligence data research company that builds reinforcement learning environments for training more capable, better-aligned AI models. Founded by Jennifer Zhou, an early member of Anthropic's data team, and Ning Cao, an early employee at DatologyAI, the San Francisco company spent its first year building reward-hacking-resistant RL environments for frontier AI laboratories, focused on machine-learning engineering and AI research tasks. Alongside its $16M seed round led by Andreessen Horowitz, it open-sourced Karotte, the production framework it uses to build and grade adversarially hardened training environments.

• Seed
• AI/ML
• Open Source Software
• Software & Internet