# Hamel Husain

Hamel Husain teaches engineers and product managers how to evaluate AI systems and connect evaluation results to customer needs. His work includes courses and a book about AI evaluations.

- Profile: https://b2binfluencer.marketing/creators/hamel-husain/
- LinkedIn: https://www.linkedin.com/in/hamelhusain
- LinkedIn followers: 34,579
- Median of last 3 posts: 45 reactions, 24 comments (checked 2026-11-02)
- YouTube: https://www.youtube.com/channel/UC__dUuqF5w4OnbW221JxmKg
- Topics: artificial-intelligence, product-discovery, ai-in-education
- Based on 9 dated posts, latest 2026-09-02

## Summary

Hamel Husain is most useful for teaching engineers and product managers to evaluate AI systems and connect evals to customer needs.

- Best for: A B2B AI tooling company upskilling product and engineering teams on evals and AI product discovery.
- Not for: Consumer lifestyle app seeking paid social promotion for short video series to broad demographic.
- Check first: Confirm whether bespoke workshops or advisory retainers are available, since only course seats and book are documented.

### Takeaways

- Public OCR benchmarks can expect absent information and penalize correctly preserved line breaks.
- Ask customers to describe a specific past problem, then let evals test the fix.
- Question whether an eval dashboard is the best way to share results.

## Main ideas

- **A benchmark score is a claim to audit, not a ranking to trust**: Hamel Husain argues that public benchmark scores are weak grounds for choosing a model, because the benchmarks themselves contain errors that distort the ranking. His reasoning is that a single overall score hides the capability differences a buyer actually cares about, and that scoring rules and postprocessing can move results more than real model quality does. He makes the case through a talk by Vik Paruchuri, presenting it as an argument he endorses rather than his own finding.
- **Be skeptical of eval dashboards**: Hamel Husain treats the dashboard as the wrong unit of analysis. Asked what a good eval dashboard looks like for stakeholders, he answers that teams should be very skeptical of dashboards and that with AI available they can do better. The argument follows from his benchmark criticism: a compiled score loses the specifics that would tell you whether the system is improving where it matters.
- **Ask about a real past problem, then let evals test the fix**: Hamel Husain argues that most customer conversations engineers run produce weak signal, because asking what someone thinks of an idea or showing a demo invites polite reaction rather than evidence. He describes Teresa Torres's method, attributing it to her: ask customers to describe a specific time they experienced a problem, then walk through what they did. The interviews are synthesized to prioritize work, which he notes is harder than it sounds and takes practice. Evals then serve as a second feedback source, testing whether the product solves the problem customers described.
- **Teach fundamentals in a book, refresh the changing parts live**: Hamel Husain splits his teaching by how fast the material moves. The live course has been the only guided way to learn evals precisely because the format allows the curriculum to be updated as methods emerge; the book covers fundamentals he expects to hold as models improve. Shreya Shankar is named as his co-teacher for the refreshed course.
- **Demand for eval skills is visible in hiring data**: Hamel Husain uses a simple repeated count to argue that evaluation work has moved from a niche interest to a routine expectation. His method is counting eval mentions in every Hacker News "Who is hiring?" thread since late 2022 and reporting the change as a share. He reads the pattern as companies that shipped an AI product needing someone to measure whether it is getting better and where it needs work, across engineering, infrastructure, and product management roles.

## Work to read

- [Why public benchmarks can mislead model buyers](https://www.linkedin.com/posts/hamelhusain_should-you-rely-on-public-benchmark-scores-activity-7523165476286345218-tnry)
- [Book launch: Evals for AI Engineers](https://www.linkedin.com/posts/hamelhusain_we-are-launching-our-book-%F0%9D%97%98%F0%9D%98%83%F0%9D%97%AE%F0%9D%97%B9%F0%9D%98%80-%F0%9D%97%B3%F0%9D%97%BC%F0%9D%97%BF-activity-7521251035919593472-qrBl)
- [Customer discovery before building an AI product](https://www.linkedin.com/posts/hamelhusain_its-easy-to-build-the-wrong-thing-with-ai-activity-7501029958111145984-f4IW)
- [Evals are showing up in AI job postings](https://www.linkedin.com/posts/hamelhusain_i-counted-how-many-ai-job-postings-mention-activity-7486435054219558913-0WxV)
- [A skeptical answer on eval dashboards](https://www.linkedin.com/posts/hamelhusain_q-what-does-a-good-eval-dashboard-look-like-activity-7521255239228964864-ja_P)
- [How an experienced educator is redesigning education with AI](https://www.linkedin.com/posts/hamelhusain_the-call-to-adventure-activity-7490771164798251008-Unyq)

## Brand relationships

- **AI Evals for Engineers & PMs** (owned_business)
- **Raindrop** (disclosed_partnership)

## Partnerships

Hamel Husain's posts describe two paid commercial products: a live cohort course, "AI Evals for Engineers & PMs," co-taught with Shreya Shankar, and a book, "Evals for AI Engineers." Both teach teams how to evaluate AI systems. The posts report that the course has taught more than 4,500 students, names past corporate participants such as Google, OpenAI, Meta, and Amazon, and advertises a next cohort start date; a separate post marks the book launch and a Bay Area meetup. These enrollment and event links are the only commercial routes named in the material reviewed. None of the posts states a sponsorship policy, says whether Hamel Husain accepts brand partnerships, lists rates, deliverables, or availability, or points to a partnership contact. The practical consequence for a buyer: the documented way to work with Hamel Husain is to buy seats in a scheduled cohort or buy the book, and there is no published channel for a bespoke workshop, advisory retainer, or sponsored content.

## Alternatives

- Sabina Sobhani: Sabina Sobhani is the closer substitute when the client wants someone who has shipped an AI product in a high-stakes setting and can teach the design decisions behind it, rather than benchmark methodology. The reviewed post describes a framework that combines automated evals with human review on every change, explicit bias testing across demographic contexts before users see the product, and context engineering as at least as important as prompt engineering. That maps onto a team building an AI feature whose output drives a person's next step. It is also the better pick when the training should be tied to how practitioners actually use the tool: the reviewed material links product quality to how educators support individual students, which is a concrete account of designing for trust under real consequences.
- Amanda Eldridge: Amanda Eldridge is the better choice when the brief moves away from upskilling an internal engineering team on evals and toward building a supply of AI-skilled people, or toward a community college partnership. The reviewed posts describe a programme run across 85-plus California community colleges, with more than 4,000 applicants, nine student teams pitching apps built on Google Gemini and films made with Veo 3 to Google executives, and named student projects such as a pet-symptom checker and an accountability workout app. That is a detailed, end-to-end account of running a workforce programme with an external partner. A client whose goal is recruitment, apprenticeship or regional AI talent supply would weigh that delivery record above eval methodology.
- Colin Sailor: Colin Sailor is the better choice when the buyer task is teaching practitioners to deliver instruction online to students with diverse needs. The reviewed posts describe five years of online teaching, why online formats remove stressors that students face in physical schools, how families report the approach fitting their situation, and a firm position that tools do not replace empathy, understanding and connection — with a note that students look for human solutions at the end of the school year. For a client whose AI product is used in remote or special education settings, an instructor who has taught in that setting for years is a different and useful input to a training or advisory engagement.

---
AI-assisted summary of public posts. Audience demographics are not available. Engagement appears only where we checked recent posts. Method: https://b2binfluencer.marketing/methodology/