Hackathon - Agent Eval Harness

The Friendly Data Meetup hosts a free evening hackathon where teams build an evaluation harness to test an AI customer support bot.

Participants work in teams of 2-4 to build an AI agent evaluation harness from scratch: writing realistic customer test questions, creating automatic pass/fail checks with an AI judge, and improving a fictional support bot with a before/after scorecard. The challenge is split into levels with bonus rounds on tracing, red-teaming and model shoot-outs, ending in three-minute demos. Doors open at 17:30, the hackathon runs 18:00-21:00, and attendees should bring a laptop with Docker, uv and an LLM of their choice.