Enterprise workshop
AI Evals
Learn to build rigorous, automated AI evaluation pipelines that catch hallucinations, measure accuracy, and provide the data-driven confidence needed to build high-performing, accurate, and reliable AI agents.
Next workshop
No workshop scheduled
Get Registered
Submit the form and we'll be in touch
Next workshop
No workshop scheduled
Why you should attend
Professional, in-person training to master AI evals and build reliable AI agents.
AI Agents are quickly moving from experimentation to production, but most teams struggle to design them well, measure their quality, and deploy them with confidence. Stop guessing and build certainty in measuring AI performance. Learn practical skills to design agentic systems, systematically evaluate their performance, and continuously improve quality using modern AI evaluation techniques.
Example Workshop Agenda
-
9:00 AM
Arrival, coffee and breakfast
Get settled and prepared for your workshop.
-
9:30 AM
Workshop kick-off & morning session
Introductions, building AI agents, and exploring eval fundamentals.
-
12:00 PM
Lunch break
Casual networking over a catered lunch.
-
1:00 PM
Afternoon session
Hands-on group session learning and applying systematic evaluation frameworks for an AI agent.
-
4:30 PM
Wrap-up and networking
Enjoy drinks and refreshments while connecting with your peers from the workshop.
Attendees Will:
Get Hands-On Practice Building Functional AI Agents
Use industry-leading tools to create your own AI agent for evals. Learn a reusable framework to successfully build AI agents, defining logic, guardrails, and success metrics.
Develop and Implement a Systematic Evaluation Plan
Don’t leave results to chance. Learn to evaluate AI performance, and understand where and why failures happen. With evals, turn unpredictable LLM outputs into stable, production-ready capabilities.
Receive Live, In-Person Training Led by Experts
Learn from industry experts who will get you up to speed with AI evals. Workshops are hosted in our modern, world-class spaces, built for engaging learning and collaboration.
Want to Train an Entire Team?
Book a call to discuss rapidly upskilling an entire team with modern AI tools and techniques through this collaborative, hands-on workshop.
Enterprise workshop
Workshop Curriculum
Part 1
Agentic Foundations: Building AI Agents for Evals
Learn how AI agents work and master where to apply them for real, measurable impact. Led by your expert instructor, build a functional AI agent.
- Key Skills:
- Agentic AI Foundations
- Building AI Agents
Understanding Agentic AI
Build a clear understanding of how modern AI and agentic systems work, reason, and complete tasks. Explore practical, high-value use cases across industries, and how agents differ from AI chatbots. Build the confidence to speak to these technologies and identify where they can add value in your professional workflow.
Building Your Agent
Master a practical framework to properly design AI agents. Apply your new skills to create your own agent. Define the core parameters, step-by-step logic and guardrails. Build a functional agent and take home a reusable process for future agentic work.
Part 2
Identifying Issues: AI Evals & Failure Analysis
Stop issues before they reach users. Build and implement a structured testing plan to validate your agent’s performance, document its shortcomings, and understand the root cause of critical issues.
- Key Skills:
- Evals Foundations
- Testing Agents
- Failure Mode Analysis
Understanding AI Evaluations
Skip the guesswork and systematically validate AI performance. Learn evaluation scenarios, workflows, and methods to measure and improve AI performance. Apply a clear framework to get real results with AI evals.
Systematic Failure Mode Analysis
Don’t spot check performance for scalable AI systems. To build production-ready solutions, understanding exactly where and why AI systems fail is critical. Run your AI agent and use key methods to test AI performance and understand system weakness.
Part 3
Scaling Quality: Measurement & Optimization
Quantify your agent’s quality and performance with automated evals. Leverage eval results to systematically improve agent performance.
- Key Skills:
- Prompt-Based Validation
- Continuous Optimization Techniques
Automated Evaluation & Graders
Manual testing doesn’t scale. Learn to create and run automated graders that test performance against your defined metrics. You’ll walk away with an operational eval set with clear metrics and automated testing.
Performance Optimization
Make informed improvements and use your automated grader results to drive iteration loops. Apply data-driven prompting techniques to systematically improve your agent's reliability scores
Showcase & Feedback
Present your work, share key learnings, and receive expert feedback. Connect with peers and walk away with actionable insights you can apply to future AI eval work.
Attendees Earn a BrainStation Certificate
Attendees who successfully complete the workshop will earn a Certificate of Completion, demonstrating their commitment to developing and maintaining cutting-edge tech skills.
Workshop Details
Who Should Attend
This workshop is designed for professionals from a variety of disciplines. Whether you’re in product, engineering, operations, strategy, consulting, or leadership, you’ll get hands-on, expert led instruction to design agents and perform rigorous evaluations that improve the quality, performance, and reliability of your AI implementations.
What's Included
- Local coffee & breakfast
- Catered lunch
- Cocktails and snacks
Have dietary restrictions?
To ensure we can accommodate your dietary needs, please inform us at least 48 hours before the workshop start date.
What to Bring
- A laptop and charger
- A notebook (optional)
Leading Brands Learn from BrainStation®
BrainStation is respected in the industry, it really sets you up for success. All the instructors go the extra mile by layering on lessons and professional experience.
Jon
Sr. Integrated Marketing Lead at Microsoft, BrainStation Instructor
BrainStation is respected in the industry, it really sets you up for success. All the instructors go the extra mile by layering on lessons and professional experience.
Jon
Sr. Integrated Marketing Lead at Microsoft, BrainStation Instructor
Welcome to BrainStation New York in SoHo
How to Get Here
BrainStation New York
136 Crosby St, 4th Floor, New York City, NY 10012
-
Finding Your Way
Located on Crosby St just south of Houston between Broadway and Lafayette, just steps away from the Lafayette B D F M train.
-
Entering The Building
A BrainStation team member will be waiting for you at the elevators to take you to the 4th floor.
-
Once You're Inside
Our staff will be there to greet you at the campus reception.
Frequently Asked Questions
-
How can I learn more about AI?
BrainStation has a number of AI training options that can support your learning goals. We have a series of core, hands-on workshops teaching how to use AI meaningfully and effectively as part of your workflows. Check out our core workshops: AI for Marketers, AI for Product Managers, and AI for Designers. For training focused on specific areas of AI, we have a series of specialized workshops: AI Evals, and AI Search Optimization.
For more general knowledge on AI, professionals often take our Artificial Intelligence Certification course. We also host a series of AI Labs, which are approachable, highly-focused learning experiences centered around specific tools. Some of our most popular AI Labs include, Vibe Coding with Lovable, Rapid Prototyping with Figma Make, and Building Agents with Open AI. Check out upcoming AI Labs on our Events page.
-
How do I sign up for this workshop?
Please fill out the form on this page and our team will get in touch to discuss workshop packages.
-
Can I take this workshop on my own?
This workshop is designed for corporate teams so most attendees join alongside multiple colleagues and co-workers.
-
Can I attend this workshop online?
In order to get the most out of the time, this workshop is only available in-person.
-
Will I receive a certificate by completing this workshop?
Yes, all attendees who complete the workshop are awarded a Certificate of Completion.
-
I have more questions, can I speak with someone?
Of course! Please fill out the form on this page and our team will be in touch to answer any questions you have.