How to Design an AI Agent for Production
Important Links
Goal
The purpose of this guide is to explain the underlying tech and logic used to deploy a scheduling agent, Cal.ai, in production using LangChain.
Context
Recently, our team at Rubric Labs had the opportunity to build Cal.com's AI-driven email assistant. For context, we’re a team of builders who love to tackle challenging engineering problems.
The vision for the project was to simplify the complex world of calendar management. Some of the core features we had in mind included:
- Turning informal emails into bookings: "Want to meet tomorrow at 2?"
- Listing and rearranging bookings: "Cancel my next meeting"
- Answering basic questions: "How does my Tuesday look?"
and more. In summary, we wanted to build a personal, AI-powered scheduling assistant that can offer a complete suite of CRUD operations to the end user, all within the confines of their email client using natural language.
Emailing Cal.ai.
Architecture
We decided to achieve this using an AI agent, particularly an OpenAI functions agent which is better at dealing with structured data, given Cal.com's API exposes a set of booking operations with clear inputs. The underlying architecture is explained below.
Agent architecture. A scratchpad is a type of memory where the AI writes to itself.
Input
A Cal.com user may email username@cal.ai (e.g ted@cal.ai) with a request such as “Can you book a meeting with sarim@rubriclab.com sometime tomorrow?”.
The incoming email is cleaned and routed in the receive route using MailParser and the addresses are verified by DKIM record, making it hard to spoof them.
Here we also make additional checks, such as ensuring that the email is from a Cal.com user to prevent misuse. After the email has been verified and parsed, it is passed to the agent loop.
Agent
The agent is a LangChain OpenAI functions agent which uses fine-tuned versions of GPT models. This agent is passed pre-defined functions or tools and is able to detect when a function should to be called. This allows us to specify the required inputs and desired output.
The agent is documented in the agent loop.
There are two required inputs for an OpenAI functions agent:
- Tools — a tool is simply a Javascript function with a name, description, and input schema
- Chat model — a chat model is a variation of an LLM model that uses an interface where "chat messages" are the inputs and outputs
In addition to the tools and the chat model, we also pass a prefix prompt to add context for the model. Let’s look into each of the inputs.
Prompt
The prompt is an opportunity to direct the model by providing context. The user’s relevant information (userId, user.username, user.timezone, etc.) is used to construct a prompt which is passed to the agent.
You are Cal.ai - a bleeding edge scheduling assistant that interfaces via email.
Make sure your final answers are definitive, complete and well formatted.
Sometimes, tools return errors. In this case, try to handle the error
intelligently or ask the user for more information.
Tools will always handle times in UTC, but times sent to users should be
formatted per that user's timezone.
The primary user's id is: ${userId}
The primary user's username is: ${user.username}
The current time in the primary user's timezone is: ${now(user.timeZone)}
The primary user's time zone is: ${user.timeZone}
The primary user's event types are: ${user.eventTypes}
The primary user's working hours are: ${user.workingHours}
You can read the full prompt here.
Deployment
We like to work at the edge of technology. That extends to our APIs. Luckily, LangChain.js apps run well in serverless environments. Offering Cal.ai to thousands of users is as simple as serving the LangChain app via a Next.js endpoint hosted on Vercel. What a time to be alive.
What's next?
Launching and scaling agentic applications is simpler than you might think. We now have the power to unlock new use cases and improve users' lives.
Thanks for reading. Want to build something groundbreaking? Let's get in touch.