Building AI at Scale: Insights from Rashi Agrawal at the Women in Tech Global Conference
Hello, everyone! I'm Rashi Agrawal, leading engineering teams at Good Leap, where we develop cutting-edge AI products and solutions. Today, I want to take you behind the scenes of what it truly takes to build and scale AI within a company. This journey isn't just about the hype; it's about hard work, strategic planning, and overcoming various challenges. Let's dive in!
The Illusion of AI Magic
At first glance, artificial intelligence can appear almost magical. You type a prompt, and it responds with remarkable insights. Executives love showcasing demos with sleek user interfaces, and the "wow" factor undeniably exists. However, what's often hidden from view is the extensive full-stack engineering required to make AI function at scale. Here are some crucial components:
- Data pipelines
- Evaluations
- Security
- Latency constraints
A seemingly simple chatbot might conceal the hard work of a dedicated team of engineers and a complex platform spanning several layers.
Shifting the Mindset Towards AI
As we step into the era of AI transformation, business leaders are exploring how to leverage AI as an essential factor for change. It's essential to focus not just on technology but on understanding business problems. Instead of asking, "Can we use ChatGPT for this?" ask, "What are our biggest challenges?" This mindset reformation can significantly alter outcomes.
A valuable step we took early on was performing an AI fitness check to ascertain whether we had:
- The right data
- The appropriate infrastructure
- The necessary talent
- The right risk appetite
This objective assessment clarified our goals and helped set realistic expectations.
Importance of Leadership Alignment
Yet, readiness alone isn't sufficient for success. For instance, we initially developed an AI-powered conversation review tool aimed at helping compliance teams manage risks. While the technical aspects were strong, we overlooked a crucial step: securing executive buy-in. Without strategic sponsorship, the project lacked momentum and eventually faded away.
The lesson here is clear. Even when identifying the right business challenges, neglecting any foundational layer, particularly leadership alignment, can prevent AI from scaling effectively. Prototyping is essential, but real transformation requires commitment from the top down.
The Engine Room: Teams and Collaboration
Another critical aspect of AI at scale is team composition and collaboration. We recognized that traditional hiring could slow our progress, leading us to enhance our teams with a blend of:
- Strong engineers
- Trusted contractors
This blended model allowed us to swiftly transition from prototype to production.
Moreover, fostering cross-functional collaboration between product managers, machine learning experts, backend engineers, and UX designers created an environment to transform complex AI models into user-friendly experiences.
Building the Tech Framework
At Good Leap, we approach AI as a system to architect rather than a single model to implement. The efficiency of the entire pipeline is more critical than just prompt engineering. Our architecture encompasses multiple layers, from:
- Raw compute
- Data pipelines
- Model logic
- Knowledge retrieval
- User-facing interfaces
Each layer serves a purpose; omitting any one of them can cause instability at scale.
Facing Challenges in AI Development
Although our journey has been rewarding, numerous challenges arose along the way:
- Privacy: Failing to plan for privacy can stall your launch.
- Latency and Cost: Avoiding high response times and unforeseen costs is essential.
- Team Alignment: Misalignment among teams leads to project failure.
- Enterprise and Risk Management: AI involves ethics, compliance, and bias considerations.
Driving Adoption: Enablement, Governance, and Integration
The true measure of AI's success lies not just in deployment but in its adoption across the organization. Here are three key areas we focused on:
- Enablement: We aimed to make AI accessible through training, documentation, and hackathons to encourage creativity.