Generative AI: Essentials for Startups - Notes

This session provides an overview of Google's foundational AI technologies – Vertex AI, Gemini models, Model Garden, databases, and storage.

In the early days, there were only Large language models (LLMs). Then, retrieval augmented generation (RAG) emerged and evolved to Al agents for reasoning and orchestration 60+ models in Model Garden.

Large language models (LLMs) can now enable building solutions that previously required research teams, potentially achievable "over a weekend".

Generative AI involves large language models and often retrieval augmented generation (RAG) for information retrieval, leading to more factual and less hallucinated responses.

The current era includes AI agents, which add logic and control to the LLM and RAG stack, empowering them with tools to take actions.

Vertex AI in Google Cloud provides a user interface (Vertex AI Studio) to experiment with models like Gemini and offers a Model Garden with various foundation and partner models.

A generative AI app is essentially an app where generative AI is integrated, whether for content creation or driving backend logic. Examples include LLM chatbots, retrieval augmented generation systems, and prompt chains.

Instead of being overwhelmed by the 200+ Google Cloud services, startups can often build impressive apps by focusing on a few key areas: AI/ML (Vertex AI), compute (e.g., virtual machines, serverless compute), and storage (various data storage options).

A simple stack for a generative AI application could involve Vertex AI as middleware connecting existing databases (like Cloud SQL and BigQuery) to provide more powerful insights.

Thinking in terms of a three-tier architecture (Generative AI tool, Compute resource, Data source) can help demystify new technologies.

Organising the Google Cloud AI toolbox into generative AI tools, compute, and storage can provide a clearer understanding of how to build applications.

Internal employee chatbots can focus on information discovery and management with less stringent requirements around hallucinations compared to customer-facing applications.

Autonomous backend AI agents can enhance processes like product development and code reviews.

Startups don't need to throw away their entire tech stack; generative AI can be integrated as a new building block.

One of the key resources mentioned in the talk is the blog post about 321 real-world gen AI use cases from the world's leading organizations grouped under the following types of Agents:

• Customer Agents 
• Creative Agents
• Code Agents
• Security Agents 
• Employee Agents
• Data Agents 

Best Practices for Building with AI:

• Embrace open source - Explore open-source projects such as LangChain, LangGraph, open-weight models, and the entire OSS ecosystem.
• Combine tools when possible - Use Gemini with BigQuery for large-scale analysis, call Imagen from Gemini for image generation, or integrate Gemini with Google Workspace for document handling.
• Implement modular architectures - Adopt a modular approach to building your AI applications by breaking down tasks into discrete, reusable components.

Evaluation and quality assurance are essential from the beginning, with defined metrics, user testing, and monitoring.