Chip has worked as a researcher at Netflix, was a core developer at NVIDIA (building NeMo, NVIDIA’s GenAI framework), and co-founded Claypot AI. She also taught Machine Learning at Stanford University.
Q: How would you define AI Engineer or AI Engineering?
A: It's a shift from more machine learning to more engineering and more product. Previously, building machine learning applications required building your own models with your own data and expertise. Now, you can use direct API calls.
Q: What are typical steps to build an AI application, from choosing a model through using RAG all the way to fine-tuning?
A: Start by understanding what makes a 'good' vs. 'bad' response. Build guidelines for the model, add examples, and evaluate using both automated metrics and human evaluations. Augment the context with documents (RAG pattern). Consider starting with keyword retrieval before jumping into embedding-based retrieval. When fine-tuning, be aware of the new problems that you need to deal with.
Q: What are practical ways for software Engineers to get started building AI applications?
A: Start with prompting and simple solutions before moving to advanced techniques. Data preparation can give a huge performance boost. If you have tried a lot of solutions already, and now you try Gen AI and you think this Gen AI is a solution for you, understand what makes a good response, then add examples.
Q: How did you write this book, how were you able to write a book about such a fast moving industry?
A: When you write a topic you're trying to bet on whether it's going to stay relevant in the future.
Focused on fundamentals that have been around for a while.
Determined if a problem is due to fundamental limitations of AI or current capabilities.
Q: Given that AI engineering moves so quickly, how did you ensure your book stays current?
A: Prioritised progress and future relevance over current trends. Focused on the underlying fundamentals rather than fleeting capabilities. Paid attention to whether advancements are due to fundamental limitations or temporary capabilities of AI, and how quickly capabilities are changing.
Q: What are the most common techniques used when building AI applications, things that a software engineer who's going into building AI applications should know about?
A: Techniques like RAG (Retrieval-Augmented Generation) and fine-tuning are useful. Focus on understanding what constitutes a good response. RAG is a powerful pattern, but don't jump straight to vector search; start with keyword retrieval.
Q: Given this rush to use Gen AI, what are your thoughts on staying focused on the problem?
A: The more we want to not miss out on things, the more things we will miss. Pick a problem that you care about and only focus on things that help you solve that problem. If some news comes out, ask if it helps solve the problem; if not, wait.
Q: When building an AI system, you need to evaluate the output. Why is it difficult to evaluate AI systems, and what are common ways to do that?
A: It's challenging because the smarter AI becomes, the harder it is for humans to evaluate. Use functional correctness to evaluate how well it performs a task. Use AI to evaluate other AI. Comparative evaluations can also be helpful. The quality of the AI judge depends on the underlying model and the prompt, and it is so non-deterministic.
Q: In evaluating AI, how do you prioritise users’ needs and experience?
A: Evaluate an application based on what users want. Go talk to users and look at their interactions. Manual data inspection has a high value-to-effort ratio. Measure what matters. Don't forget human evaluations.
Q: What are some common mistakes you've seen when teams are building AI applications?
A: Using Gen AI when you don't need it. Jumping straight to complex solutions or fancy agent frameworks.
Systematic thinking and problem-solving approaches usually don't change, even when technologies do. Don't use GenAI when you don't need GenAI
Q: Speaking of picking up new technology, as someone learning GenAI, a software engineer who wants to get into AI engineering, what would your recommendation be to learn?
A: Balance project-based learning with structured learning. Be able to stop and ask questions, and sometimes structured learning can help you ask the right questions. Pick a project, complement it with structured learning, and read papers. Tutorial based learning is great, but it is very important to be able to stop and ask questions.
Q: Will AI mean the end of software engineering?
A: AI can automate coding, but it won't fully automate problem-solving. Software engineering is about solving problems, not just the physical act of coding.
Q: What other use cases are you excited about for that AI could bring outside of just coding?
A: Education. Entertainment.
Q: What programming language did you use most when you built AI applications or did ML engineering?
A: Python and JavaScript.
Q: Which one is your favorite LLM model right now and why?
A: I don't really have a favorite.
Q: What are one or two books that you've read and would recommend?
There was a question recently on my CodeProject article "Dynamically generate a MS Word document using HTML & CSS" . The article describes how to generate a Word document programmatically without using any components, by exploiting the formatting features exposed through Office XML and CSS. The questioner wanted to know how to add a custom header and footer and show something like Page X of Y (total pages). Based on what I have tried so far, showing a header and footer using Office XML in MS Word is not as easy as it is in Excel . If you want to keep the code simple & don't need any great fireworks i.e. you will just settle for page numbers in the header or footer then it's a matter of adding a few lines to the original source code. The code can be viewed at this Github Gist Update (1-Sep-2010): To add a custom header & footer, check this new post To add a footer that shows Page number at the bottom right, here are the steps - 1) Add these classes...
Way back in 2004, a project I was working on required a web page to be exported as a Word document (.DOC). Without relying on any components, I utilized the Office XML & HTML technique to implement this feature. I posted my sample on CodeProject to seek feedback. Over the years, I've received some generous comments & feature requests. Many wanted to know how to add a header & footer to the dynamically generated Word document. With the help of the Microsoft Office HTML and XML Reference , I devised a hack to add a lacklustre header and/or footer . Some folks wanted to customize the contents of the header & footer but when they tried with my hack that offered limited functionality, the header/footer text showed up in the document body to their annoyance. This July, an ingenious developer posted a hack that can overcome this problem, in the Comments section of my CodeProject article. His practical workaround was to pack the duplicating header/footer text inside a t...
This Week I Learned - * Meta has acquired Moltbook, the viral "social network for AI agents" where bots (many powered by OpenClaw) interact while humans observe. * Meta acquired Manus, the AI agent startup for approximately $2 billion. * OpenClaw’s creator, Peter Steinberger, has joined OpenAI. * The OpenClaw project has been moved to an independent open-source foundation to ensure it remains free and accessible under its MIT license. * The Batch - - The U.S. military uses AWS to run the unclassified version of Anthropic Claude. - Claude is integrated with Maven Smart System (MSS), a system for targeting and logistics built by Palantir. - Claude/MSS played a role in the January operation that captured Venezuelan president Nicolás Maduro, but the actions in Iran are its first use in “major war operations.” - The Qwen3.5 family of open-weights vision-language models includes impressive larger models as well as a smaller one that outperforms an OpenAI open-weights model 1...
Comments
Post a Comment