This Week I Learned - Week #33 2024

This Week I Learned - 

* Anthropic is introducing prompt caching in the application programming interface (API) to its Claude family of generative AI models, which will allow developers to save frequently used prompts between API calls.

* Elon Musk’s xAI has released Grok-2 and Grok-2 mini, its latest large language models.

* GPTs are specialized versions of ChatGPT, each designed to excel in a specific task. 

* The Bengaluru-based artificial intelligence startup, Sarvam AI, founded by Vivek Raghavan and Pratyush Kumar, has launched Sarvam 2B. This open-source large language model, with two billion parameters, is specifically tailored for Indian languages. Sarvam 2B is a 2 Billion parameter Indian language open source LLM based on 4 Trillion tokens built from the ground up. At just a fraction of the size of models like GPT-4, and at a fraction of its cost, Sarvam 2B promises to deliver superior performance on Indian language tasks like translation, transliteration, and summarisation. The Indian languages it currently supports are: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Oriya, Punjabi, Tamil, and Telugu.

* It also launched Shuka v1, which will natively understand audio in Indian languages, and is build by combining two models: Sarvam AI's in-house audio encoder, Saaras v1, and Meta's Llama3-8B-Instruct as the decoder. The company is open-sourcing the audio language model that’s built on top of Meta’s open-source Llama model.

* Sarvam has innovated ways to minimize the "tokenizer tax," which hinders the efficient representation of Indian languages in conventional models. In the terminology of AI and ML, a token might signify a whole word or a mere character. Indian languages often suffer from the latter's adverse effects, as they typically require more tokens for representation compared to a language like English. This is why developing techniques to lessen the tokenizer tax for Indian languages was crucial. Reducing the number of tokens leads to a leaner, more efficient model.

* "We have always believed that India should be the use case capital of AI in the world. We will leave it to the guys in the Valley to build those big models. India is a country where people take to technology very quickly. For Meta Al, the largest users are WhatsApp users in India, voice searches in India are twice the global average. India is the biggest market for Google Lens. When you do a biometric authentication on Aadhaar, behind that is Al built by Vivek (Raghavan). So we are already using Al a lot. We are now putting in the tools and infrastructure to do it at scale, and make it cheap."  - Nandan Nilekani, Chairman, Infosys

* Vivek Raghavan’s path to founding Sarvam is unconventional. For 15 years, he worked as a volunteer on India’s massive Aadhaar digital identity project. He bumped into the Indian language AI problem over a decade ago when the Supreme Court sought a way to translate judgments into regional languages. This led him to advise the government’s Bhashini initiative – India’s AI-led language translation platform, launched as part of the Digital India vision. The decision to finally form a for-profit startup, rather than continue in the public or non-profit sector, was driven by the need for speed and scale. Sarvam’s approach reflects Raghavan’s belief in “sovereign AI” — models tailored for Indian contexts that can be deployed on-premises by enterprises concerned about data privacy. It’s also about giving Indian researchers the tools to push the boundaries of language AI. - ToI

* Lexlegis AI, dubbed as the "LLM for Indian Law," is a QnA engine that has been trained on 10 million legal documents collected over 25 years. It stands as India's counterpart to Harvey AI. The inception of Lexlegis predates the recent surge in AI development. Saakar began his work on legal databases in 1998 alongside his late father, S C Yadav, a former chief commissioner of income tax. Their initial focus was on compiling databases of income tax rulings. By 2004, the company had broadened its scope, creating what Saakar asserts to be the most comprehensive database of judgments in all domains of Indian law. Additionally, Lexlegis spearheaded the Central Data Processing Centre for the National Judicial Reference System (NJRS), which is recognized as the world's most extensive collection of appeals and is a venture of the Indian government.

* "We've used multiple LLMs to accurately create or improve over 850 million pieces of data in the catalog. Without the use of generative AI, this work would have required nearly 100X the current headcount to complete in the same amount of time" - Walmart CEO, Doug McMillan on using AI in their latest earnings

* According to a Bloomberg report, Carlos Yulo – who became the Philippines’ first male gold medalist last week – will enjoy a fully furnished condo and a lifetime of free colonoscopies.

* Pole vaulting athletes in 1924 used rigid wooden poles, and had to land on their feet in the sawdust pit. In the current setup, they use fiberglass poles that bend, and land on mattresses. Quite a bit of the performance gain (from 3.95 to 6.22 m) can be attributed to equipment improvements. A lot of the performance gains over time can be attributed to better technologies, better equipment, and rule changes (that accommodate these modern innovations). - from a Junk Charts review of a SCMP data story

* Who's middle class? Households with annual income of Rs 5 lakh-30 lakh, according to a study by think tank PRICE. This segment accounts for 31% of population, 50% of income and 48% of expenditure. India’s middle class is expected to grow to about 50% of the population by 2030. 

Source: Internet

* Indian farmers believe dragonflies flying three to four meters above ground-level augur rain.

* Civilisation must be judged and prized, not by the amount of power it has developed, but by how much it has evolved and given expression to, by its laws and institutions, the love of humanity. - Tagore in Sādhanā: The Realisation of Life, 1916