Hello and welcome to the third episode of the AI Report. We aim to inform you with trends in the world of AI— from research to products to everyday life — and throw light on the latest trends. Please subscribe for actionable insights and share this newsletter on social media.
Trends in AI
No-gradient Optimization
Simulated annealing and other such methods were popular long back for optimization. Such methods are no-gradient methods. For the past decade or longer, gradient-based methods have been incredibly popular (all the neural network stuff). Now, we think no-gradient optimization methods will rise in popularity again. LLMs are fairly good at reasoning now and we can start using them to use reasoning to solve optimization problems. The paper linked below called Voyager is a great example of being an early pioneer for this trend.
Long context lengths
Context lengths are increasing. For example Anthropic released 100k context length. A lot of people are now saying this rings a death bell for vector databases and finetuning. Quite the contrary. Such context lengths take extremely long time to compute and not to mention they are expensive. So just from that point of view, it is always better to be able to use vector databases to select appropriate chunks rather than send everything in. Also, finetuning is not going anywhere as well. For some applications, it will make sense to spend $ to finetune, rather than to spend $$$ to use long context windows (in order to encode all the necessary information).
Open-source lags behind
A few weeks back, an internal document at Google, called We have No Moat, And Neither Does OpenAI. A lot of people bought into the argument that open-source is closing in. However, a recent paper breaks this bubble. It argues that LLaMA-derived models that were finetuned on ChatGPT data, while they look good and produce some convincing outputs, if you examine closely, the results are not any good.
The paper basically recommends getting hold of much better pretrained LLMs, which open-source now lacks.
Research in AI
📝 [Paper] Voyager: An Open-Ended Embodied Agent with Large Language Models
Voyager is an AI-powered agent in Minecraft designed for lifelong learning and exploration. It employs an automatic curriculum, a growing skill library for complex behaviors, and a novel prompting mechanism that considers environmental feedback and self-verification. The skills Voyager acquires are extensive, understandable, and buildable, enhancing its abilities and preventing catastrophic forgetting. Outperforming previous models, Voyager shows high proficiency in playing Minecraft, reaching milestones up to 15.3x faster, and it is capable of applying its learned skills to novel tasks in new Minecraft worlds. Amazing application of the emerging area of no-gradient optimization.
📝 [Paper] SLiC-HF: Sequence Likelihood Calibration with Human Feedback
You know about RLHF (Reinforcement Learning from Human Feedback), the technique that made GPT3.5 into ChatGPT (amongst others), that was critical to teach the LLM about human preferences. Can we do RLHF without RL? The paper explores the use of Sequence Likelihood Calibration (SLiC) as a method for learning from human preferences in language models. The authors demonstrate that SLiC-HF, which utilizes SLiC with human feedback data collected for a different model, improves supervised fine-tuning (SFT) baselines and provides a simpler, easier to implement, and more computationally efficient alternative to past Reinforcement Learning from Human Feedback (RLHF) approaches.
📝 [Paper] Active Retrieval Augmented Generation
This research paper discusses an advanced approach to improving LLMs that sometimes produce factually incorrect information. The new method, named Forward-Looking Active Retrieval augmented generation (FLARE), repeatedly consults external knowledge sources during the text generation process, predicting upcoming sentences and using this prediction to search for relevant information, especially when the model is unsure about its output. Tests have shown that FLARE performs better than or as well as existing methods on multiple complex text generation tasks, indicating its potential effectiveness.
📝 [Paper] Towards Expert-Level Medical Question Answering with Large Language Models
The study introduces Med-PaLM 2, an advanced artificial intelligence model, which combines language model improvements, medical domain fine-tuning, and novel prompting strategies for answering medical questions. This model significantly outperforms its predecessor, Med-PaLM, by scoring 86.5% on the MedQA dataset and approaching or surpassing the previous state-of-the-art in several other medical datasets. Although further validation is required in real-world settings, Med-PaLM 2 is progressing towards a physician-level performance in medical question answering, with physicians preferring its answers over their own in eight out of nine clinical utility categories.
Please also follows us at @the_ai_report on Twitter. Please help us by spreading the word on social media!