The AI Report - #1
Hello and welcome to the first inaugural newsletter for AI Report. We aim to inform you with trends in the world of AI— from research to products to everyday life — and throw light on the latest trends. Please subscribe for actionable insights and share this newsletter on social media.
Research in AI
📝 [Paper] Unlimiformer: Long Range Transformers with Unlimited Length Input
CMU authors released a preprint that claims to allow unlimited context length in Transformers. Context length is a critical factor that allows how many tokens the Transformer can attend to in the input, and determines. Having long context-length is very useful, for example, to summarize long documents.
📝 [Paper] A Cookbook of Self-Supervised Learning
Meta released a very nice (and long) paper on Self-supervised learning, the technology that underpins a lot of the latest AI technology, such as Large Language Models and some of the vision models.
[LLM] Together.xyz Releases a few 3B and 7B LLMs
Together.xyz releases three variants of a 3B LLM and their 7B models are 80% trained. This is going to democratize the LLM adoption quite substantially, given especially that these models can be used for commercial purposes.
World of AI
[News] Crypto.com announces Amy, a chat-bot
Crypto.com has announced the launch of its first-ever AI-powered initiative, Amy, an AI companion designed to help users learn about the crypto industry with real-time information on tokens, projects, price listings, and historical events. It’s based on ChatGPT and is being finetuned on Crypto.com’s data. More info.
[News] Microsoft may charge 10x more for private ChatGPT instances
MS may capitalize around the concerns about data being sent back for training models, and might offer a private ChatGPT instances. The catch is that it could be 10x more expensive.
Tools
[Repo] Jsonformer: Generate JSON from Language Models
This is a very useful tool because a lot of the times, LLMs get it very hard to get the JSON right.
Prompts
Related to the above, but you can also ask ChatGPT to always provide JSON as output:
"You only speak JSON. Do not write text that isn't JSON."