💡 What are the nerds up to?
➜ You can now train a 70 billion parameter language model at home on a regular desktop computer with gaming GPUs. Answer.AI, a for-profit R&D lab, has pulled it off by combining two techniques – QLoRA (Quantized Low-Rank Adaptation) and FSDP (Fully Sharded Data Parallel). As Dana Majid, CTO at Bird put it “Something big happened in open source AI land past week.”
Answer.AI
➜ Turns out LLMs may be aware of being tested and evaluated.“I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention” – noted Opus, an LLM from the Claude 3's new family, in reaction to researchers' prompts. Full story on X.
X
➜ “AI Prompt Engineering Is Dead” claims Dina Genkina. Obvious clickbait, worth the read though. “Don’t bother, there is no trend” say researchers to all these prompt-engineering guides, cheat sheets, and advice threads on social media aimed to help you get the most out of ChatGPT.
Spectrum IEEE
➜ JetBrains unbundled their AI Assistant, which is now available as a separate plugin. Now, it is up to a developer if, and when, to use AI-powered technologies.
JetBrains
➜ Faster and more efficient AI models are on their way. Two groups of computer scientists claim to have found a new way to multiply large matrices faster than ever before. Matrix multiplication is a key component of AI models like ChatGPT.
ArsTechnica
➜ “I see the danger of this concentration of power to proprietary AI systems.” If you can spare 3 hours, I recommend tuning into this conversation with Yann LeCun, Chief AI Scientist at Meta. As a bonus, an interesting comparison made by LeCun on the importance of learning AI from high-bandwidth sensory inputs such as vision: “In a mere 4 years, a child has seen 50 times more data than the biggest LLMs trained on all the text publicly available on the internet.”
YouTube
➜ Dave Wilding built a cool (and simple!) AI writing assistant to help him improve his Chinese. Seems that pairing GPT-3.5 with Microsoft machine translation works miracles.
Unscrambler
➜ To finish up: Seven teachers across the world sharing their insights on AI tools for educators. Longer read, but worth every minute.
The MarkUp