20240605 QFM017 Machine Intelligence Reading List May 2024

Everything I found interesting about machines behaving intelligently during May 2024

1. Quantum Fax Machine
QFM017: Machine Intelligence
Reading List May 2024
quantumfaxmachine.com
1

2. QFM017: Machine Intelligence
Reading List May 2024
In this month’s edition of the Quantum Fax Machine’s Machine Intelligence Reading List, we explore topics in machine intelligence and generative AI:
We start with Anthropic — Prompt Engineering, an interactive tutorial on crafting effective prompts for Claude language models. It covers basic
techniques to advanced strategies for avoiding hallucinations and optimizing role prompting.
For cutting-edge retrieval augmented generation (RAG), State-of-the-Art Exact Binary Vector Search for RAG in 100 lines of Julia demonstrates
achieving state-of-the-art performance with minimal code, emphasizing efficiency and cost benefits of binary vector spaces.
Exploring broader AI implications, Malleable software in the age of LLMs speculates on a future where large language models democratize software
development, potentially empowering everyday users to create and modify software tools.
Regarding safety and ethics, Refusal in LLMs is mediated by a single directive examines how altering specific activation directions can manipulate
refusal behaviour in large language models, highlighting the fragility of safety fine-tuning and the need for robust AI safety mechanisms.
On the industry front, OpenAI’s anticipated challenge to Google with its own search engine is detailed in OpenAI to Challenge Google with Its Own
Search Engine in May. This move could disrupt the search engine market, offering a new paradigm in web search.
Finally, AI Copilots Are Changing How Coding Is Taught explores the integration of generative AI into coding education, discussing how AI tools
reshape curricula, aid problem-solving, and focus on higher-level thinking despite challenges like AI hallucinations and ethical considerations.
Enjoy this month's selection and, as always, the Quantum Fax Machine Propeller Hat Key will guide your reading.
Key:
: Mentions technology
: Talks about technology in real-world use cases
: Talks about details of machine intelligence technologies
: Using and working with machine intelligence technologies in software
: Programming new machine intelligence concepts and implementations
Source: Photo by Michael Dziedzic on Unsplash
2

3. Anthropic - Prompt Engineering
(thenameless.net): The Anthropic Prompt
Engineering Interactive Tutorial provides a
step-by-step guide for individuals with a
basic understanding of AI to master the art of
crafting effective prompts for Claude
language models. The course covers essential
techniques from basic prompt structuring to
advanced methods like avoiding
hallucinations and using role prompting,
ensuring participants can build and
troubleshoot prompts for various
applications.
#AI #PromptEngineering #Claude
#TechTutorial #LanguageModels
3

4. State-of-the-Art Exact Binary Vector Search
for RAG in 100 lines of Julia (domluna.com):
The article demonstrates how to achieve
state-of-the-art exact binary vector search
for Retrieval Augmented Generation (RAG)
using only 100 lines of Julia code. It highlights
the benefits of binary vector spaces, such as
reduced server costs and improved in-
memory retrieval feasibility, and provides
benchmarking results showing the efficiency
and speed of this approach compared to
traditional methods.
#BinaryVectorSearch #RAG
#JuliaProgramming #MachineLearning
#DataScience
4

5. Malleable software in the age of LLMs
(geoffreylitt.com): The article explores the
potential impact of large language models
(LLMs) like GPT-4 on software development,
suggesting that soon all computer users may
be able to create and modify small software
tools. This shift could lead to significant
changes in software production and
distribution, enabling users to perform tasks
such as creating one-off scripts and GUIs,
developing custom in-house software, and
extending existing applications without
traditional programming skills.
#LLMs #SoftwareDevelopment
#EndUserProgramming #AI #GPT4
5

6. 5 Big Myths Of AI and Machine Learning
Debunked (splunk.com): Artificial
intelligence (AI) and machine learning (ML)
are often misunderstood, leading to
misconceptions about their capabilities and
limitations. This article debunks five
common myths, clarifying that AI and ML
are not the same, are not magic solutions,
do not require advanced degrees to
understand, will not replace all jobs, and do
not need perfect data to be effective.
#AI #MachineLearning #TechMyths
#BusinessInnovation #FutureTech
6

7. Ilya 30u30 AI papers for John Carmack
(arc.net): Ilya Sutskever provided John
Carmack with a curated list of around 30
research papers, suggesting that
mastering this selection would cover 90%
of the essential knowledge in the field of
artificial intelligence today.
#AI #Research #JohnCarmack
#IlyaSutskever #TechReading
7

8. John Carmack's 'Different Path' to Artificial
General Intelligence (dallasinnovates.com):
John Carmack, a renowned tech innovator,
is independently pursuing artificial general
intelligence (AGI) through his startup Keen
Technologies, aiming to achieve human-
level AI by 2030. Carmack believes his
unique approach and extensive background
in gaming, aerospace, and virtual reality
position him well to contribute significantly
to this challenging goal.
#ArtificialIntelligence
#JohnCarmack #TechInnovation #AGI
#FutureTech
8

9. Diffusion Models - Paper Explanation -
Math Explained (youtube.com): This article
provides an in-depth explanation of
diffusion models, covering their
mathematical foundations and operational
principles in generating data. It details how
these models function by iteratively
refining noise into coherent data through a
reverse diffusion process.
#MachineLearning #AI
#DiffusionModels #DataScience
#Mathematics
9

10. I'm Bearish OpenAI
(stovetop.substack.com): The author
argues that OpenAI is unlikely to surpass
Apple, Google, and Meta in software or
hardware and must be the first to achieve
Artificial General Intelligence (AGI) to
remain competitive. However, due to
significant brain drain and a focus on
consumer products that waste valuable
computing resources, the author doubts
OpenAI's ability to reach AGI soon.
#OpenAI #ArtificialIntelligence
#TechIndustry #AGI #BigTech
10

11. I Want Flexible Queries, Not RAG (win-
vector.com): John Mount argues that while
retrieval augmented generation (RAG) attempts
to address the limitations of large language
models (LLMs) by supplementing them with
external references, the true value of LLMs lies in
their flexible natural language query interface
rather than their generative capabilities, which
often produce unreliable results. He illustrates
this with a personal anecdote where a traditional
search engine, despite its flaws, provided a more
satisfactory outcome than a purely generative AI.
#AI #NaturalLanguageProcessing
#MachineLearning #SearchEngines
#GenerativeAI
11

12. Introduction to gpt-4o (openai.com):
GPT-4o integrates text, audio, and visual
inputs into a single model, enhancing its
ability to process and generate multimodal
outputs more cohesively. Currently, the API
supports text and image inputs, with audio
capabilities expected soon.
#AI #MachineLearning
#TechInnovation #OpenAI #GPT4
12

13. Diffusion Models | Paper Explanation |
Math Explained (youtube.com): The video
provides a detailed explanation of diffusion
models, covering their fundamental
concepts, mathematical derivations, and
comparisons with other methods, aiming
to make the topic accessible and
understandable.
#DiffusionModels #MachineLearning
#AI #DeepLearning #MathExplained
13

14. Ways to think about AGI (ben-evans.com):
The article discusses the long-standing
debate over the potential and implications
of Artificial General Intelligence (AGI),
referencing historical predictions, current
developments in AI, and differing expert
opinions on its feasibility and risks. It
highlights how AGI, if achieved, could
fundamentally change automation and
poses significant ethical and existential
considerations.
#AGI #ArtificialIntelligence
#AIResearch #TechDebate
#FutureTech
14

15. AI and problems of scale (ben-evans.com):
The article discusses how generative AI
enables the automation of tasks at a
massive scale, fundamentally changing
their nature and raising new ethical and
practical concerns. It illustrates this
through examples like surveillance, face
recognition, and deepfakes, emphasizing
that these issues are often rooted in
societal and cultural perceptions rather
than the technology itself.
#AI #Automation #Surveillance
#Ethics #Technology
15

16. What I mean when I say that machine
learning in Elixir is production-ready
(cigrainger.com): The article argues that
machine learning in Elixir is not only ready
for production but is highly efficient due to
its integration with the BEAM and OTP,
providing robust, fault-tolerant systems.
The Nx library's architecture and
capabilities, such as multi-stage
compilation and hardware agnosticism,
further enhance Elixir's suitability for
machine learning tasks.
#Elixir #MachineLearning #BEAM #Nx
#ProductionReady
16

17. How LLMs Work, Explained Without Math
(miguelgrinberg.com): The article explains
how Large Language Models (LLMs) like GPT
work without using advanced mathematics,
describing their core functionality of
predicting the next token in a text sequence
based on the input provided. It covers the
concepts of tokenization, probability
prediction, and the use of neural networks
with a focus on the Transformer
architecture and its attention mechanism.
#GenerativeAI #LLMs
#MachineLearning #AIExplained
#Tokenization
17

18. Apple to unveil AI-enabled Safari with iOS
18 & macOS 15 (appleinsider.com): Apple
plans to launch a new AI-powered version
of Safari with iOS 18 and macOS 15,
featuring advanced content blocking, an
"Intelligent Search" tool for summarising
web pages, and a "Web Eraser" for
removing unwanted page elements, all
accessible through an updated user
interface.
#Apple #Safari #AI #iOS18 #macOS15
18

19. alessiodm-drl-zh - Deep Reinforcement
Learning - Zero to Hero (github.com): The
"Deep Reinforcement Learning: Zero to
Hero" course provides a practical
introduction to foundational deep
reinforcement learning algorithms, guiding
participants to write and understand key
algorithms such as DQN, SAC, and PPO,
while training AI to play Atari games and
perform tasks like landing on the Moon.
#DeepReinforcementLearning #AI
#MachineLearning #Python #Coding
19

20. Daniel Dennett 'Where Am I?'
(thereader.mitpress.mit.edu): Daniel
Dennett's essay "Where Am I?" explores the
philosophical and practical implications of a
brain transplant, where his brain is removed
and placed in a vat while his body operates
remotely via radio links. This story delves
into questions of identity, consciousness,
and the nature of the self, illustrating the
complexities and paradoxes that arise when
considering where "self" truly resides.
#Philosophy #Consciousness
#Identity #CognitiveScience
#BrainInAVat
20

21. AlphaFold 3 predicts the structure and
interactions of all of life’s molecules
(blog.google): Google DeepMind and
Isomorphic Labs have introduced AlphaFold 3,
an advanced AI model that predicts the
structure and interactions of a wide range of
biomolecules with unprecedented accuracy,
aiming to transform biological research and
drug discovery. The model is available for free
to scientists via the AlphaFold Server,
enhancing the accessibility and speed of
molecular predictions critical for understanding
diseases and developing new treatments.
#AI #Biotechnology #DrugDiscovery
#MolecularBiology #AlphaFold3
21

22. OpenAI Releases Its First AI-Made Music
Video (techchilli.com): OpenAI has released
its first AI-made music video, "The Hardest
Part," directed by Paul Trillo and produced
using their new text-to-video generator,
Sora, for the artist Washed Out. This launch
aims to attract collaborations with major
studios and media executives while
sparking debate over the impact of AI on
the creative industry.
#OpenAI #AIgenerated #MusicVideo
#TechInnovation
#ArtificialIntelligence
22

23. GenAI in 2024 - Another Decade in One Year?
(medium.com): The article discusses how the
rapid adoption of generative AI technologies
in 2023, driven by tools like ChatGPT, has
accelerated advancements typically seen
over a decade into a single year. It outlines
three major trends for 2024: the evolution of
model orchestration, the transition from
prototyping to production with a focus on
data utilization, and the emphasis on return
on investment (ROI) for AI investments,
predicting continued innovation and practical
applications across various industries.
#GenAI #AI2024 #MachineLearning
#AITrends #TechInnovation
23

24. I will never go back - Ontario family doctor
says new AI notetaking saved her job
(globalnews.ca): Dr. Rosemary Lall, a family
physician in Ontario, credits new AI note-
taking software with saving her job by
significantly reducing the time spent on
administrative tasks, thus restoring her work-
life balance and joy in her profession. The AI
Scribe technology, which transcribes and
organizes patient notes in real time, is being
piloted with promising early results but raises
concerns about data privacy and patient
confidentiality.
#AI #Healthcare #TechInnovation
#Ontario #MedicalTechnology
24

25. Better & Faster Large Language Models via
Multi-token Prediction 2404.19737:
Training large language models to predict
multiple future tokens at once enhances
sample efficiency, leading to improved
performance on generative tasks and
faster inference times without additional
training overhead.
#LanguageModels #AI
#MachineLearning #AIResearch
#TokenPrediction
25

26. Refusal in LLMs is mediated by a single
direction (lesswrong.com): The article discusses
a study finding that refusal behavior in large
language models (LLMs) is controlled by a
single direction in the model's activation space.
By modifying this direction, researchers can
either bypass or induce the refusal of harmful
or harmless instructions, demonstrating the
fragility of safety fine-tuning in open-source
chat models. This method of controlling refusal
behaviors validates interpretability results and
highlights potential vulnerabilities in LLMs'
safety mechanisms.
#AI #MachineLearning #LLM
#TechResearch #CyberSecurity
26

27. OpenAI to Challenge Google with Its Own
Search Engine in May (beebom.com):
OpenAI is reportedly planning to launch a
search engine in May, potentially to
coincide with Google's annual developer
conference, as suggested by a subdomain
and SSL certificate for search.chatgpt.com.
Speculations are based on information
shared by Jimmy Apples and comments by
OpenAI's CEO Sam Altman, who hinted at a
new approach to web search rather than
merely replicating Google.
#OpenAI #Google #SearchEngine #AI
#TechNews
27

28. AI Copilots Are Changing How Coding Is
Taught (spectrum.ieee.org): Generative AI is
reshaping how coding is taught by aiding
students in understanding complex concepts
and problem-solving, while educators strive
to balance this technology with foundational
computer science education. The integration
of AI tools into coursework is evolving the
curriculum to focus more on higher-level
thinking and problem decomposition,
despite challenges like AI hallucinations and
ethical considerations.
#AI #CodingEducation #GenerativeAI
#FutureOfLearning #TechInEducation
28

29. Thank you!
[email protected]
matthewsinclair.com
masto.ai/@matthewsinclair
medium.com/@matthewsinclair
twitter.com/@matthewsinclair
Originally published on
quantumfaxmachine.com
If you’d like to sign up for this content as an
email, click here to join the mailing list.
Image: Photo by Austin Chan on Unsplash
29