Open Source and AI - ByWater Closing Keynote Presentation.pdf

ByWater Solutions, a leader in open-source library software, will discuss the future of open-source AI Models and Retrieval-Augmented Generation (RAGs). Discover how these cutting-edge technologies can transform information access and management in special libraries. Dive into the open-source world, where transparency and collaboration drive innovation, and learn how these can enhance the precision and efficiency of information retrieval.

This session will highlight practical applications and showcase how open-source solutions can empower your library's growth.

1. SLA 2024 - University of Rhode Island
OPEN SOURCING AI AND RAGS

2. Today's
Presentation
DISCUSSION POINTS
Why Open Source
How does Open Source work AI
Process to get started

3. Jessie
Zairo
OPEN SOURCE ADVOCATE
OPEN SOURCE
CONTRIBUTIONS
Koha - ILS
Aspen Discovery
Libki - PC Management
EXPERIENCE
Law Librarian
Technology Trainer
Outreach Librarian

4. What are your concerns
with using Open Source?

6. STRENGTHS
ACCESS
Developments require
Quality Assurance and
release manager testing
and approval.
COMMUNITY
Users that support,
develop, and continually
improves Open Source
Software.
IT STAFF
Work with Support Vendors
that support & maintain
your software, freeing time
for staff.
SECURITY
Potential vulnerabilities
are collaboratively
addressed by a large
community of developers.

7. My thoughts about AI...

9. OPEN SOURCING AI
ACCESSIBLE
access, modify,
and distribute the
code
COLLABORATE
diverse contributors
from around the
world
AVAILABLE
algorithms and
models publicly
available

10. Factors
for
Success
OPEN DATA
Eliminates the barriers put in
place for licensing, allowing
more rapid and open sharing
even if only internally
TRANSPARENCY
Building the trust with users,
allows for oversight, control and
auditing to ensure inclusive
results
INCLUSIVE
Ensure results are free from
unintentional bias or
discrimination, specific tailoring
for your users

11. The
Process
AUTOMATION
How can we empower our frontline
support staff using AI to fast
access to our knowledgebase and
content.
EFFICIENCY
How can we save time using AI to
focus more on the human
connection with our library
partners.
ACCURACY
We want to ensure accuracy by
keeping a human component in the
review process, (do not blindly
trust).

12. Open Source
AI & RAG
tools

13. With LM Studio, you can ...!
-Run LLMs on your laptop, entirely offline" –
-Use models through the in-app Chat UI or an OpenAI
compatible local server# –
-Download any compatible model files from
HuggingFace $ repositories%
-- Discover new & noteworthy LLMs in the app's home
page

14. LLMs FROM Hugging Face
HOW IT WORKS
IN DAILY LIFE
Look at the size of those LLMs, smaller than one would expect

15. HTTPS://GITHUB.COM/LMSTUDIO-
AI/LMSTUDIO.JS

16. FUNCTION
COMPLETELY
OFFLINE
What are the minimum hardware / software requirements?
•Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer
•Windows / Linux PC with a processor that supports AVX2 (typically
newer PCs)
•16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is
recommended
•NVIDIA/AMD GPUs supported

17. DOESN’T TAKE A
SERVER TO RUN
Stats of mac
RESOURCE USAGE
8 GB and 75% of processor

18. CAN LOAD MULTIPLE
MODELS TO COMPARE

19. Hugging Face
GITLAB FOR LLMs
You can see the code behind the scenes

20. PLAY WITH LLM’S IN “SPACES”

21. ANIMATIONS IN
HUGGING FACE
HTTPS://HUGGINGFACE.CO/SPACES/KWAIVGI/LIVEPORTRAIT

23. HAVE CONFIDENTIAL AND
PROPRIETARY
CONVERSATIONS TAKING
PLACE BUT WANT AI TO
TRANSCRIBE?
Building
Offline
Transcripts
HTTPS://WWW.GLADIA.IO/BLOG
/BEST-OPEN-SOURCE-SPEECH-
TO-TEXT-MODELS
There’s Open
Source for
that!

24. WHISPERCONVERT SPEECH TO TEXT

25. STANDS FOR RETRIEVAL
AUGMENTED GENERATION
WHAT’S A
RAG?
YOU CAN USE LMSTUDIO TO
POINT TO THE RAG AND
CONVERSE WITH YOUR
PRIVATE OFFLINE DATA
Repository
of your
information

27. RAG – Retrieval Augmented Generation
https://iwasnothing.medium.com/build-a-local-rag-server-on-macbook-pro-
6b45f86f2ce9
https://community.amd.com/t5/ai/how-to-enable-rag-retrieval-augmented-
generation-on-an-amd-ryzen/ba-p/670670
Resources about RAGS

28. Unlocking the
Future: AI
Innovations in
Libraries
Virtual Conference
September 16 & 17 2024

29. Learn More
SLIDES
SlideShare
CONTACT
[email protected]
LINKEDIN
https://www.linkedin.com/in/jessicazairo/