Chat bot generative ai presentation majer project submit

Bachelor of art's and science

1. CHATBOT USING GENERATIVE AI
A major project work submitted in partial fulfilment of the requirements
for the degree of
BACHELOR OF COMPUTER SCIENCE
To the
Periyar University, Salem-11
By
S.NANDHAMANI (REG.NO:C23UG156CSC038)
T.PAVITHRA (REG.NO: C23UG156CSC044)
Under the Guidance of
Mrs. S. THANJAMMA M. Sc., B. Ed., M. Phil




PACHAMUTHU COLLEGE OF ARTS AND SCIENCE FOR
WOMEN (AFFILIATED TO PERIYAR UNIVERSITY)
DHARMAPURI-636701
OCTOBER-2025

2. PACHAMUTHU COLLEGE OF ARTS AND SCIENCE FOR
WOMEN (AFFILIATED TO PERIYAR UNIVERSITY)
DHARMAPURI-636701




This is to certify that the Project Work entitled CHATBOT USING GENERATIVE
AI submitted in Partial fulfilment of the requirements of the degree of Bachelor of
Computer Application to the Periyar University , Salem is a record of Bonafide work carried
out by S.NANDHAMANI Reg.No:C23UG156CSC038, T.PAVITHRA, C23UG156CSC044 under
my Supervision and guidance.



Head of the Department Internal Guide



Submitted for Viva-Voce Examinations held on ……………………….



External Examiner Internal Examiner

3. 1

ACKNOWLEDGEMENT

At the outset I offer my humble prayers to the lord and my parents for giving me the
strength and determine not only to this talk but to face all walks of life.
I would like to express our sincere gratitude to the Management of Pachamuthu College
of Arts and Science for Women for providing me with this opportunity and the necessary
facilities to carry out this project successfully.
I thank to our honourable principal Dr.J.THAVAMANI, M.COM., M.Phil., Ph.D. for
providing necessary facilities in carrying out my project work successfully.
My whole hearted and proud gratitude Mrs.R.Manimozhi, Msc., M.phil., M.Ed., NET
Head of the department for this cooperation, guidance and motivation that she has given me
in every step throughout this project.
My whole hearted and proud gratitude goes to the project guide, to Mrs. S. Thanjamma
M. Sc., B. Ed., M. Phil for sub serving me to choose this project enlighten my attitude on this
project.
I thank all the staff members of the department of computer science for giving me
guidance and finally, my gratitude goes to my parents, friends and everyone who have made
direct and indirect contribution to meet this project a grand success.

4. 2

CONTENTS
CHAPTER TITLE

PAGE NO


College Bonafide

Certificate Company
Attendance Certificate
Acknowledgement
Synopsis
Abstract
1 Introduction
1.1 Organization Profile
1.2 System Specification
1.2.1 Hardware Configuration
1.2.2 Software Specification
5-7
2 System Study
2.1 Existing System
2.1.1 Description
2.1.2 Drawbacks
2.2 Proposed System
2.2.1 Description
2.2.2 Features
7-11
3
System Design and Development
3.1 File Design
3.2 Input Design
3.3 Output Design
3.4 Code Design
3.5 Database Design
3.6 Description of Modules
11-
4 Conclusion
5 Bibliography
Appendices

5. 3


A. Data Flow Diagram
B. Table Structures
C. Sample Coding
D. Sample Input
E. Sample Output

6. 4

ABSTRACT
The PDF Question-Answering Chatbot using Generative AI is an advanced
system designed to extract, understand, and generate responses based on the
content of PDF documents. It leverages NLP techniques and large language
models to enable users to query documents in natural language and receive
contextually relevant answers.
The system begins by ingesting PDF files, extracting text using libraries
such as PyMuPDF, PDFPlumber, or OCR-based solutions like Tesseract for
scanned documents. The extracted text undergoes preprocessing, including
tokenization and chunking, to ensure efficient retrieval. Each chunk is then
transformed into vector embeddings using models like OpenAI’s Embeddings or
LLaMA-Embed and stored in a vector database such as FAISS, Pinecone, or
ChromaDB.
When a user submits a query, the chatbot processes the input and retrieves
relevant document sections through semantic search techniques. These sections
are passed to a generative AI model, such as GPT-4, Claude, or Mistral, using a
Retrieval-Augmented Generation (RAG) approach to ensure the generated answer
remains accurate and grounded in the document content. The chatbot then delivers
the response in a conversational manner through an interactive UI, supporting
multi-turn conversations for deeper clarification.
The system is designed to handle both structured and unstructured
documents efficiently, making it highly scalable for applications in legal, research,
business, and enterprise domains. Additionally, it supports multimodal inputs,
integrating image processing with OCR for extracting text from embedded images
within PDFs. By leveraging generative AI, this chatbot significantly enhances
document comprehension and information retrieval, streamlining workflows and
improving accessibility to large volumes of textual data.

7. 5

1. INTRODUCTION
The PDF Question Answering Chatbot powered by GenAI is designed to
revolutionize the way users interact with PDF documents. By leveraging advanced
natural language processing (NLP) and machine learning (ML) technologies, this
chatbot enables users to ask questions about the content of PDF documents,
receive real-time, context-aware responses, and quickly find relevant information.
It eliminates the need for manual document searches, providing a faster and more
efficient solution for working with complex documents. This documentation
introduces a PDF Question Answering Chatbot powered by GenAI—an advanced
AI model designed to intelligently process and respond to user queries about the
content of PDF files. By leveraging cutting-edge natural language processing
(NLP) techniques and the power of GenAI, this system allows users to interact
with PDF documents in an intuitive, conversational manner.
The PDF Question Answering Chatbot integrates with GenAI’s robust
language understanding capabilities, enabling the chatbot to:
• Extract relevant information from PDFs, even with varied formats and
structures.
• Answer specific questions by analyzing the context and content of the
PDF.
• Provide summaries of documents or specific sections based on user input.
• Support multiple languages to cater to a wide range of users.
This system is particularly valuable in scenarios where users need quick access to
specific information from large PDF documents—whether it's extracting legal
clauses, summarizing research papers, or answering questions about product
manuals. The chatbot interface is designed to be easy to use, where users simply
upload their PDF documents and interact with the chatbot via text queries. The
combination of PDF processing and GenAI’s intelligent responses opens up a new
realm of document interaction, offering enhanced efficiency, accessibility, and a
richer user experience. In the following sections, we'll dive deeper into how the
chatbot works, its architecture, and the different features it offers. We'll also
explore the setup process, integration guidelines, and provide examples of real-
world applications.

8. 6

1.1 SYSTEM SPECIFICATION
1.1.1 Hardware Requirements
• Processor: A multi-core processor (Intel i5 or AMD Ryzen 5 or higher)
ensures that the system
• can handle complex AI computations and process large PDF files
efficiently.
• Memory: Minimum of 8GB of RAM is recommended for smooth
operation when handling
• multi-page PDFs and maintaining a responsive chatbot interface.
• Storage: SSD (Solid State Drive) with 50GB of free space is optimal for
storing PDFs, logs,
• and any temporary data generated by the system.
• Network: Reliable internet connection (Broadband or higher) is required
to facilitate model
• inference from cloud services or for interacting with an online
knowledge base.
• Optional GPU: If fine-tuning a GenAI model locally, a modern GPU
(e.g., NVIDIA GTX 1660 or higher) may be required.
1.1.2 Software Requirements
• Operating System: Windows, macOS, or Linux (Ubuntu preferred for
machine learning
• development).
• Programming Languages: Python 3.8+ for backend development, with
libraries like Flask,
• Django, or FastAPI for API development.
• Libraries:
o PDF Parsing: PyMuPDF, PDFMiner, or Apache Tika for text
extraction.
o Machine Learning: TensorFlow or PyTorch for AI model
deployment.
o Natural Language Processing (NLP): Hugging Face
Transformers (for BERT, GPT
o Web Framework: Flask/Django for setting up APIs and
integrating with the frontend.
Database: PostgreSQL, MongoDB, or MySQL for storing document
metadata, user queries, and chatbot logs.

9. 7

1.2 SOFTWARE DISCRIPTION
The PDF Question Answering Chatbot is a state-of-the-art application
designed to facilitate document querying by using GenAI technology (e.g., GPT-
4 or other transformers). Users can upload PDF files, which are parsed for textual
content, and then they can interact with the chatbot, asking natural language
questions about the content.
Key Features:
• Text Extraction: The system extracts and preprocesses text from PDF
documents, making it searchable and analyzable by the GenAI model.
• Real-Time Querying: The chatbot provides answers in real-time,
handling a variety of questions about the content.
• Context-Aware Responses: The chatbot considers the entire document
context when answering questions, ensuring accurate and contextually
relevant responses.
• Scalable Architecture: The system supports large documents, multi-
threaded querying, and can scale horizontally if required.
• OCR-based Systems: Tools like Tesseract can convert scanned
documents into text, but their accuracy is often compromised, especially
with poor-quality scans.
• Keyword Search: Basic PDF readers allow for keyword-based search,
but they cannot understand the context of a query, leading to vague or
irrelevant results.

10. 8

2.SYSTEM STUDY
2.1 Existing System
In the existing system of the PDF Question Answering Chatbot powered
by GenAI, organizations and users typically relied on the following systems to
manage PDF files and extract information:
• Basic PDF Readers: Users manually open PDF files and search for
specific keywords using PDF readers like Adobe Acrobat or Foxit
Reader. These tools provide basic search capabilities but are limited to
exact phrase or word matching and cannot interpret the context or
provide intelligent answers.
• Keyword-Based Search Engines: Some systems employ keyword-based
search engines or full-text search utilities to locate specific words or
phrases within PDF documents. However, these systems lack the ability
to understand the content’s meaning, often returning results that are
irrelevant or require further analysis.
• Manual Review: In industries like law, finance, or healthcare,
professionals often manually scan through entire PDFs, extracting the
necessary information based on their own expertise. This process is
time-consuming, error-prone, and inefficient, especially for large
documents or when handling large volumes of documents.
• OCR-Based Systems: Optical Character Recognition (OCR) technology
is sometimes used to extract text from scanned or image-based PDFs.
While OCR can be effective for basic text extraction, it often struggles
with accuracy, especially for complex layouts, non-standard fonts, or
low-quality scans. Additionally, OCR tools typically don’t offer
advanced query capabilities or context-aware responses. Existing
systems for interacting with PDFs primarily include basic PDF viewers
and Optical Character Recognition (OCR) tools. These systems can
handle simple text-based queries, but they fall short in more advanced
querying and handling large documents.
2.1.2 Drawbacks
• Accuracy Issues: OCR tools often fail to accurately interpret complex
layouts, images, and poor-quality scans.
• Slow Processing: Manual search and OCR are both time-consuming
processes, requiring significant effort to extract relevant information.

11. 9

• No Context-Aware Interaction: Traditional systems do not understand
user queries in context, often providing incomplete or irrelevant
answers.
• Limited Scalability: Handling large documents or multiple user queries
at once is difficult with existing systems.
2.2 Proposed System
The PDF Question Answering Chatbot is designed to enable users to
interact with PDF documents in a more intuitive, efficient, and effective manner.
The system uses GenAI, a state-of-theart AI model, to process the text extracted
from PDFs and provide meaningful answers to user queries.
Users can simply upload a PDF, ask questions about the document’s
content, and receive relevant, accurate responses instantly. The PDF Question
Answering Chatbot powered by GenAI represents an innovative solution to the
limitations of traditional document search and query systems. By leveraging the
capabilities of advanced natural language processing (NLP) and artificial
intelligence (AI), this system transforms how users interact with PDF documents,
offering intelligent, context-aware responses to user queries. The proposed system
aims to streamline workflows, improve productivity, and provide more accurate
and insightful document interactions.
2.2.2 Features
• Real-Time Text Extraction: The system dynamically extracts text from
uploaded PDFs, making it ready for immediate querying.
• Natural Language Querying: Users interact with the chatbot using
natural language, and the system understands the intent behind the query,
answering accurately.
• Context-Aware Responses: Unlike simple keyword searches, the chatbot
generates answers based on the context of the entire document, ensuring
relevancy.
• Multi-Document Support: Users can upload multiple PDFs, and the
system handles crossdocument queries.
• Seamless User Interface: The chatbot offers a simple, intuitive UI,
allowing users to focus on their queries without navigating through
complex menus.

12. 10

2.2.3 Feasibility Study
Technical Feasibility: The system uses proven technologies like GenAI and PDF
parsing tools, which are already mature and reliable. Machine learning models are
accessible through APIs or open-source frameworks.
Operational Feasibility: The system can be integrated into existing document
management systems with minimal operational overhead. It can run on common
servers and requires little maintenance.
Financial Feasibility: While the initial setup costs may include training models
and setting up infrastructure, the system’s ability to automate querying and
improve productivity will result in long-term savings.

13. 11

3. SYSTEM DESIGN AND DEVELOPMENT
3.1 File Design
The File Design outlines how the system organizes, manages, and processes
PDF documents within the application. The primary function of the file design is
to ensure that uploaded PDFs are efficiently stored, easily accessed, and processed
to extract relevant information for user queries. When a user uploads a PDF
document, the system stores it in a predefined directory, where it is saved in its
original format. A folder structure is implemented to categorize and store the PDFs
based on their metadata or user input (e.g., document name, date of upload).
To facilitate quick retrieval, the system extracts the text content from the
PDFs and stores it in a structured format, such as plain text or JSON, where the
text is segmented into logical sections, such as headings, paragraphs, tables, and
lists. This structured text is indexed to allow efficient searching and querying later
during the interaction with the chatbot. The file design also ensures that each
document undergoes a preprocessing step to clean the extracted text by removing
irrelevant content like images, footnotes, and non-text elements.
Furthermore, the system maintains logs and metadata associated with each
PDF (e.g., upload date, document size, file type) to ensure proper document
handling, troubleshooting, and auditing. In addition to text extraction, the system
might use a compression method for storing large documents to optimize disk
space usage. Overall, the file design emphasizes scalability, efficiency, and easy
access to documents, supporting a smooth user experience when querying large
volumes of data.
3.2 Input Design
The Input Design outlines how data is received and processed by the
system. Below are the 10 key steps involved in the input design for the PDF
Question Answering Chatbot:
1.User Authentication:
The user may first need to log in to the system, especially in environments
where multiple users need to access and upload documents. Authentication can be
through a simple username and password, or through an OAuth system for added
security.

14. 12

2.Document Upload Interface:
The system provides a simple drag-and-drop or browse file button that
allows users to upload their PDF documents. The file input is validated to ensure
that only PDF files are accepted.
3.PDF File Validation:
After the document is uploaded, the system checks whether the file is a valid,
readable PDF. It performs basic checks, such as file size, file type, and integrity
of the document.
4.Pre-processing of the PDF:
The uploaded PDF file is pre-processed to extract text content. This step
involves parsing the PDF using a library (e.g., PyPDF2 or pdfplumber) to convert
the raw PDF into machine-readable text.
5.Error Handling for Corrupted Files:
If the system detects any errors or issues while processing the PDF (e.g.,
corrupted file or unsupported format), it will display an error message prompting
the user to upload a different document.
6.Text Segmentation:
Once the text is extracted, it is segmented into logical components such as
paragraphs, headings, and tables. This segmentation enables the system to map
content to specific sections, making it easier to generate more precise answers.
7.Natural Language Input:
The user can input a question in natural language, either via a chat interface
or search bar. The system ensures that the input is properly formatted for further
processing (e.g., removing unnecessary characters, correcting spelling mistakes,
etc.).
8.Query Validation:
The system validates the user's query for any obvious syntax errors or
ambiguous terms. It checks for common mistakes or asks the user for clarification
if needed.
9.Query Parsing and Intent Recognition:
The system uses Natural Language Processing (NLP) techniques to parse
the query, identify intent, and extract key terms and entities that are relevant to the
document. This step is crucial for generating an appropriate response.

15. 13

10.Initiating Response Generation:
After successfully parsing the query, the system generates a response based
on the content of the document. This involves calling the GenAI model or another
NLP system to match the query with the relevant information in the document.


3.3 Output Design
The Output Design outlines how the system delivers the results to the user
in a meaningful and understandable format. Below are the 10 key steps involved
in the output design for the PDF Question Answering Chatbot:
1.Response Generation:
The system generates a response based on the content extracted from the
uploaded PDF document. This response is formulated by interpreting the user’s
query and finding the relevant information within the document.
2.Natural Language Output:
The output is generated in natural language, ensuring that the response is
coherent and easy for the user to understand. The language model is used to format
the answer in a conversational manner.
3.Display of Answers:
The answer is presented directly to the user in the chat interface, typically
as a text response.
The response is concise and relevant to the query, ensuring that users don’t
have to sift through large portions of the document.
4.Highlighting Relevant Text:
When appropriate, the system highlights or underlines the specific text
sections from the document that were used to generate the answer. This helps the
user cross-reference the answer with the original document.
5.Handling Multiple Answers:
If the query is ambiguous or if multiple answers exist within the document,
the system provides a list of possible answers and asks the user to select the most
relevant one. The chatbot might also prompt the user for further clarification.

16. 14

6.Confidence Scores:
The output may include a confidence score or likelihood indicator, which
tells the user how certain the system is about the accuracy of the answer. This
feature helps users assess the reliability of the response.
7.Error Handlingin Output:
If the system cannot find an answer to the user’s query, it displays a polite
error message, such as “Sorry, I couldn’t find any relevant information in the
document.” It may also offer alternative queries or suggestions.
8.Follow-Up Questions:
The system can encourage further interaction by asking the user if they need
more details or if
they have additional questions related to the document. This keeps the
conversation flowing and improves user engagement.
9.Formatting of the Output:
The output response may include different formatting styles such as bold,
italics, bullet points, or numbered lists to enhance readability, especially if the
answer contains key information or references multiple sections of the document.
10.Logging and Output History:
The system logs all outputs and interactions for future reference. Users can
revisit past answers or queries through the system’s history or log feature, ensuring
transparency and accountability for responses given by the chatbot.

17. 15

3.4 Code Design
PDF QUESTION ANSWERING CHATBOT
│── docs/ # Documentation files
│── img/ # Images (used in UI)
│── .env # Environment variables (API keys)
│── faiss_index/ # FAISS database storage
│ ├── index.faiss # FAISS vector index
│── chatapp.py # Streamlit-based UI (as seen in the screenshot)
│── requirements.txt # Dependencies
│── LICENSE # Project license
│── README.md # Project documentation


3.5 System Devlopment
The development of the PDF Question Answering Chatbot using GenAI
(Generative AI) involves several stages, including the design, implementation,
training, and deployment of the system. In this section, we will cover each of the
key development steps, outlining the processes, tools, and methods used to create
the system.
1.System Design
Before diving into the development process, the system design lays the
groundwork for how the PDF Question Answering Chatbot operates. This system
uses Generative AI models to process user queries and generate answers based on
the content from uploaded PDF files. Below are the major components involved
in the system design:
• User Interface (UI): Allows users to upload PDF files and interact with
the chatbot via a chat interface. The UI is designed to be intuitive and
easy to use.
• Backend: The backend handles the processing of uploaded PDFs,
extraction of text, querying, and interfacing with the GenAI model to
generate answers.

18. 16

• PDF Parser: Extracts the text from the uploaded PDF files and converts
them into a structured format for easier querying.
• Natural Language Processing (NLP): This component is responsible for
processing the user’s natural language queries and generating responses
using a trained model.
2.Technology Stack
The following technologies and tools were used for the development of the
PDF Question Answering Chatbot:
Programming Languages: Python is used for the backend development, as it
is wellsuited for machine learning and NLP tasks.
Libraries/Frameworks:
 Transformers (Hugging Face): Used for leveraging pretrained models
like GPT, BERT, or T5 for question answering.
 PyMuPDF (fitz) or PDFMiner: These libraries are used to extract text
from PDFs. o Flask/Django: Web frameworks for creating a REST API
to interact with the frontend and backend.
 TensorFlow/PyTorch: Frameworks for machine learning, used to fine-
tune or implement the GenAI model for question answering.
 SQLite/MySQL: Used for storing metadata about the uploaded PDFs
and user queries.
 Cloud: If deployed in the cloud, tools such as AWS or Google Cloud
are used for hosting the application and handling any large-scale
computational tasks.
3.Data Preprocessing and Text Extraction from PDFs
To enable effective question answering, the system must first extract
meaningful text from the uploaded PDF files. The preprocessing phase involves
several key tasks:
• PDF Upload: The user uploads a PDF document to the system. The backend
API receives this document.
• Text Extraction: Using libraries like PyMuPDF (fitz) or PDFMiner, the PDF
file is parsed, and the text content is extracted. This may involve extracting
the raw text from each page or section of the document.
• Text Cleaning and Structuring: After extraction, the text is cleaned up to
remove irrelevant characters, page numbers, or any other noise from the
document. The text is then divided into sections, paragraphs, or sentences,
making it easier for the questionanswering model to process.

19. 17

• Storage: The structured text content is stored in the system’s database, along
with metadata about the document (e.g., file name, number of pages, etc.).

4.Question Answering Model Development
The heart of the PDF Question Answering Chatbot is the Generative AI
(GenAI) model, which generates answers based on the content of the uploaded
PDF documents. Here's how the model is developed:
 Model Selection: Initially, a pre-trained model such as BERT, T5, or GPT
is selected. These models are fine-tuned on the task of question answering.
Pre-trained models have the advantage of already having knowledge of
language, which allows them to generate reasonable answers even without
extensive task-specific data.
 Model Fine-Tuning:
o Training Data: While pre-trained models have general knowledge,
they need task-specific training. Fine-tuning is done using labeled
data such as the SQuAD (Stanford Question Answering Dataset) or
a custom dataset that contains question-answer pairs related to PDF
content.
o Fine-tuning Process: Fine-tuning is performed by inputting pairs of
questions and context (i.e., text extracted from PDFs) to the model,
allowing it to learn how to predict answers based on the context.
 Evaluation: The model is evaluated using metrics such as F1 score and
Exact Match (EM) score to measure its accuracy in predicting correct
answers.
 Inference: Once the model is trained, it is deployed to the backend, where
it receives queries from the user, processes the question, and generates a
response based on the context derived from the PDF document.
 Backend Development :The backend is responsible for managing user
interactions, handling the PDF files, and querying the question-answering
model.

20. 18

1.API Development:
▪/upload: Accepts PDF uploads from the user.
▪/query: Accepts a question from the user and returns an answer generated
by the AI model.
2.Integration with the AI Model:
o The backend processes user queries and extracts relevant content from
the previously uploaded PDFs.
o The question-answering model is integrated into the backend to process
the queries and return relevant answers. o For each query, the backend
retrieves sections of the document related to the question, feeds them into
the model, and then returns the generated answer.
6. Frontend Development
The frontend provides the user interface through which users interact with the
chatbot. The frontend communicates with the backend API and displays the results
to the user.
1. UI Design: The user interface is designed to allow users to upload PDF
files, ask questions, and view answers easily.
a. A file upload section allows users to upload PDFs.
b. A chat interface allows users to type in their questions and receive
answers.
2. Real-time Interaction: The frontend sends user queries to the backend
via AJAX or WebSocket, and dynamically displays the chatbot's
responses.
3. Error Handling: The frontend handles errors gracefully, informing the
user if the document cannot be processed or if the question cannot be
answered.
7.Testing and Validation
Testing is an essential part of the development process to ensure the PDF
Question Answering Chatbot works as expected. The following types of testing
are conducted:
1. Unit Testing: Individual components, such as the text extraction
function, model inference, and API endpoints, are tested for correctness.
2. Integration Testing: The entire system, including the frontend and
backend, is tested to ensure all components work together seamlessly.

21. 19

3. User Acceptance Testing (UAT): Real users are asked to interact with
the system to provide feedback on usability and accuracy. Their
feedback helps fine-tune the user interface and model performance.
4. Performance Testing: The system is tested for scalability to handle
multiple users, large PDF files, and real-time query processing.
8.Deployment
Once the system is fully developed, tested, and validated, it is deployed
to a cloud service or local server for access.
1. Cloud Deployment: The system is deployed on a cloud platform (e.g.,
AWS, Google Cloud, or Azure) for scalability and high availability.
2. Server Setup: A server is configured to host the application, allowing
users to upload PDFs and interact with the chatbot.
9.System Maintenance and Updates
After deployment, the system undergoes continuous monitoring and
maintenance to ensure that it remains operational and efficient.
1. Model Updates: The question-answering model is periodically retrained
with new data to improve its performance.
2. Bug Fixes: Any issues identified in the system are promptly addressed
through bug fixes and updates.
3. Scalability Improvements: As the number of users grows, the system is
optimized to handle increased traffic and data.

3.6 DESCRIPTION OF MODULES
1. Document Ingestion Module
• Functionality:
• Upload and process PDFs. o Extract text from PDFs using OCR
(if scanned) or text extraction techniques.
• Store extracted text in a structured format (e.g., MongoDB, Vector
Database).
Technologies:
 PyPDF2
 FAISS
 Data Preprocessing Module

22. 20

Functionality:
1. Clean and preprocess text (remove noise, special characters,
stopwords).
2. Chunk large text into manageable sections for better retrieval.
Generate embeddings for document text.
Technologies:
 NLTK
 Sentence Transformers (BERT, OpenAI Embeddings)
3.Embedding and Retrieval Module
Functionality:
• Convert document text into vector embeddings. o Store
embeddings in a vector database for efficient retrieval.
• Implement similarity search for relevant document sections.
Technologies:
• FAISS
4.Generative AI Answering Module
Functionality:
• Accept user queries and retrieve relevant document sections. o
Pass retrieved data to a Generative AI model for response generation.
• Generate human-like responses with citations from the document.
Technologies:
• OpenAI GPT / Llama / Claude / Mistral/Google API o
LangChain for integrating retrieval-augmented generation (RAG)
5.API and Chat Interface Module
Functionality:
• Provide a REST API for interaction. o Develop a web-based or
chatbot UI for users to ask questions.
• Support multi-turn conversations and history tracking.
Technologies:
• Streamlit

23. 21





4.TESTING AND IMPLEMENTATION:
To implement and test the PDF Question-Answering Chatbot, the system
needs to go through several stages, from setting up the environment to performing
comprehensive testing to ensure the chatbot works effectively. Here’s how to
structure the Testing and Implementation section for documentation:
9.1 Implementation Process
The implementation process involves setting up the necessary environment,
developing the components of the system, and deploying it to a cloud service or
local server for testing.
Step 1: Setting Up the Environment
1.Backend Setup:
• Install necessary dependencies:
• pip install flask transformers spacy pypdf2 pdfplumber gunicorn o
Install the AI model dependencies, e.g., Hugging Face Transformers and
PyTorch:
• pip install torch transformers
2.Frontend Setup:
• Set up a React application:
• npx create-react-app pdf-chatbot
• Install additional UI libraries like Material-UI or Bootstrap for better
design and layout: o npm install @mui/material
3.Model Setup:
• If using OpenAI's GPT-4, set up API access (via OpenAI API keys).
• If using Hugging Face transformers (e.g., BERT, T5, or GPT-2), make
sure the model is downloaded and the inference pipeline is properly
configured.

24. 22

4.Cloud Hosting:
• Set up a cloud environment (AWS, GCP, or Azure) to host the backend. For
example, you can use AWS EC2 instances to deploy the backend and a
simple S3 bucket for storing PDFs.
• Ensure security configurations (e.g., SSL certificates) and deploy the
backend using Docker for easier scaling.


Step 2: Developing the Core Components
1.PDF Parsing Module:
• Develop the PDF parsing component using PyMuPDF or PDFplumber to
extract text from uploaded PDFs. o Implement text cleaning to remove
irrelevant sections like headers, footers, or page numbers.
2.NLP Preprocessing:
• Tokenize the extracted text using SpaCy or NLTK to prepare it for further
NLP tasks.
• Implement Named Entity Recognition (NER) to identify key phrases and
entities.
• Use pre-trained transformers (like BERT) to generate sentence embeddings
for semantic search.
3.Generative AI Integration:
• Integrate GPT-4 or another generative model to handle question-answering
tasks. Fine-tune the model if necessary to focus more on the domain of the
PDFs being processed.
• Pass the relevant extracted context to the model for answering the user’s
query. 4. API Layer:
• Develop the backend API using Flask/Django to handle requests from the
frontend. The API will accept the uploaded PDF, process the file, and return
the AI-generated answer to the user’s query. o Implement user
authentication if necessary, to handle different types of users (e.g., public
or registered).

25. 23

5.Frontend Integration:
Set up React to handle user input for both the PDF upload and the question
submission. o Connect the frontend with the backend using Axios or Fetch API to
communicate with the backend for PDF processing and querying.
9.2 Testing Process
Testing is critical to ensure that all components work as expected, and to
handle edge cases like malformed PDFs, unclear queries, or unexpected model
behavior. Below are the key stages in the testing process:
Step 1: Unit Testing
Unit tests focus on the individual components, such as PDF parsing, NLP
preprocessing, and the AI model.
1.PDF Parsing Tests:
 Test different types of PDFs (e.g., scanned vs. text-based) to ensure that
the PDF parsing function works reliably.
 Check if the extracted text retains formatting and content integrity.
Example (Python):
def test_pdf_parsing():
pdf_text = extract_text_from_pdf('test_document.pdf')
assert pdf_text != "" assert len(pdf_text.split()) > 50 #
Ensure content is meaningful
2.NLP Preprocessing Tests:
• Ensure that tokenization is working correctly and that NER is identifying
entities as expected.
Example (Python):
def test_nlp_preprocessing():
text = "The AI model is trained on various datasets."
processed_text = nlp_preprocessing(text)
assert processed_text['tokens'] == ['The', 'AI', 'model', 'is', 'trained', 'on', 'various',
'datasets'] assert 'AI' in processed_text['entities'
]

26. 24

3.Generative AI Tests:
• Test the chatbot with sample queries to ensure it generates contextually
accurate responses. Example (Python):
def test_answer_generation():
query = "What is the purpose of AI?"
context = "AI refers to systems that mimic human intelligence to perform tasks."
answer = generate_answer(query, context) assert "intelligence" in answer


Step 2: Integration Testing
Integration testing ensures that different parts of the system (PDF parsing,
AI generation, frontend-backend communication) work together as expected.
1. PDF Upload and AI Interaction:
 Test the entire flow: uploading a PDF, parsing it, generating embeddings,
and providing an answer.
 Simulate multiple user queries on the same document and check if the
answers remain consistent.
2. Frontend-Backend Communication:
 Verify that the React frontend can successfully communicate with the
Flask/Django backend. o Check if the PDF upload process works
without errors and if the backend is able to return the correct answers.
Step 3: End-to-End Testing
End-to-end testing ensures that the entire system, from user interaction to
answer generation, functions correctly.
1.User Interaction:
 Upload different types of PDFs (e.g., legal documents, academic papers,
business reports) and test how well the model answers questions.
 Test the chatbot with common questions (e.g., "What is the summary of
the document?") to check for reliability

27. 25

2.Performance Testing:
Test the chatbot with large PDFs and measure how long it takes to process
the document and generate answers. o Identify any bottlenecks in the system, such
as long PDF processing times or slow query responses.
3.Usability Testing:
Perform usability testing with real users to evaluate the user interface (UI)
and user experience (UX). o Gather feedback on how easy it is to upload
PDFs, ask questions, and interpret responses.
Step 4: Load Testing and Scalability
Once the system is stable, you should perform load testing to evaluate its
performance under heavy traffic.
1.Simulate Multiple Users:
• Use tools like Apache JMeter or Locust to simulate a large number of users
querying the chatbot at once.
• Measure response times and server performance.
2.Test Scalability:
• Test the scalability of your backend by deploying multiple instances behind
a load balancer to ensure the system can handle traffic spikes.

28. 26

9.3 Deployment & Maintenance
After thorough testing, the chatbot can be deployed for production use.
Here's how to approach deployment:
1.Containerization (Docker):
• Containerize the backend application using Docker, ensuring consistency
across environments.
• docker build -t pdf-chatbot .
• docker run -p 5000:5000 pdf-chatbot
2.CI/CD Pipeline:
• Set up a continuous integration and deployment pipeline using GitHub
Actions, Jenkins, or GitLab CI to automate the deployment process.
3.Monitoring and Logging:
• Implement monitoring tools like Prometheus and Grafana to monitor
application performance and health.
• Use ELK Stack (Elasticsearch, Logstash, Kibana) or CloudWatch to log and
track errors, issues, or performance metrics.
4.Regular Model Updates:
• Update the generative AI model periodically to improve accuracy and
efficiency. Retrain the model with new documents or fine-tune it for better
performance in specific domains.

29. 27

5.CONCLUSION
The PDF Question-Answering Chatbot system leverages advanced
Generative AI models and Natural Language Processing (NLP) techniques to
enable seamless interaction with PDF documents. By integrating tools like
PyMuPDF and PDFplumber, the system extracts text from PDFs while preserving
important content, which is then preprocessed using NLP methods like
tokenization and Named Entity Recognition (NER). The heart of the chatbot lies
in its ability to process user queries through Generative AI models such as GPT-
4, which understand context and generate precise, relevant responses.
The system is designed to handle diverse document types, from simple text-
based PDFs to more complex, multi-column documents, ensuring flexibility and
reliability. Through a robust frontend interface built with React, users can easily
upload PDFs and interact with the chatbot, receiving answers that are directly
derived from the content of the document. Additionally, the backend is structured
to handle file uploads, process queries, and return responses in a highly efficient
manner.
Extensive testing throughout the development cycle ensures that the system
is both reliable and accurate, while performance optimizations make it suitable for
large documents and high volumes of queries. Future enhancements, such as
multilingual support, voice interaction, and real-time collaboration, promise to
further expand the system’s capabilities. This chatbot represents a significant leap
in the accessibility of information from documents, providing a cutting-edge
solution for users across various industries.

30. 28

6.BIBLIOGRAPHY
BOOKS:
1.Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
• A comprehensive book that covers the foundations of deep learning
techniques, including neural networks, which are central to understanding
the models used in the chatbot.
2.Jurafsky, D., & Martin, J. H. (2021). Speech and Language Processing (3rd ed.).
Pearson.
• This book provides an in-depth understanding of natural language
processing (NLP) and computational linguistics, foundational for
developing the NLP techniques used in the chatbot.
3.Chollet, F. (2018). Deep Learning with Python. Manning Publications.
• A practical guide to deep learning using Python, with clear explanations of
using Keras and TensorFlow for implementing machine learning models,
which is useful for understanding how the Generative AI models function.
4.Raghu, A., & Radhakrishnan, A. (2020). Transformers for Natural Language
Processing. Packt Publishing.
• This book focuses specifically on transformer models, including BERT and
GPT, and details how these architectures can be applied in NLP tasks, such
as question answering.
5.Manning, C. D., & Schütze, H. (1999). Foundations of Statistical Natural
Language Processing. MIT Press.
• A classic text that explains the foundations of statistical NLP, including
part-ofspeech tagging, dependency parsing, and other techniques used for
processing text.

31. 29


WEBSITE REFERENCES:
1. Hugging Face Transformers Documentation. (2021). Transformers: State-of-the-art
Natural Language Processing.
o https://huggingface.co/transformers/
The Hugging Face library provides tools for leveraging transformer-based
models, like BERT and GPT, and was used in the chatbot for query processing
and generating responses.
2. OpenAI GPT-4 API Documentation. (2021). API for GPT-4 and other Models.
o https://beta.openai.com/docs/
This website provides detailed information about the GPT-4 API, which was
integrated into the chatbot to generate context-based answers for user queries.
3. PyMuPDF Documentation. (2021). PyMuPDF: A Python Binding for MuPDF, an
Open-Source PDF and XPS Viewer.
o https://pypi.org/project/PyMuPDF/
PyMuPDF is a library used in the project for extracting text and images from
PDF files.
4. PDFplumber Documentation. (2021). Extracting Data from PDFs.
o https://github.com/jsvine/pdfplumber
PDFplumber is another tool utilized for extracting structured data, such as
tables, from PDF files.
5. SpaCy Documentation. (2021). Industrial-Strength Natural Language Processing.
o https://spacy.io/
SpaCy is an open-source library for advanced NLP tasks like tokenization,
partof-speech tagging, and named entity recognition, all of which were essential
for preprocessing text extracted from PDFs.
6. Flask Documentation. (2021). A micro web framework for Python.
o https://flask.palletsprojects.com/
Flask is used for building the backend web service that handles API requests
and integrates with the Generative AI model for question answering.
7. Docker Documentation. (2021). Containerization and Application Deployment.
o https://docs.docker.com/

32. 30

Docker was used for containerizing the backend application, ensuring it is
portable and scalable across different environments.
8. AWS Documentation. (2021). Cloud Computing Services and Infrastructure.
o https://aws.amazon.com/documentation/
Amazon Web Services (AWS) provides cloud-based hosting and infrastructure,
which was used to deploy and scale the chatbot system.
9. Locust Documentation. (2021). Performance Testing and Load Simulation Tool.
o https://locust.io/
Locust was employed to simulate user load and test the performance of the
system under heavy usage.
10. Jupyter Notebooks Documentation. (2021). Interactive Development and Data
Analysis Environment.
o https://jupyter.org/
Jupyter Notebooks were used during the development phase to experiment
with AI models and prototype different solutions in an interactive environment.

33. 31

APPENDICES

A.DATA FLOW DIAGRAM
+------------------+ +------------------+ +------------------+
| User Query | ----> | Query Processor | ----> | Document Search |
+------------------+ +------------------+ +------------------+
| |
v v
+------------------+ +------------------+
| Embedding DB | | PDF Storage |
+------------------+ +------------------+
| |
v v
+-------------------+ +------------------+
| GenAI Model (LLM) | ----> | Response Engine |
+-------------------+ +------------------+
|
v
+------------------+
| Response API |
+------------------+
|
v
| User Interface |

34. 32

B.TABLE STRUCTURE:
1.pdf_documents Table (Stores uploaded PDFs)
Column Name Data Type Description
id UUID / INT Unique ID for each document
file_name TEXT Name of the uploaded PDF
file_path TEXT Path/URL of the stored file
uploaded_at TIMESTAMP Timestamp when PDF was uploaded
status ENUM(TEXT) Status (processed, pending, failed)

2. document_text Table (Stores extracted text from PDFs)
Column Name Data Type Description
id UUID / INT Unique ID for extracted text chunk
pdf_id UUID / INT Foreign key referencing pdf_documents.id
page_number INT Page number of the extracted text
text_chunk TEXT Chunked text for retrieval
created_at TIMESTAMP Timestamp when text was extracted

3. embeddings Table (Stores vector embeddings of document text)
Column Name Data Type Description
id UUID / INT Unique ID for embedding
pdf_id UUID / INT Foreign key referencing pdf_documents.id
chunk_id UUID / INT Foreign key referencing document_text.id
embedding VECTOR Vector representation of the text
created_at TIMESTAMP Timestamp when embedding was generated

35. 33

4. user_queries Table (Stores user queries)
Column Name Data Type Description
id UUID / INT Unique ID for the query
user_id UUID / INT ID of the user (if authentication is used)
query_text TEXT The question asked by the user
timestamp TIMESTAMP Time of the query submission

5. responses Table (Stores AI-generated responses)
Column Name Data Type Description
id UUID / INT Unique ID for the response
query_id UUID / INT Foreign key referencing user_queries.id
response_text TEXT AI-generated answer
source_pdf_id UUID / INT PDF source reference
retrieved_chunks JSON Text chunks used for answer generation
timestamp TIMESTAMP Time when the response was generated

6. chat_history Table (Stores user chat interactions)
Column Name Data Type Description
id UUID / INT Unique ID for chat entry
user_id UUID / INT Foreign key referencing user (if applicable)
query_id UUID / INT Foreign key referencing user_queries.id
response_id UUID / INT Foreign key referencing responses.id
timestamp TIMESTAMP Time of the interaction

36. 34

C.SAMPLE CODING:
import streamlit as st from
PyPDF2 import PdfReader
from langchain.text_splitter import RecursiveCharacterTextSplitter import
os
from langchain_google_genai import GoogleGenerativeAIEmbeddings
import google.generativeai as genai from langchain.vectorstores
import FAISS
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain.chains.question_answering import
load_qa_chain from langchain.prompts import PromptTemplate
from dotenv import load_dotenv import os
from dotenv import load_dotenv

load_dotenv()
print(os.getenv("GOOGLE_API_KEY")) # Check if the key is loaded

load_dotenv()
# os.getenv("GOOGLE_API_KEY")
genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))
api_key='AIzaSyCe3kRjMkHy7GZbyoh7Eg8QNL1-dCS9S_s' def
get_pdf_text(pdf_docs):
text="" for pdf in
pdf_docs:

37. 35

pdf_reader= PdfReader(pdf)
for page in pdf_reader.pages:
text+= page.extract_text() return
text


def get_text_chunks(text):
text_splitter = RecursiveCharacterTextSplitter(chunk_size=50000,
chunk_overlap=1000)
chunks = text_splitter.split_text(text)
return chunks

def get_vector_store(text_chunks):
embeddings = GoogleGenerativeAIEmbeddings(model =
"models/textembedding-004")
vector_store = FAISS.from_texts(text_chunks, embedding=embeddings)
vector_store.save_local("faiss_index")

def get_conversational_chain():

prompt_template = """
Answer the question as detailed as possible from the provided context, make
sure to provide all the details, if the answer is not in
provided context just say, "answer is not available in the context", don't
provide the wrong answer\n\n
Context:\n {context}?\n
Question: \n{question}\n

38. 36


Answer:
"""

model = ChatGoogleGenerativeAI(model="gemini-1.5-pro",
temperature=0.3)

prompt = PromptTemplate(template = prompt_template, input_variables =
["context", "question"])
chain = load_qa_chain(model, chain_type="stuff", prompt=prompt)

return chain


def user_input(user_question):
embeddings = GoogleGenerativeAIEmbeddings(model =
"models/textembedding-004")

new_db = FAISS.load_local("faiss_index", embeddings,
allow_dangerous_deserialization=True)

docs = new_db.similarity_search(user_question)

chain = get_conversational_chain()


response = chain(
{"input_documents":docs, "question": user_question}

39. 37

, return_only_outputs=True)

print(response)
st.write("Reply: ", response["output_text"])



def main():
st.set_page_config("Multi PDF Chatbot", page_icon = ":scroll:")
st.header("Multi-PDF's - Chat Agent ")
user_question = st.text_input("Ask a Question from the PDF Files uploaded
..
")

if user_question:
user_input(user_question)

with st.sidebar:

st.image("img/Robot.jpg")
st.write("---")

st.title(" PDF File's Section")
pdf_docs = st.file_uploader("Upload your PDF Files & \n Click on the
Submit & Process Button ", accept_multiple_files=True) if
st.button("Submit & Process"): with st.spinner("Processing..."): #

40. 38

user friendly message. raw_text = get_pdf_text(pdf_docs) # get
the pdf text text_chunks = get_text_chunks(raw_text) # get the
text chunks get_vector_store(text_chunks) # create vector store
st.success("Done")

st.write("---")
st.image("img.jpg")
st.write("AI App created by @ SHOBIKA) # add this line to display the
image
)

if __name__ == "__main__":
main()
D.SAMPLE INPUT:

41. 39

E.SAMPLE OUTPUT: