Beyond Thumbs Up/Down: Using AI to Analyze Movie Reviews

This project explores sentiment analysis, a technique used to understand emotions expressed in text. We delve into the world of movie reviews, applying sentiment analysis techniques to uncover audience sentiment towards various films. This can provide valuable insights for filmmakers, studios, and moviegoers alike. For more analysis and artificial intelligence related content visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/

1. SENTIMENT ANALYSIS ON MOVIE REVIEWS PRESENTED BY: MANSI CHOUDHARY

2. INDEX 01 02 03 04 05 10 DATA COLLECTION DATA PREPROCESSING FEATURE EXTRACTION MODEL SELECTION MODEL TRAINING: FINE-TUNING AND ITERATION 09 08 07 06 CONCLUSION INTERPRETABILITY ERROR ANALYSIS MODEL EVALUATION

3. Data Collection For this project, I selected the IBDM dataset from the website Kaggle Link for dataset

4. DATA PREPROCESSING

5. HTML tags removed All uppercase word converted into lower case Non-Alphanumeric Characters removed Extra Whitespaces removed Tokenization done Stemming used instead of Lemmatization because Lemmatization take more run time Duplicate values removed

6. FEATURE EXTRACTION Term Frequency-Inverse Document Frequency (TF-IDF) is a statistical measure used in natural language processing (NLP) and information retrieval to evaluate the importance of a word in a document relative to a collection of documents, typically a corpus. TF-IDF combines two components: Term Frequency (TF) and Inverse Document Frequency (IDF). Term Frequency-Inverse Document Frequency (TF-IDF) is used

7. MODEL SELECTION Model preformed: LSTM NAIVE BAYE’S RANDOM FOREST SVM

8. MODEL TRAINING Split the data

9. MODEL EVALUATION Approximately all models got same result Accuracy: Proportion of correctly classified instances among all instances. Precision: Proportion of true positive predictions among all positive predictions. Recall: Proportion of true positive predictions among all actual positive instances. F1-score: Harmonic mean of precision and recall, providing a balance between them. ROC-AUC: Area under the Receiver Operating Characteristic (ROC) curve, measuring the model's ability to discriminate between positive and negative instances.

10. ERROR ANALYSIS Approximately all models got same result,tried to resolve but not got solutions When i was converting X_train, X_test into array the session was crashing so i minimized the number of inputs and output while training this maybe the reason output efficiency is not good

11. INTERPRETABILITY Visualization techniques such as bar plots or count plot is used WordCloud Used

12. Most frequent positive words

13. Most frequent negative words

14. FINE-TUNING AND ITERATION Selected Random Forest Model and done fine tuning

15. CONCLUSION Random Forest run well given expected outputs

16. Future expectations: Instead of TFID Word2Vec can be used More models can be used like CNN, Logistic Regression Instead Of Stemming , Lemmatization could be used