Empirical_Methods_in_Management_Study_case.pptx

Study case about a reddit experiment with bots

1. Assignment 2 363-0305-00L | Empirical Methods in Management ETH Zurich | Department of Management, Technology, and Economics

2. Research Approach & Key Findings “ Careful shipping , beautiful packaging , ring perfectly presented as a gift .” “ One ring was perfect , but the other loosened quickly ..” Design Service Quality unique natural aesthetic craftsmanship and attention to detail described as “ beautiful ,” “ refined ,” and “ gift-worthy.” Fast, secure, and well-packaged Helpful and responsive customer support, multilingual support. solid and durable a few note loose clasps or watch drift. balance of design beauty with everyday wearability. «Overall, the reviews show predominantly 5-star and 4-star ratings, reflecting high customer satisfaction with Holzkern’s design and service, and only occasional minor quality concerns.» Department of Management, Technology, and Economics 1

3. LLM Insights & Critical Reflection Critical Reflection Strengths: Efficiency : The LLM processes large, multilingual datasets in minutes. Consistency : It applies the same logic to all data, reducing human bias in coding and interpretation. Pattern Recognition : Excellent at clustering reviews into coherent themes based on semantic similarity, even across languages. Scalability : Enables analysis of hundreds of reviews, something impractical through manual coding. Discovery Power : Can uncover latent patterns or subtle correlations. 2 Department of Management, Technology, and Economics Limitations: Context Blindness: LLMs can miss cultural nuance, sarcasm, or emotional subtleties present in human language. Bias Replication: The model can reflect or amplify biases in training data or majority opinion. Surface Understanding: It identifies what is said, but not always why it’s said- lacking interpretive reasoning. Fake Reviews: It might not realize that a rating could be fake. Need for Human Oversight: Results still require interpretive validation - a researcher’s judgment gives meaning beyond pattern detection. LLM Insights Three key themes identified via multilingual clustering: Design , Service and Quality Revealed strong positive sentiment centred on aesthetics and trust . Identified minor concerns (clasps, durability) through tone variation. Enabled fast, multilingual coding that mirrored manual qualitative results. Takeaway LLM tools enable rapid cross-language synthesis of customer sentiment but still require human interpretation to capture emotional nuance and contextual depth. Design Service Quality Good Rating Design / Service / Quality

4. Holzkern . (2023, August 17). Campania Ring (rose & mother-of-pearl) [Product image]. Holzkern . https://www.holzkern.com/media/catalog/product/1/_/1_q_20230817_campania_ringe_rose_sk_17.jpg Holzkern . (n.d.). Campania Ring – Perlmutt & Gold . Retrieved October 7, 2025, from https://www.holzkern.com/de/campaniaring-perlmutt-gold.html Reviews.io . (n.d.). Customer reviews for Holzkern . Retrieved October 7, 2025, from https://www.reviews.io/company-reviews/store/www.holzkern.com OpenAI. (n.d.). ChatGPT . Retrieved October 7, 2025, from https://chatgpt.com Brandfetch . (n.d.). OpenAI brand assets . Retrieved October 7, 2025, from https://brandfetch.com/openai.com StickPNG . (n.d.). Reviews.io icon [Logo]. Retrieved October 7, 2025, from https://www.stickpng.com/img/icons-logos-emojis/review-platforms-logos/reviewsio-icon 3 Department of Management, Technology, and Economics References