[DSC DACH 24] Can AI make e-commerce and online marketplaces become more human - Mladen Fernezir

It might seem counterintuitive, but modern technologies (like AI, speech recognition, etc) are set to make online shopping more human and intuitive. In this presentation, Velebit AI will demonstrate concrete use cases and existing tools for how these technologies can improve how we interact with e-commerce platforms and online marketplaces. Imagine a shopping experience without tedious browsing and scrolling, but with a personal assistant that understands your voice descriptions of the items you want and recommends matching items or items that fit your style or body type. These advancements not only simplify the buying and selling process but also transform customer support, making online shopping more enjoyable and user-friendly.

1. Can AI make e-commerce and online marketplaces more human? Mladen Fernežir Co-founder & Lead Data Scientist www.velebit.ai

2. Today’s agenda 01 About Velebit AI 02 Current e-commerce vs the future 03 Matching buyers & sellers - use cases Buyers - SEARCH Sellers - PRODUCT LISTING 04 Conclusion

3. AI focused agency 10+ years in Machine Learning, Deep Learning and Data Science Research and data driven approach Our services: AI & ML Consultancy Data & ML Engineering Custom AI solutions Pre-built models & API for e-commerce & online marketplaces About Velebit AI We have built and deployed high-impact AI systems across multiple industries

4. CURRENT E-COMMERCE CHALLENGES Difficulty finding product you like Non-intuitive shopping experience Lack of personalized recommendations Limited customer engagement AI-DRIVEN E-COMMERCE BENEFITS Efficient product discovery User-friendly interfaces Personalized shopping journey Interactive experience Current e-commerce vs the future

5. Matching buyers & sellers BUYERS - SEARCH SELLERS - PRODUCT LISTING PRODUCT DISCOVERY

6. Matching buyers & sellers BUYERS - SEARCH SELLERS - PRODUCT LISTING PRODUCT DISCOVERY

7. The evolution of search in e-commerce Keyword based search 01 The accuracy of search results depended mainly on the exact match of text based queries Contextual and semantic search 02 With advancements in NLP, search engines began to understand the context and intent behind search queries Multimodal AI search 03 Understanding and integration of text, images, videos together with LLM -driven interactions

8. Poor search experiences on e-commerce looks like: Lack of autocorrect or ability to detect misspelled words Inability to process broad search queries Poor or absent understanding of user intent Lack of personalized recommendations Lack of filters (customers cannot refine search results) Irrelevant or unrelated search results (especially for broad queries)

9. How AI tackles buyers’ challenges?

10. Multimodal AI Typical elements: VECTOR ARCHITECTURE HYBRID SEARCH TRANSFORMER MODELS CNNs API INTEGRATION https://www.unite.ai/googles-multimodal-ai-gemini-a-technical-deep-dive/

11. CO-Slides_EN_4.1_16by9.pptx AI solutions for e-commerce search Search by image ( Visual search ) AI fashion search Interactive chat-based search Outfits finder that match your face color type Virtual try-on search Real-time object detection search & categorization 01 02 03 04 05 06

12. Allows buyers to search by taking a photo or uploading it Search results are visually similar items We developed our 1st solution of this type back in 2016. Custom-trained Convolutional Network with many tweaks to extract good image descriptors We used binary vectors for fast retrieval 01 Search by image (Visual search)

13. AI model that understands both text and images simultaneously Text-based image search - search within images using queries - matches images with those features, without keywords mentioned in the title/description We convert the text and image separately into vectors The two vector spaces are mutually aligned, so we can make meaningful comparisons and retrieval 02 AI fashion search

14. Conversational search with the help of a chatbot S uggests attributes customers didn't specify (size, color, etc.) Overcomes misspelled keywords We use an LLM to parse user inputs to create the final search query iteratively We then use multimodal search capabilities as in the AI fashion search use-case to retrieve the relevant images 03 Interactive chat-based search

15. Analyzes user's facial features (skin tone, eye & hair color) to recommend outfits that best complement their natural features Provides palette for user’s color type Style matching is a popular feature in the fashion industry Multiple deep learning algorithms, such as metric learning , to make similar styles have similar vector representations 04 Outfits finder that match your face color type https://styledna.ai/color-analysis

16. Allows customers to digitally "try on" clothes & visualize how the brand's clothes will look on them Provides accurate size recommendations and style outfits Multiple modern generative AI techniques, such as Generative Adversarial Networks and Stable Diffusion 05 Virtual try-on https://shopexp.io/

17. By pointing a camera at an item, it’s instantly detected & recognized SEARCH - finds visually similar items on e-commerce sites instantly CATEGORIZATION - instantly categorizes an item, provides price estimate & other info for selling on marketplaces Multiple machine learning models: object detection, categorization, image similarity, and price prediction modeling 06 Real-time object detection search & categorization

18. Matching buyers & sellers BUYERS - SEARCH SELLERS - PRODUCT LISTING PRODUCT DISCOVERY

19. CO-Slides_EN_4.1_16by9.pptx AI solutions for product listing on e-commerce & marketplaces Adding attributes & product tagging Generating product descriptions Accurate product categorization Video generation based on product images Duplicate detection 01 02 03 04 05

20. AI automatically tags and labels product photos by attributes such as color, fabric pattern, brand , etc. Eliminates manual tagging Accelerates product posting Ensures consistent and accurate tagging across all products 01 Adding attributes & product tagging ATTRIBUTES Category Color Sleeve length Material Pattern Neckline Style Blouse Pink Long Cotton Striped V-neck Casual VELEBIT AI Color detection API Fabric Pattern detection API

21. AI generated product descriptions LLMs formulate compelling yet informative product descriptions Improves overall quality of descriptions ensuring consistency on e-commerce & online marketplaces 02 Generating product descriptions Shopify Magic

22. Identifies and suggests the most fitting category for each item S implifies the product posting process - for customers adding a new product on a marketplace or an administrator managing e-commerce inventory Consistent product taxonomy Based on both images & text 03 Accurate product categorization

23. Generates engaging videos from static product images (URLs), showcasing products from multiple angles High-quality video content that can be used across social media, websites and advertising campaigns 04 Video generation based on product images https://boolv.video/feature/product-to-video

24. Employs visual similarity analysis to assess how similar two images are Identifies duplicate products if imported from multiple sources Can identify various duplicates: identical items and listings near identical duplicates watermarked images semantically similar text diverse shots of the same items/properties 05 Duplicate detection

25. Conclusion

26. Conclusion - AI humanizing e-shopping experience AI IMPACT on e-commerce & marketplaces INTUITIVE SEARCH Delivers relevant search results for all queries, including broad searches INTERACTIVE SHOPPING Enables chat-based search to find desired items, making online shopping more engaging PERSONALIZATION Provides tailored and personalized search results & smart recommendations IMPROVED UX Optimized search process improves user experience which shortens the path to purchase and conversions

27. Thank you Mladen Fernežir , Co-founder & Lead Data Scientist Velebit Artificial Intelligence LLC Zagreb, Croatia e-mail: [email protected] web: https://www.velebit.ai/