share - Lions, tigers, AI and health misinformation, oh my!.pptx

• Pitfalls and pivots needed to use AI effectively in public health
• Evidence-based strategies to address health misinformation effectively
• Building trust with communities online and offline
• Equipping health professionals to address questions, concerns and health misinformation
• Assessing risk and mitigating harm from adverse health narratives in communities, health workforce and health system

1. TINA D PURNAT Prajna Leadership Fellow and DrPH Student TH Chan School of Public Health, Harvard University Fellow of the Australasian Institute of Digital Health tinapurnat.com Unless credited, images generated by ChatGPT 4o. This presentation  © 2024  by  Tina D Purnat  is licensed under  Attribution- NonCommercial - ShareAlike 4.0 International        

2. How you can use this slide deck Thanks for your interest in this topic. I developed this deck to support public health efforts and have made it available for others to use it as well. I’ve made full effort to acknowledge sources of information and adaptation of slides from other people. You are welcome to adapt the slide deck as per the license below. Please make an effort to properly credit the efforts of others that you use. This presentation  © 2024 by  Tina D Purnat  is licensed under  Attribution- NonCommercial - ShareAlike 4.0 International         You are free to: Share  — copy and redistribute the material in any medium or format Adapt  — remix, transform, and build upon the material The licensor cannot revoke these freedoms as long as you follow the license terms. Under the following terms: Attribution  - You must give  appropriate credit  , provide a link to the license, and  indicate if changes were made  . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. NonCommercial   - You may not use the material for  commercial purposes  . ShareAlike   - If you remix, transform, or build upon the material, you must distribute your contributions under the  same license  as the original. No additional restrictions  - You may not apply legal terms or  technological measures  that legally restrict others from doing anything the license permits. Creator: https://www.linkedin.com/in/tinadpurnat/ Work is published at: https://tinapurnat.com

3. Toto, I have a feeling we’re not in Kansas anymore... 1938 https://www.scientificamerican.com/article/what-taylor-swift-conspiracies-reveal-according-to-science/ 2024 https://www.tnonline.com/20211101/war-of-the-worlds-hoax-broadcast-sparked-panic-83-years-ago/

4. AI could benefit public health communication What is the evidence of its harms? AI could be very harmful to public health communication What are the public health use cases for benefits?

5. AI strengthening public health communication Use cases Tools Automating analysis of social media Talkwalker, Pulsar, Meltwater,... Generating text, images and video ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Testing for readability/understandability ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Draft translations, editing, synthesizing information ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Making content more accessible (e.g. alt text, descriptions) ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Brainstorming/organizational tools ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Offering more tailored content/responses to users (e.g. chatbots) Many commercial offerings ...but all come with limitations and biases.

6. AI eroding public health communication ... all the same tools can be either misinterpreted or misused. Use cases Tools Automating analysis of social media Talkwalker, Pulsar, Meltwater,... Generating text, images and video ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Testing for readability/understandability ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Draft translations, editing, synthesizing information ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Making content more accessible (e.g. alt text, descriptions) ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Brainstorming/organizational tools ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Offering more tailored content/responses to users (e.g. chatbots) Many commercial offerings

7. If a similar dashboard were available for epidemiological surveillance, it would never be used by a health department: data of uncertain provenance needs triangulation, not comparisons geographically non specific measures the wrong thing analytic “black box” The most common response to: How should we track health misinformation? A “magic” social media dashboard!

8. AI tools sometimes spit out irrelevant analyses that are not useful for public health action. Sentiment analysis Reach metrics Hashtag tracking The reason these don’t make sense is because they were built for commercial brand promotion. ...and they analyze data that’s easiest to obtain and analyze See: https://researchworld.com/articles/10-challenges-of-sentiment-analysis-and-how-to-overcome-them-part-1 Generalized top narratives

9. However, there are good examples of public-health and community-focused insights reports https://iris.who.int/bitstream/handle/10665/359144/WER9727-eng-fre.pdf https://www.cdc.gov/vaccines/covid-19/vaccinate-with-confidence.html https://sites.brown.edu/informationfutures/2024/01/23/is-the-2024-election-still-happening/ https://sites.brown.edu/informationfutures/the-information-project-pilot-page/

10. Infodemic insights analysis and reporting Includes: Manual Editable templates Workbook Teaching simulation eLearning modules Annex with emergency, outbreak response and immunization resources

11. Digital anthropology puts the community experience at the center of analysis of the information environment. Rapid community assessment tools can also include information environment components. See US CDC’s RCA toolkit and addenda for adolescents and digital spaces. Other tools that might be helpful https://www.gethealthysmc.org/post/youth-rapid-community-assessment For public health professionals, civil society organizations, and interested community members on how to analyze community stories to foster a healthier information environment https://communitystoriesguide.org/the-guide/ https://www.cdc.gov/vaccines/covid-19/vaccinate-with-confidence/rca-guide/index.html https://liivcenter.org/digital-anthropology-toolkit/ https://www.gethealthysmc.org/post/youth-rapid-community-assessment

12. Working with communities’ expressions of questions, opinions, concerns, and circulating narratives requires ethical care in (AI and other) data collection, analysis and use for health communication. Ensure you have someone with ethics expertise involved in your team Anonymize what you can (data collection, analysis, reporting) Consider discarding data sources where ethical nature of how data was collected is in doubt, such as social listening data from closed messaging networks Emphasize triangulation of data sources and selection of research methods appropriate for the task to ensure best level of evidence, even when working at speed Publish your methods transparently Ensure that accountability and ethical principles are reinforced in policy, especially when working with subcontractors who may be unfamiliar with public health Ensure stronger M&E framework and one that includes perspectives from affected population Ensure that communities where data is extracted from also benefit from hearing how the data was used and how analysis was used to support improving health Perform a Risks, Harms and Benefits Assessment when planning - different health topics have different ethical considerations

13. Ask yourself, when was the last time you changed someone’s mind with a simple message? The most common response to: What messages should I use to promote health guidance? Use a “magic” message bank

14. Example challenge: Encourage people to eat more diverse food groups public health approach social marketing approach AI approach “ChatGPT, give me a message about healthy eating tailored to the African-American community.” Recommendation: Make it cheaper and easier for people to find and purchase fresh fruits and vegetables in this neighborhood. “Get out of the beige routine: eat your rainbow and turbocharge your health!” “Let's come together as a community to embrace healthy eating habits that celebrate our culture and support our well-being. “

15. AI lets you take shortcuts, which can help speed up processes. However, AI is an unreliable source of content and analysis for expert-driven communication.

16. Even if we can do some tasks faster by using technology, this doesn’t let us off the hook that health communications make an implicit promise about quality, accessibility, affordability, and acceptability of health guidance, health services and products. https://substack.com/home/post/p-145099430 https://www.statnews.com/2024/05/09/h5n1-communication-didnt-federal-government-learn-anything-from-covid/ https://www.washingtonpost.com/travel/2024/02/18/air-canada-airline-chatbot-ruling/

17. Health communication must be fully aligned with health service delivery and experience, and health guidance Demand for health information Demand for health services and products Adherence to health guidance PULL – people want something from the system PUSH - the system wants something from the people See: https://www.linkedin.com/pulse/demand-promotion-trust-trouble-information-tina-d-purnat-cw3of/

18. Beware of “easy” solutions to health misinformation and information environment challenges “We fight AI with AI.” “There’s a dashboard for that.” “We can debunk our way out of this.” “If we focus on the top ten misinfo spreaders, we’ll solve this.” “We’ll inoculate against misinformation with empirical truth!” “We’ll hire a social media person for that.” They are as snake oil is to people desperately looking for a cure.

19. It comes down to principles and values of working in public health: do no harm/prevent and reduce harm communicate what is known and what is not known use best available evidence (triangulate) protect the vulnerable evaluate as you implement use multilevel interventions involve the community of focus ...the common denominator is that people don’t apply public health principles to information environment concerns like they would if they were related to an outbreak or tobacco or obesity or environmental health.

20. We should demand and expect better tools and solutions that are evidence-based, multi-level and embedded in public health practice. (c) WHO/Sam Bradd , Drawing Change

21. Key takeaway #1: P roviding credible, accurate health information is the basis for any other health communication intervention. Example: In 20121, only 8% of VA web sites are conforming to Section 503, which is the federal standard for accessibility of web sites for people with disabilities https://www.casey.senate.gov/imo/media/doc/report_on_the_accessibility_of_websites_of_the_department_of_veterans_affairs1.pdf

22. Key takeaway #1: Ensure the basics first web site is up-to date, accessible, and search engine optimized ensure web site is crosslinked from other reputable sources, such as county, state, or jurisdiction-specific web sites people with questions can reach a live human if they need to editorial policy and content moderation policy across social media channels point of contact for the media and for inquiries FAQ section on web site content available in multiple formats and languages SOPs developed on how to respond to and address urgent questions, concerns, information voids and mis/disinformation

23. Key takeaway #2: Addressing health misinformation is not just the health department’s job . Actually, if you try to do this solo, you will fail. https://xkcd.com/386/

24. Only few categories of health misinformation narratives can be directly addressed by a health department Adapted from: FirstDraft https://firstdraftnews.org/wp-content/uploads/2020/11/FirstDraft_Underthesurface_Fullreport_Final.pdf?x21167 Development, provision and access of healthcare services and products Safety, efficacy and necessity of diagnostics, therapeutics and vaccines Political and economic motives Conspiracy theories Liberty and freedom Morality and religion Can be addressed by a health department

25. Don’t assume that the health information that you share will be seen as trusted and credible by your audiences. Instead, partner with organizations and voices that are trusted.

26. https://libguides.sdsu.edu/library-toolkit-addressing-health-misinformation https://www.hsph.harvard.edu/chc/resources/digital-safety-kit/ https://www.aap.org/en/patient-care/media-and-children/center-of-excellence-on-social-media-and-youth-mental-health/ Community organizations that are focused on healthy social and digital environments Factcheckers & media Libraries Science organizations Museums Medical associations Science & health journalists Key takeaway #2: Pick your battles and partner with organizations and influencers where people play, pray, work, study and gather

27. Key takeaway #3: Think like a fire marshal, not like a firefighter, when it comes to addressing health misinformation. Prevention is key.

28. Instead of focusing on mis- and disinformation, focus on low hanging fruit: addressing questions, concerns and information voids Questions Concerns Information voids Misinformation Disinformation More common, easier to address by health departments PREVENTIVE action Less common, harder to address by health departments REACTIVE action See: Managing Infodemics in the 21st Century: https://link.springer.com/book/10.1007/978-3-031-27789-4

29. Focus on what is your control vs what is outside of your control Identify information voids that you can fill Better address questions and concerns Work with community leaders and members to address questions and concerns Assess risk for health information challenges Look ahead to potential events that might cause additional confusion or concerns and plan for it Be proactive in providing messages and answers to questions https://misinforeview.hks.harvard.edu/article/misinformation-reloaded-fears-about-the-impact-of-generative-ai-on-misinformation-are-overblown/ Key takeaway #3: Preventative actions you can take now

30. Key takeaway #4: If you address health misinformation, think beyond communications and social media responses.

31. ‹#› 20 tried and tested strategies to address health misinformation 1 Monitoring and detection of narratives, questions, concerns, and misinformation in the information ecosystem Public health social listening and integrated analysis 2 Science and health communication Rapid content development to address emerging information voids Quality health information materials and resources Science and knowledge translation Debunking and misinformation correction 3 Education & training to build resilience to misinformation Capacity building to identify and address health misinformation Prebunking Improving self-efficacy to adhere to recommended health behaviors Improving ability to develop a more accurate risk perception, and judge benefits of recommended health behaviors 4 Engagement Individual and community-level engagement Psychological support to individuals and communities Health worker level engagement Social media-based strategies Coalition building and digital engagement Digital design for the user experience of information environment 5 Laws, regulations and ethics Cybersecurity laws or cyber-resiliency strategies Public advisory and guidelines with legal mandate/enforcement Legislative ban of misinformation spread or legislation for right to health information Regulation of information and related platforms and ethics Content moderation policies, oversight and implementation Consumer protection laws SUBCATEGORY INTERVENTION CATEGORY See examples in: Purnat T, John O, Pundir P, Ishizumi A, Murthy S, Rajwar E. et al. https://bit.ly/evidencegapmapinfodemic . Open Science Framework. 2022.

32. Key takeaway #5: A checklist for your boss Fix the health system’s digital footprint Introduce new ways of developing and promoting health content, including experimenting with AI responsibly Integrate infodemic management into emergency structures before the next emergency Leverage infodemic insights generation in routine for the highest priority health policy areas Train health care workers on how to communicate more effectively online and offline with people they care for, and to address misinformation. Promote digital, information, media, science and health literacy. Incorporate metrics on information environment indicators into regular reporting. Do a better job of protecting health care workers from online harassment and doxing.

33. Thank you very much! [email protected] Resources for infodemic managers

34. https://issues.org/to-fix-health-misinformation-think-beyond-fact-checking/ Recommended reading and listening: https://issues.org/misunderstanding-misinformation-wardle/ https://www.cjr.org/special_report/truth-pollution-disinformation.php

35. See mini talk at: https://www.youtube.com/watch?v=YnMJ1sOkCUM&list=PLwmB5Aqso7V43SnUrCDOCyw08aBi9ljSA&index=1&t=1s&ab_channel=purnatt Illari, L., Restrepo, N.J. & Johnson, N.F. Rise of post-pandemic resilience across the distrust ecosystem. Sci Rep 13, 15640 (2023). https://doi.org/10.1038/s41598-023-42893-6 Johnson, N.F., Velásquez, N., Restrepo, N.J. et al. The online competition between pro- and anti-vaccination views. Nature 582, 230–233 (2020). https://doi.org/10.1038/s41586-020-2281-1 Social network analysis can show gaps in engagement, and communities that are the “amplifying mediators” of narratives

36. Familiarize yourself with the current scientific debate about social inoculation and prebunking: https://www.conspicuouscognition.com/p/misinformation-poses-a-smaller-threat versus https://www.nature.com/articles/d41586-024-01587-3

37. PUBLIC HEALTH AND THE CHALLENGE OF INFORMATION ENVIRONMENT Examples hijack health-conscious communities Industry marketing and influencers Heathcare professionals undermine evidence-based health advice Trusted tech platform services misdirect and mislead people searching for health information Communities with multiple vulnerabilities may make health choices based on low quality information. Well-funded and well-organized “anti” movement counters health advice and politicizes health. Health fraud, scams and deceptive marketing exploit vulnerabilities within the information environment When health workers are harassed, doxxed and attacked this is the alarm that something has gone very wrong between community and health system For examples of challenges and possible actions, see: tinapurnat.com/blog

38. https://www.kff.org/coronavirus-covid-19/poll-finding/kff-health-misinformation-tracking-poll-pilot/

39. https://pubmed.ncbi.nlm.nih.gov/36531901/ https://www.cdc.gov/vaccines/covid-19/hcp/engaging-patients.html

40. https://www.mitre.org/news-insights/impact-story/whats-all-hipe-mitre-its-building-community-resilience https://meedan.com/programs/digital-health-lab

41. Infodemic monitoring and management are integrated into updates of country pandemic plans and global emergency architecture…. https://www.who.int/publications/m/item/preparedness-and-resilience-for-emerging-threats-module-1-planning-for-respiratory-pathogen-pandemics-version-1 https://www.who.int/publications/m/item/strengthening-the-global-architecture-for-health-emergency-prevention--preparedness--response-and-resilience

42. … into planning for pandemic influenza preparedness and vaccine demand promotion ... https://www.who.int/publications/i/item/9789240070141 https://apps.who.int/iris/handle/10665/372248 https://demandhub.org/operational-framework-for-demand-promotion/ https://iris.who.int/handle/10665/373534 https://www.who.int/publications/i/item/9789240084513

43. https://www.who.int/publications/i/item/9789240051980 https://www.who.int/publications/i/item/9789240051980 https://ihrbenchmark.who.int/document/16c-infodemic-management IHR (2005) benchmarks … and into emergency capacity planning and benchmarking

44. OpenWHO courses on the infodemic management channel Infodemic management 101 Infodemic management: addressing health misinformation Infodemic management: developing an infodemic insights report Infodemic management: using human-centered design good practices Infodemic management: designing strategies and using the toolbox Infodemic management: designing and testing messages for social marketing  Infodemic management: defining a taxonomy for social listening Infodemic management: working effectively as an infodemic manager in the field ©WHO/Sam Bradd https://https://openwho.org/channels/infodemic-management English Spanish French Arabic Farsi If you want to learn more:

45. See CDC’s Crisis and Emergency Risk Communication (CERC) manual and training https://emergency.cdc.gov/cerc/manual/index.asp CERC provides a useful frame for responding during emergencies that complements infodemic management

46. See: https://www.who.int/teams/epi-win/infodemic-management/3rd-who-training-on-infodemic-management Free training videos WHO and partners Booket: https://www.who.int/news-room/articles-detail/call-for-applicants-for-comprehensive-training-for-promotion-of-vaccine-demand-to-maintain-and-restore-routine-immunization-and-promote-COVID-19-vaccination Videos: https://www.youtube.com/channel/UChNXDnWCc4bi9-bP6_LorHw

47. https://rootedintrust.org/wp-content/uploads/2023/08/Inequity-Driven-Mistrust-Paper-Web-2.pdf Infodemic management in humanitarian emergencies https://rootedintrust.org/wp-content/uploads/2023/11/Information-Ecosystem-Preparedness.pdf

48. https://centerforhealthsecurity.org/2024/center-launches-new-practice-oriented-playbook-for-addressing-health-misinformation https://centerforhealthsecurity.org/sites/default/files/2022-12/spars-pandemic-scenario.pdf https://centerforhealthsecurity.org/sites/default/files/2022-12/spars-pandemic-scenario.pdf

49. the recency of an LLM’s pre-trained dataset is a notable limitation to its overall effectiveness and accuracy In emergencies, because of changes in policy or guidance (and their specificity to countries), or evolution in evidence, and pretraining datasets on health topics may inadvertently provide inaccurate or decontextualized health information, especially on health questions and concerns or epidemiology of disease that change relatively quickly over time. What is considered accurate in health information is not just a mere check but is specific to national guidelines, the population in question, and the context. GPTs don’t pick up humor or sarcasm really well. GPT, based on a corpus of English language data and predominantly (white male) western sources, it won't represent perspectives from other places where less data is available. Some limitations of using generative AI (GPTs) in analysis or generation of messages in health communication

50. https://xkcd.com/386/ https://xkcd.com/2051/ Key takeaway: Pick your battles and choose your partners wisely.

51. The effectiveness of infodemic management in emergencies will be limited by the level of preparatory work done in routine From: https://www.who.int/initiatives/preparedness-and-resilience-for-emerging-threats

52. Explicitly support needs of vulnerable populations (incl. access to information and creating feedback loops) BEFORE AN EMERGENCY, MAINTAIN AND DEVELOP TRUST AND RESILIENCE. Reinforce trusted relationship between patient and provider - build skills to address patient concerns and misinformation Increase linkages between health authorities and where people work, pray, play, study and gather Continuously build on what works to address low-level infodemics through stronger policy mechanisms and interventions Collaborate with communities so that their needs are reflected in health policies and they are partners in implementation Promote digital, information, science and health literacies to increase resilience to misinformation and infodemi cs

53. EXAMPLES Source: UNICEF Source: International Pediatric Association Source: Trust & Safety Tycoon web site Source: Cranky Uncle web site

54. Identify and address questions, concerns, information voids, and debunk misinformation DURING EMERGENCY, LEVERAGE EXISTING TRUST AND PREVENT EROSION OF TRUST. Taking timely and consistent action with words, actions, and interventions Account for health worker and health system needs (e.g. HW burnout) W orking within structured and unstructured networks to ensure information is disseminated by those who are trusted Be nimble and address changing population information needs - ongoing updates of health guidance Foster an information environment where communities themselves can actively address information voids and curate conversations on health topics

55. EXAMPLES Source: I nterNews web site Source: WikiMedia foundaton Source: WHO

56. Update preparedness and response plans AFTER EMERGENCY, LEARN FROM GAPS IN TRUST AND REINFORCE RESILIENCE. Leverage community-based evaluation approaches on infodemic management and trust-building efforts Incorporate lessons learned into capacity building for health workforce, media and civil society Institutionalize new partnerships, networks, and ways of working t hat were stood up during emergency response Integrate or pivot capacity that was stood up during emergency to other health priorities in routine Learn from what did not work and conduct targeted operations research on what did work to identify the transferability and adaptability of tools, practices and policies.

57. EXAMPLES Source: Nigeria CDC Source: Social Science Research Counci l Source: Sabin Institute Source: NEJM Catalyst Source: JMIR infodemiology