Health communication, AI and health misinformation.pptx

A slide deck with the key takeaways for a health communicator in the modern information ecosystem.

1. TINA D PURNAT Prajna Leadership Fellow and DrPH Student TH Chan School of Public Health, Harvard University Fellow of the Australasian Institute of Digital Health tinapurnat.com Unless credited, images generated by ChatGPT 4o.

2. Toto, I have a feeling we’re not in Kansas anymore... 1938 Source: Twitter. Read more at: https://www.scientificamerican.com/article/what-taylor-swift-conspiracies-reveal-according-to-science/ 2024 Source: https://www.tnonline.com/20211101/war-of-the-worlds-hoax-broadcast-sparked-panic-83-years-ago/

3. AI could benefit public health communication What is the evidence of its harms? AI could be very harmful to public health communication What are the public health use cases for benefits?

4. AI strengthening public health communication Use cases Tools Automating analysis of social media Talkwalker, Pulsar, Meltwater,... Generating text, images and video ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Testing for readability/understandability ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Draft translations, editing, synthesizing information ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Making content more accessible (e.g. alt text, descriptions) ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Brainstorming/organizational tools ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Offering more tailored content/responses to users (e.g. chatbots) Many commercial offerings ...but all come with limitations and biases.

5. AI eroding public health communication ... all the same tools can be either misinterpreted or misused. Use cases Tools Automating analysis of social media Talkwalker, Pulsar, Meltwater,... Generating text, images and video ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Testing for readability/understandability ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Draft translations, editing, synthesizing information ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Making content more accessible (e.g. alt text, descriptions) ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Brainstorming/organizational tools ChatGPT, Claude, Poe, Bard, Copilot, ... DALL·E 3, Midjourney, Stable Diffusion, Adobe Firefly, Generative AI, ... Canva, Miro, Mentimeter, ... Offering more tailored content/responses to users (e.g. chatbots) Many commercial offerings

6. If a similar dashboard were available for epidemiological surveillance, it would never be used by a health department because: data of uncertain provenance needs triangulation, not comparisons geographically non-specific measures the wrong thing analytic “black box” The most common response to: How should we track health misinformation? A “magic” social media dashboard!

7. AI tools sometimes spit out irrelevant analyses that are not useful for public health action. Sentiment analysis Reach metrics Hashtag tracking The reason these don’t make sense is because they were built for commercial brand promotion. ...and they use data that’s easiest to obtain and easiest to analyze See: https://researchworld.com/articles/10-challenges-of-sentiment-analysis-and-how-to-overcome-them-part-1 Generalized top narratives See: https://www.wired.com/story/death-of-truth-misinformation-advertising/ See: https://arxiv.org/html/2402.10230v1

8. Data that is available for AI-assisted analysis is biased and incomplete. Most social media platforms are walled gardens. Since the COVID-19 pandemic, data available from these platforms has been further restricted. The information environment is changing rapidly. Users have dispersed across new social media platforms, many of which don’t have data sharing and governance policies in place for health. Dashboards consisting of socio-behavioral indicators that are more than 3 weeks old are less useful for action. Sufficient subnational or population- or community-specific analysis is rarely available to understand health behaviors.

9. However, there are good examples of public-health and community-focused insights reports https://iris.who.int/bitstream/handle/10665/359144/WER9727-eng-fre.pdf https://www.cdc.gov/vaccines/covid-19/vaccinate-with-confidence.html https://sites.brown.edu/informationfutures/2024/01/23/is-the-2024-election-still-happening/ https://sites.brown.edu/informationfutures/the-information-project-pilot-page/

10. Infodemic insights analysis and reporting Includes: Manual Editable templates Workbook Teaching simulation eLearning modules Annex with emergency, outbreak response and immunization resources See link to resources, webinars and trainings: https://www.who.int/news/item/06-07-2023-introducing-rapid-social-listening-and-infodemic-insights-for-action-who-and-unicef-launch-manual-on-6-steps-to-build-an-infodemic-insights-report

11. Digital anthropology puts the community experience at the center of analysis of the information environment. Rapid community assessment tools can also include information environment components. See US CDC’s RCA toolkit and addenda for adolescents and digital spaces. Other tools that might be helpful https://www.gethealthysmc.org/post/youth-rapid-community-assessment For public health professionals, civil society organizations, and interested community members on how to analyze community stories to foster a healthier information environment https://communitystoriesguide.org/the-guide/ https://www.cdc.gov/vaccines/covid-19/vaccinate-with-confidence/rca-guide/index.html https://liivcenter.org/digital-anthropology-toolkit/ https://www.gethealthysmc.org/post/youth-rapid-community-assessment

12. Working with communities’ expressions of questions, opinions, concerns, and circulating narratives requires ethical care in (AI and other) data collection, analysis and use for health communication. Ensure you have someone with ethics expertise involved in your team Anonymize what you can (data collection, analysis, reporting) Consider discarding data sources where ethical nature of how data was collected is in doubt, such as social listening data from closed messaging networks Emphasize triangulation of data sources and selection of research methods appropriate for the task to ensure best level of evidence, even when working at speed Publish your methods transparently Ensure that accountability and ethical principles are reinforced in policy, especially when working with subcontractors who may be unfamiliar with public health Ensure stronger M&E framework and one that includes perspectives from affected population Ensure that communities where data is extracted from also benefit from hearing how the data was used and how analysis was used to support improving health Perform a Risks, Harms and Benefits Assessment when planning - different health topics have different ethical considerations

13. Ask yourself, when was the last time you changed someone’s mind with a simple message? The most common response to: What messages should I use to promote health guidance? Use a “magic” message bank

14. Example challenge: Encourage people to eat more diverse food groups public health approach social marketing approach AI approach “ChatGPT, give me a message about healthy eating tailored to the African-American community.” Recommendation: Make it cheaper and easier for people to find and purchase fresh fruits and vegetables in this neighborhood. “Get out of the beige routine: eat your rainbow and turbocharge your health!” “Let's come together as a community to embrace healthy eating habits that celebrate our culture and support our well-being. “ Reproducible Reproducible Not reproducible

15. AI lets you take shortcuts, which can help speed up processes. However, AI is an unreliable source of content and analysis for expert-driven communication.

16. Even if we can do some tasks faster by using technology, this doesn’t let us off the hook that health communications make an implicit promise about quality, accessibility, affordability, and acceptability of health guidance, health services and products. https://substack.com/home/post/p-145099430 https://www.statnews.com/2024/05/09/h5n1-communication-didnt-federal-government-learn-anything-from-covid/ https://www.washingtonpost.com/travel/2024/02/18/air-canada-airline-chatbot-ruling/ https://www.ualberta.ca/folio/2020/01/virtual-assistants-provide-disappointing-advice-when-asked-for-first-aid-emergency-information-study.html

17. Health communication must be fully aligned with health service delivery and experience, and health guidance Demand for health information Demand for health services and products Adherence to health guidance PULL – people want something from the system PUSH - the system wants something from the people See: https://www.linkedin.com/pulse/demand-promotion-trust-trouble-information-tina-d-purnat-cw3of/

18. Beware of “easy” solutions to health misinformation and information environment challenges “We fight AI with AI.” “There’s a dashboard for that.” “We can debunk our way out of this.” “If we focus on the top ten misinfo spreaders, we’ll solve this.” “We’ll inoculate against misinformation with empirical truth!” “We’ll hire a social media person for that.” They are as snake oil is to people desperately looking for a cure.

19. It comes down to principles and values of working in public health: do no harm/prevent and reduce harm communicate what is known and what is not known use best available evidence (triangulate) protect the vulnerable evaluate as you implement use multilevel interventions involve the community of focus How do we apply public health principles to health misinformation concerns?

20. We should demand and expect better tools and solutions that are evidence-based, multi-level and embedded in public health practice. (c) WHO/Sam Bradd, Drawing Change

21. Key takeaway #1: P roviding credible, accurate health information is the basis for any other health communication intervention. Example: In 2021, only 8% of VA web sites are conforming to Section 503, which is the federal standard for accessibility of web sites for people with disabilities https://www.casey.senate.gov/imo/media/doc/report_on_the_accessibility_of_websites_of_the_department_of_veterans_affairs1.pdf

22. Key takeaway #1: Ensure the basics first web site is up-to date, accessible, and search engine optimized ensure web site is crosslinked from other reputable sources, such as county, state, or jurisdiction-specific web sites people with questions can reach a live human if they need to editorial policy and content moderation policy across social media channels point of contact for the media and for inquiries FAQ section on web site content available in multiple formats and languages SOPs developed on how to respond to and address urgent questions, concerns, information voids and mis/disinformation

23. Key takeaway #2: Addressing health misinformation is not just the health department’s job . Actually, if you try to do this solo, you will fail. https://xkcd.com/386/

24. Only few categories of health misinformation narratives can be directly addressed by a health department Adapted from: FirstDraft https://firstdraftnews.org/wp-content/uploads/2020/11/FirstDraft_Underthesurface_Fullreport_Final.pdf?x21167 Development, provision and access of healthcare services and products Safety, efficacy and necessity of diagnostics, therapeutics and vaccines Political and economic motives Conspiracy theories Liberty and freedom Morality and religion Can be addressed by a health department

25. Don’t assume that the health information that you share will be seen as trusted and credible by your audiences. Instead, partner with organizations and voices that are trusted.

26. https://libguides.sdsu.edu/library-toolkit-addressing-health-misinformation https://www.hsph.harvard.edu/chc/resources/digital-safety-kit/ https://www.aap.org/en/patient-care/media-and-children/center-of-excellence-on-social-media-and-youth-mental-health/ Community organizations that are focused on healthy social and digital environments Factcheckers & media Libraries Science organizations Museums Medical associations Science & health journalists Key takeaway #2: Pick your battles and partner with organizations and influencers where people play, pray, work, study and gather

27. Key takeaway #3: Think like a fire marshal, not like a firefighter, when it comes to addressing health misinformation. Prevention is key. (c) WHO/Sam Bradd, Drawing Change

28. Instead of focusing on mis- and disinformation, focus on low hanging fruit: addressing questions, concerns and information voids Questions Concerns Information voids Misinformation Disinformation More common, easier to address by health departments PREVENTIVE action Less common, harder to address by health departments REACTIVE action See: Managing Infodemics in the 21st Century: https://link.springer.com/book/10.1007/978-3-031-27789-4

29. Focus on what is in your control vs what is outside of your control Identify information voids that you can fill Better address questions and concerns Work with community leaders and members to address questions and concerns Assess risk for health information challenges Look ahead to potential events that might cause additional confusion or concerns and plan for it Be proactive in providing messages and answers to questions https://misinforeview.hks.harvard.edu/article/misinformation-reloaded-fears-about-the-impact-of-generative-ai-on-misinformation-are-overblown/ Key takeaway #3: Preventative actions you can take now

30. Key takeaway #4: If you address health misinformation, think beyond communications and social media responses.

31. ‹#› 20 tried and tested strategies to address health misinformation 1 Monitoring and detection of narratives, questions, concerns, and misinformation in the information ecosystem Public health social listening and integrated analysis 2 Science and health communication Rapid content development to address emerging information voids Quality health information materials and resources Science and knowledge translation Debunking and misinformation correction 3 Education & training to build resilience to misinformation Capacity building to identify and address health misinformation Prebunking Improving self-efficacy to adhere to recommended health behaviors Improving ability to develop a more accurate risk perception, and judge benefits of recommended health behaviors 4 Engagement Individual and community-level engagement Psychological support to individuals and communities Health worker level engagement Social media-based strategies Coalition building and digital engagement Digital design for the user experience of information environment 5 Laws, regulations and ethics Cybersecurity laws or cyber-resiliency strategies Public advisory and guidelines with legal mandate/enforcement Legislative ban of misinformation spread or legislation for right to health information Regulation of information and related platforms and ethics Content moderation policies, oversight and implementation Consumer protection laws SUBCATEGORY INTERVENTION CATEGORY See examples in: Purnat T, John O, Pundir P, Ishizumi A, Murthy S, Rajwar E. et al. https://bit.ly/evidencegapmapinfodemic . Open Science Framework. 2022.

32. Key takeaway #5: A checklist for your boss Fix the health system’s digital footprint Introduce new ways of developing and promoting health content, including experimenting with AI responsibly Integrate health misinformation prevention and response into emergency structures before the next emergency Leverage community and digital insights generation in routine for the highest priority health policy areas Train health care workers on how to communicate more effectively online and offline with people they care for, and to address misinformation. Promote digital, information, media, science and health literacy. Incorporate metrics and indicators on information environment into regular reporting. Do a better job of protecting health care workers from online harassment and doxing.

33. Key takeaway #6: Funding this work requires adaptability to health topics and linkage to other health system functions Tap into partnerships and funding in topics other than emergency preparedness. Develop capacity for insights generation by partnering with other teams. You can provide expertise for: routine programming (immunization, adolescent health, injury prevention, etc) high-priority policy topics (opioid use disorder, nutrition, suicide, etc) You can start with ad hoc reports (before routinizing them) and collaboration in design of interventions. Put together a frame for the team you need with the breadth of expertise and intervention capacity. Then apply for grants that support those capacities. grants.gov has 378 posted opportunities that involve “health” & “misinformation” that local and health departments are eligible to apply for. Be careful not to overemphasize one strategy as a “magic solution”. Often we apply for grants with narrow specific aims, but need to implement them in ways that strenghten a broader capacity and system.

34. Key takeaway #7: You can start work on health misinformation at different capacity and resource levels Bigger vehicle = larger toolbox of interventions, more ways of mitigating the information environment impacts, including health misinformation Mountain Bike Very off terrain, usually one driver (person working on health misinformation issues) Mostly an emergency response function Limited access to robust data sources Limited human and financial resources Lack of political/strategic buy-in Health misinformation-related products are often requested ad-hoc Sturdy 4x4 Have sufficient staff and resources for moderate capacity to address health misinforamtion Can leverage existing resources, tools and relationships for responding to health misinformation, bust mostly just in emergency response Regularly develops information environment insights but struggles with ad-hoc requests Funding may not be stable May not be able to generate speedy insights Insights recommendations not routinely considered or implemented by health system or communication team Luxury Vehicle Well staffed, resourced and supported unit with multiple staff with defined roles Has health misinformation mitigation integrated in routine and emergency functions Regularly develops insights products and handles ad-hoc requests rapidly Can offer subnational insights at speed, as well as segmented insights for specific communities Insights are regularly discussed with key stakeholders who regularly act on recommendations to improve programs and communications (c) WHO/Sam Bradd, Drawing Change

35. Key takeaway #7: You can start work on health misinformation at different capacity and resource levels Bigger vehicle = larger toolbox of interventions, more ways of mitigating the information environment impacts, including health misinformation I nsights generation Work with health program staff to identify questions that can be answered through monitoring of the information environment and insights generation. Identify and use existing data sources and focus on understanding populations experiencing vulnerability Leverage partners, such as other health programs, academic and nonprofit organizations, federal agencies (US Census Bureau, CDC) Work side-by side with epidemiologists to improve coordination. Ask them what insights they’re missing and brainstorm how you might be able to help. Strategy and interventions Refer to slide of 20 strategies to address health misinformation and identify a subset appropriate for your context. Don’t focus only on social media, prebunking or debunking. Pilot them. It may be that your access to social-behavioral resources may be limited, so work with your partners and contacts to develop simpler interventions. Still, make an effort to evaluate what you are doing, even if you use simple evaluation methods. Share your experiences with others working in health misinformation Consult with an ethics expert, if possible. Coordinate as much as possible with epidemiologists and other health communication colleagues for data sharing and developing interventions that support their work Do some basic user testing to figure out how to improve existing channels for communication, including the health department web site and social media presences Focus on making small, doable improvements (c) WHO/Sam Bradd, Drawing Change

36. Key takeaway #7: You can start work on health misinformation at different capacity and resource levels Bigger vehicle = larger toolbox of interventions, more ways of mitigating the information environment impacts, including health misinformation I nsights generation Add more data sources, especially those with better access to populations experiencing vulnerability Set up a team with dedicated staff for monitoring the information environment (e.g. analysts focused on specific data sources) Actively work to create demand for insights products Coordinate with community-based organizations to share insights to inform communications and programmatic activities. Strategy and interventions Utilize participatory research approaches in design and development of interventions. Try to develop interventions that work on multiple levels (e.g. individual, community, health system, government,..) Collaborate with behavioral scientists and epidemiologists within and outside the health system, whenever you can. Share your experiences with others working in mitigating and preventing health misinformation. Consult with an ethics expert. Watch out for unintended consequences and harms of interventions you’ve implemented. Advocate for stronger data collection and indicators in communication activities Consider developing specific metrics on individual, community or health programme resilience to health misinformation Codevelop guidance with communities of focus where possible. Seek out innovative partnerships and new community-based data sources. (c) WHO/Sam Bradd, Drawing Change

37. Key takeaway #7: You can start work on health misinformation at different capacity and resource levels Bigger vehicle = larger toolbox of interventions, more ways of mitigating the information environment impacts, including health misinformation Insights generation Develop new data sources where you have limited availability; invest in routine data sources and in representative data collection Regularly assess the needs of insights report stakeholders in insights products and modify accordingly Work with communications and program staff to develop improved messages and actions Innovate new rapid message testing methods Provide ways for partners and networks to use, edit and remix messages for different formats (eg, use templates, remove your logo from assets) Promote regular recurring reports with ways to take action that include recommendation for communications and messaging Strategy and interventions Build users into the entire design cycle, from ideation, to development, to implementation to evaluation. Develop robust research designs and protocols to evaluate interventions. Share your experiences with others working on health misinformation in any health topic Ensure there’s ethical and scientific oversight to all parts of intervention design, rollout and use. Develop tailored insights and products for different communities. Collect additional data to address information voids and data gaps about communities experiencing vulnerability Support implementation science and evaluation of health misinformation prevention and mitigation interventions in specific communities. Seek feedback from stakeholders on insights usefulness of reports and interventions for specific communities. (c) WHO/Sam Bradd, Drawing Change

38. Thank you very much! [email protected] Resources for infodemic managers

39. This presentation is available in Canva and you’re welcome to use it. https://www.canva.com/design/DAGGugtEDYc/jpD-XZ8sF46bMK4Q2NILow/edit?utm_content=DAGGugtEDYc&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton

40. https://issues.org/to-fix-health-misinformation-think-beyond-fact-checking/ Recommended reading and listening: https://issues.org/misunderstanding-misinformation-wardle/ https://www.cjr.org/special_report/truth-pollution-disinformation.php

41. See mini talk at: https://www.youtube.com/watch?v=YnMJ1sOkCUM&list=PLwmB5Aqso7V43SnUrCDOCyw08aBi9ljSA&index=1&t=1s&ab_channel=purnatt Illari, L., Restrepo, N.J. & Johnson, N.F. Rise of post-pandemic resilience across the distrust ecosystem. Sci Rep 13, 15640 (2023). https://doi.org/10.1038/s41598-023-42893-6 Johnson, N.F., Velásquez, N., Restrepo, N.J. et al. The online competition between pro- and anti-vaccination views. Nature 582, 230–233 (2020). https://doi.org/10.1038/s41586-020-2281-1 Social network analysis can show gaps in engagement, and communities that are the “amplifying mediators” of narratives

42. Familiarize yourself with the current scientific debate about social inoculation and prebunking: https://www.conspicuouscognition.com/p/misinformation-poses-a-smaller-threat versus https://www.nature.com/articles/d41586-024-01587-3

43. PUBLIC HEALTH AND THE CHALLENGE OF INFORMATION ENVIRONMENT Examples hijack health-conscious communities Industry marketing and influencers Heathcare professionals undermine evidence-based health advice Trusted tech platform services misdirect and mislead people searching for health information Communities with multiple vulnerabilities may make health choices based on low quality information. Well-funded and well-organized “anti” movement counters health advice and politicizes health. Health fraud, scams and deceptive marketing exploit vulnerabilities within the information environment When health workers are harassed, doxxed and attacked this is the alarm that something has gone very wrong between community and health system For examples of challenges and possible actions, see: tinapurnat.com/blog

44. https://www.kff.org/coronavirus-covid-19/poll-finding/kff-health-misinformation-tracking-poll-pilot/

45. https://pubmed.ncbi.nlm.nih.gov/36531901/ https://www.cdc.gov/vaccines/covid-19/hcp/engaging-patients.html

46. https://www.mitre.org/news-insights/impact-story/whats-all-hipe-mitre-its-building-community-resilience https://meedan.com/programs/digital-health-lab

47. Infodemic monitoring and management are integrated into updates of country pandemic plans and global emergency architecture…. https://www.who.int/publications/m/item/preparedness-and-resilience-for-emerging-threats-module-1-planning-for-respiratory-pathogen-pandemics-version-1 https://www.who.int/publications/m/item/strengthening-the-global-architecture-for-health-emergency-prevention--preparedness--response-and-resilience

48. … into planning for pandemic influenza preparedness and vaccine demand promotion ... https://www.who.int/publications/i/item/9789240070141 https://apps.who.int/iris/handle/10665/372248 https://demandhub.org/operational-framework-for-demand-promotion/ https://iris.who.int/handle/10665/373534 https://www.who.int/publications/i/item/9789240084513

49. https://www.who.int/publications/i/item/9789240051980 https://www.who.int/publications/i/item/9789240051980 https://ihrbenchmark.who.int/document/16c-infodemic-management IHR (2005) benchmarks … and into emergency capacity planning and benchmarking

50. OpenWHO courses on the infodemic management channel Infodemic management 101 Infodemic management: addressing health misinformation Infodemic management: developing an infodemic insights report Infodemic management: using human-centered design good practices Infodemic management: designing strategies and using the toolbox Infodemic management: designing and testing messages for social marketing  Infodemic management: defining a taxonomy for social listening Infodemic management: working effectively as an infodemic manager in the field ©WHO/Sam Bradd https://https://openwho.org/channels/infodemic-management English Spanish French Arabic Farsi If you want to learn more:

51. See CDC’s Crisis and Emergency Risk Communication (CERC) manual and training https://emergency.cdc.gov/cerc/manual/index.asp CERC provides a useful frame for responding during emergencies that complements infodemic management

52. See: https://www.who.int/teams/epi-win/infodemic-management/3rd-who-training-on-infodemic-management Free training videos WHO and partners Booket: https://www.who.int/news-room/articles-detail/call-for-applicants-for-comprehensive-training-for-promotion-of-vaccine-demand-to-maintain-and-restore-routine-immunization-and-promote-COVID-19-vaccination Videos: https://www.youtube.com/channel/UChNXDnWCc4bi9-bP6_LorHw

53. https://rootedintrust.org/wp-content/uploads/2023/08/Inequity-Driven-Mistrust-Paper-Web-2.pdf Infodemic management in humanitarian emergencies https://rootedintrust.org/wp-content/uploads/2023/11/Information-Ecosystem-Preparedness.pdf

54. https://centerforhealthsecurity.org/2024/center-launches-new-practice-oriented-playbook-for-addressing-health-misinformation https://centerforhealthsecurity.org/sites/default/files/2022-12/spars-pandemic-scenario.pdf https://centerforhealthsecurity.org/sites/default/files/2022-12/spars-pandemic-scenario.pdf

55. the recency of an LLM’s pre-trained dataset is a notable limitation to its overall effectiveness and accuracy In emergencies, because of changes in policy or guidance (and their specificity to countries), or evolution in evidence, and pretraining datasets on health topics may inadvertently provide inaccurate or decontextualized health information, especially on health questions and concerns or epidemiology of disease that change relatively quickly over time. What is considered accurate in health information is not just a mere check but is specific to national guidelines, the population in question, and the context. GPTs don’t pick up humor or sarcasm really well. GPT, based on a corpus of English language data and predominantly (white male) western sources, it won't represent perspectives from other places where less data is available. Some limitations of using generative AI (GPTs) in analysis or generation of messages in health communication

56. https://xkcd.com/386/ https://xkcd.com/2051/ Key takeaway: Pick your battles and choose your partners wisely.