AI as a career skill.MCU AI conference slides.may 11.2024.pptx

This presentation paper reported the findings from interviews and assignments from 4 students who extensively used Gen AI in a 5.5-month ESP program. One of these students was followed up with another interview 6 months later, after getting a job and continuing to use Gen AI over 20 hours a week.

1. From class to career: C ase studies on Gen AI as a career skill Nigel P. Daly MCU, May 11, 2024

2. Plan AI as career skill – some research support 5-month case studies – differing AI-use patterns and their impacts on language learning 12-month case study: AI-in-class to AI-on-the-job Conclusions

3. GenAI as career skill – Research support

4. GenAI as career skill – lit review ChatGPT boost productivity and work quality for white-collar workers: Study 1: Science (Noy & Zhang, 2023) study: 444 participants in 2 groups. Task : writing and editing tasks in marketing, grant writing, data analysis, and human resources ChatGPT vs non-ChatGPT groups: A. ChatGPT group completing tasks 37% faster - initial quality the same for both groups B. repeated task iterations, the ChatGPT group's work quality improved significantly faster 2. ChatGPT group completed more work per unit time (production volume + ) while maintaining a consistent quality of work ; and 3. I terations to improve the quality of work: ChatGPT group scored higher on quality from the beginning and continued to improve.    Source : Noy, S. and Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192.  

5. GenAI as career skill – lit review ChatGPT boost productivity and work quality for white-collar workers: Study 2 : Nature ( Lenharo , 2023): 7% of all McKinsey consultants (N=758) ChatGPT for both weaker & stronger writers: productivity+, writing quality+ ONLY for weaker writers. bottom-half skilled participants: 43% increase in performance top-half skilled staff: 17% quality increases Study 3: Dell’Acqua et al., 2023 (echoed Noy and Zhang’s (2023) study) S taff using ChatGPT were more productive 12.2% more tasks 25.1% more quickly . Sources : Lenharo , M. (2023, July 13). ChatGPT gives an extra productivity boost to weaker writers. Nature . // Dell’Acqua , F., McFowland , E., Mollick , E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer , L., Candelon , F., & Lakhani, K. R. (2023, September). Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. (Working Paper No. 24-013 ). SSRN . 

6. GenAI skills will help my students be more competitive in their careers

7. Educational setting: International Trade Institute (TAITRA training center) Program : intensive 5.5 months Courses : Business knowledge and communication skills Business English, sales presentations, negotiations, meetings, marketing communications Writing: email & research reports job application writing and interview skills Students : post-graduates, looking to Apply for first jobs, Improve career opportunities Change careers Study abroad at English universities

8. AI might solve some Teaching problems Too much content … Give models, students copy … BUT Don’t really learn/remember Forget quickly Create poor quality work

9. AI Teaching solution? Hope and goal for Students Students become familiar with GenAI, get good at using it, and take it away with them after leaving ITI But …. Need AI policy …

10. “Human-first” workflow: LinkedIn assignment “Human-first” type workflows Some assignments required demonstrating this

11. Case studies – investigating AI use in ESP courses Subjects : 4 students, different ability ranges Data collection : assignments, AI use test semi-structured interviews after 5 months Interview themes: impressions of AI tools, benefits, challenges, impact on English learning

12. 3 case studies: 3 AI user types Client S tudent Apprentice

13. Case 1: Morris 42 yrs , TOEIC 550, 10+ yrs journalist, shy, introspective AI use? – CLIENT:TRANSLATOR “[The most useful function of genAI for me] is translate . It very helped me in class … if I do your Industrial report, the writing is a big problem for me so it can help me write … and it can save my time to research the information … if I want to translate, I will use the ChatGPT …  if I want to some information with the references, I will use the Bing Chat … it's amazing … I think it's amazing !”  

14. Case 1: Morris (42 yrs , TOEIC 550, 10+ yrs journalist, shy, introspective) AI use improve English? No. No noticeable improvement. He noticed recurring words but made no effort to record or remember them. Struggles Distinguish and use tone in English Prompt making : “ It’s challenging for me asking questions … because I'm not creative . … I don't have more ideas how to ask a question.” Frustrated by lack of sources from AI: “ I always want to know where [it] got the information , … if I had more asking where [it] get the information, it will help me a little, but sometimes it created some information. … it's fake, yeah so sometimes you knew it was fake.”

15. Case 2: Gavin 32 yrs , TOEIC 650, English major, cook in Japan, sociable AI use? – STUDENT: TUTOR “ I like to think my own way and type my own thing and write my own paragraph. And then I will ask ChatGPT … to correct the grammar or this make it smoother … or to make it like more professional English writing ”.   

16. Case 2: Gavin (32 yrs , TOEIC 650, English major, cook in Japan, sociable) AI use improve English? Yes. “ I did uh [learn] a lot and I always like write [new language] in the book [his personal notebook], but now I don't remember it. … [but] yes, [my English] actually improved, … like uh when … your assignment or some someone else say … I will say, oh I saw that before [from genAI] and this means that. Yeah, I think that's helped me a lot.” Struggles Worries about becoming too dependent on AI, so … “I think the if the student can learn how to use it [AI] earlier it will be better … but still try to not just give everything to the AI”. …. “ I write [my] own version first , then ask ChatGPT for revision, sometimes [ask it to] generate a lot of versions” like “when we did the slogan in PnD [Product and Development] class, I used like 20 times”.

17. Case 3: Aria 28 yrs , TOEIC 900+, US Masters, Deloitte accountant 3+ yrs , quiet, perfectionist AI use? – APPRENTICE:MASTER “ I use [it for] like pretty much everything …   on ChatGPT for my assignment … not only because it can help me generate a more … natural and fluent language but also because I think I can learn a lot of how it used the language” 2 main benefits: “ language learning”, “it increased my speed of doing everything. … it gave me an okay answer, and then I just need to modify it, so I think um it saves me a lot of time”  

18. Case 3: Aria AI use improve English? Yes! Reading + Writing + Vocab : “ I think I learned a lot of um like a more natural [English] use of ChatGPT because even though I put in a prompt, I need to read what's the outcome and I think that helped me improve a lot, [such as language] I didn't realize [read] in English novels … I think especially the [ reading] speed improved and also maybe like sometimes if there's somewhere I don't really know, I think I'll go and research so that also increase my … vocab for that”.   Prompting improved writing ability : “ when I um put in the prompt, I also need to think of how … to say whatever what I want the AI to do, so I think that also increased my writing ability because sometimes it's like if I type it [and] express in the wrong way, then it will not give me the outcome I want, so I need to keep revising”.   

19. Case 3: Aria Productivity “Before, if I don't have this AI tool, then I need to write everything by myself, that will take a lot of time. But right now, I just need to put, like, a few prompts and then make sure that the result is the one I want , and then I think that saves me a lot of time …” “[By myself,] I think I can do it within maybe half an hour for an email … I need to check that if the word I'm using is proper, so that'll take actually a lot of time … but if I use AI then I think it would be less than 15 minutes … probably less than 10 minutes. ”   Struggles None mentioned … maybe overuse?

20. Case 3: Aria – LinkedIn profile assignment Human-first workflow Step 6: students had to analyze the revised version and say what they kept and why. Aria wrote a full, single-spaced page I tried to be humble to be connected with people, so I chose to use the words    “honored to connect” and “learn more about your opportunities”. In addition, I’m    still showing my passion at the end, so I used words like “eager”, “passion”, and    “drive to work”.    As a teacher, I did not have to correct any mistakes, but was able to focus on style and coherence

21. Chloe’s story: AI-in-class to AI-on-the-job

22. Chloe’s story: Pre-ITI (Jan 2023) 25 years old 2-3 yrs work experience BA in Foreign Languages and Applied Linguistics TOEIC 920 – strong knowledge of English But lacked practice and confidence … Shy and hesitant in speaking Career dream : digital marketer Obstacle? No relevant education No experience Not technophile, did not know about ChatGPT

23. Chloe’s story: @ITI AI power user “I used ChatGPT for almost every assignment .  I really enjoyed the process of telling AI tools to do something, observing the response, and modifying it to get a very impressive outcome. I would also discuss this with my teammates. Sometimes we would get some very interesting responses, discuss them, and have a really good time.”  Assignments Sales presentations, negotiations, marketing plans, advertisements, emails, resumes, LinkedIn profiles, industry research and report writing, etc Helpful “ With ChatGPT, my IELTS 6.5-writing ability immediately jumped to 8.5 or 9 . I noticed how it changed certain phrases, grammar or vocabulary and improved my English.” Transformations Confident about the good AI content But becoming “less confident in myself … maybe rely too much on AI”

24. Chloe’s LinkedIn assignment She experimented with different prompts to make the writing more passionate , creative , humorous …      Prompt 1: please rewrite this LinkedIn about section with more details and creativity      Prompt 2:  please rewrite this LinkedIn about section in a passionate and creative way with a bit of humor     Marketers should be creative, so I made some recommendations … “smoke” and “fire”

25. Chloe’s LinkedIn assignment Step 6 analysis and reflection:   I think it’s interesting to use the imagery of fire to describe passion for things, and ‘hands-on experience’ is the phrase that didn’t occur to me when I wrote the first draft.    I didn’t know that I can use ‘bring a set of something to the table’ to convey what skills I have. I didn’t know the word ‘ self-starter’ before, and I think this is a precise word to use in my About section.    I think ‘tackle’ new challenges and ‘see through to completion’ are more vivid ways to describe what I can do. … And I think ‘ a fire in my belly’ corresponds to ‘ a fire in my heart ’ in the first paragraph. It can be an interesting way to summarize my About section.    

26. Chloe’s LinkedIn assignment 7+ versions later, she posted it on LinkedIn. A couple of months later, a stranger on LinkedIn praised Chloe’s profil e and even invited her to join a writing group he created.

27. Chloe’s story: Post-ITI Immediately got a job as a marketing specialist at DigiTimes Still an AI power user: “ I use AI tools almost every day in my job now … more than 20 hours a week ” AI use-cases Brainstorm Translate, Summarize Analyze data and create charts Write and revise copywriting (=articles for marketing/advertising purposes), eg s horten sentences when there are word limits, eg ads

28. Chloe’s story: Post-ITI - CHALLENGES 1. Attention Once did not double check a translation because of a lack of professional knowledge After the article was published on the internet, a reader from an English company wrote a critical comment about wrong word choice and accused the organization of being unprofessional . “I felt ashamed” 2. Prompting “… it has taken me a long time and practice to get good at it. I have seen coworkers use ChatGPT and assume all you need to do is copy-and-paste text to get translated . But I have seen some of their incoherent and weird-sounding results. I told one coworker you have to have a good idea what outcome you want and specify the type of language and tone you want for it.” 3. Dependence “Over-dependence is still an issue. I still feel a bit over-dependent on ChatGPT, but I now write short emails or messages myself more often and deliberately avoid AI. I think this is another part of AI learning— when not to use it.”      

29. Chloe’s reflections “ If you asked me [at the beginning of 2023] w hat career skills I needed, AI would not have been on my list . So, I am still surprised how important it has become in my job . I can generate impressive texts, summaries, and charts quite quickly and save time.  And with AI brainstorming, I think I also create better quality work than if I did it by myself—especially for more creative tasks like writing ads.  For people who need to write professional texts, perform analyses, and come up with creative ideas, I think AI is an essential career skill .”  

30. Conclusions 1. Different AI uses, different results … 4 case studies 1. Low inter: Morris: Client:AI-translator 2. Inter: Gavin: Student:AI-tutor 3-4. Adv: Aria, Chloe: Apprentice:AI-Master Students will use tools in different ways! 2. AI will be a career skill for knowledge workers AI usage is affected by training (prompting) and IDs (language level, motivation, personality (openness, creativity), tech comfort, etc )

31. Conclusions 3. Prompting requires structured training 4. Dependence is a real issue 5. AI literacy vs AI fluency: How to use AI means a change in role: Writers  editors, Content creators  content checkers/EVALUATORS BUT need competence, to evaluate, eg Morris’s problem with English tone

32. Thank you Nigel P. Daly on LinkedIn [email protected]

33. Thank you. … Q&A? 1. Different AI uses, different results … 4 case studies 1. Low inter: Morris: Client:AI-translator 2. Inter: Gavin: Student:AI-tutor 3-4. Adv: Aria, Chloe: Apprentice:AI-Master Students will use tools in different ways 2. AI is career skill for knowledge workers AI usage affected by training (prompting) and IDs (language level, motivation, personality (openness, creativity), tech comfort, etc ) 3. Prompting requires structured training 4. Dependence is a real issue 5. AI literacy vs AI fluency: Writers  editors, Content creators  content checkers/EVALUATORS BUT need competence, to evaluate, eg Morris’s problem with English tone

34. Scott’s request after reading Chloe’s story in Taipei Times Featured Section What I was thinking about was a case study about how ChatGPT is actually being used. Things I'm interesting in knowing...in no particular order. - How good was her ChatGPT when she finished your class? - Was her ChatGPT skill part of the reason she was hired? - How is her job conventionally done? How has ChatGPT changed how it can be done? - I'm also interested in a more detailed understanding of what she's doing with ChatGPT. You mentioned problems she's encountered. That's always good to hear. Even though we all know this, it's an important message. I also liked your comments about how students saw their English ability being improved by ChatGPT use.  This is significant to me because my own students overwhelming report that ChatGPT use has improved their English. But it also reinforces the viewpoint expressed in my group that the AI is changing to emphasis of school-based language teaching away from writing to reading/editing skills. CASE STUDY AI SKILL DEVELOPT PROBLEMS ENGLISH ABILITY EFFECTS