Dr. Dan 0:00 Hello, and welcome everyone, to the final session of TCC 2022. My name is Dan Hoffman and I'll be the facilitator for this final session. Here with us today is Allyson Ota, from the Learning Design and Technology Department at the University of Hawaii at Manoa. This is a 20 minute presentation that is being recorded, and will be available for review after the conference and on the conference website. So without further ado, I'll turn it over to Allyson. allyson ota 0:35 Hello, hi, everybody. Thank you so much for being here. I'm super stoked. My name is Allyson Ota, I am a librarian at Kapiolani Community College. By day, and by night, I'm a student in the learning designing technology program. So thank you for allowing me to share with you my master's project that is entitled arming students against misinformation, asynchronous instruction to motivate community college students to practice information literacy skills. All right. allyson ota 1:06 So we've all been living under the conditions of a pandemic for like over two years now. And if you, I don't know if you remember back in 2020, the director-general of the World Health Organization declared that in addition to the COVID-19 virus, we're also fighting an infodemic. And what does that mean? What's an infodemic? Well, he was referring to the massive amounts of information, but also misinformation that's available to us online. And the US surgeon general last summer actually issued a health advisory against health misinformation. So we know, you know, misinformation can be really dangerous. And you might be wondering, like, so what can I do to protect myself and my family? Well, one thing you can do is boost your information literacy skills. And librarians at Kapiolani Community College provide information literacy instruction, in order to help our students. allyson ota 2:00 So the problem that I was seeing is that we have students at our college who have difficulty searching for locating and evaluating information resources to complete their assignments. However, there are far greater implications of the importance to develop these skills in order to inform decision making in their personal lives, when they're encountering all this information online every day. allyson ota 2:23 So the target audience average age is 25. I thought they may or may not know anything about information literacy, they also may see no value in learning about information literacy. And but they're probably motivated to pass their classes and complete their assignments. So typically, before the pandemic instruction with librarians was face to face, the class came to the library with the instructor during class time. And then everything went online after March 2020. And it became kind of pretty much what we're doing right now synchronous online, same time, same Zoom Room, but we didn't really have an asynchronous type of information literacy instruction that we could just give to someone. And we actually got a request from the director of the radiologic technology program, who wanted asynchronous instruction for her spring 2022 class that would be graduating and writing this big research paper. So this would be a way by going asynchronous to support our distance education students and reach more students. allyson ota 3:28 So But Why are students at KCC having a hard time finding information resources and evaluating them? I want to tell you, it's not just our students, it's basically college students in general, that seem to be entering college and they don't have the skills. Younger people do struggle to analyze digital information, despite the fact they're able to access and use online tools. And educators perceive that their students are doing their research through sites like Google, Wikipedia, YouTube, and ot- other social media, and not really looking at books, ebooks, databases, or asking librarians. allyson ota 4:04 So my instructional goal is that upon completing this asynchronous instruction, students will be motivated and feel confident to practice evaluating information resources, analyzing the creator's authority, and utilizing search strategies. allyson ota 4:18 However, I want to know how you feel about yourself, on a scale of one to five, with one being like, very unconfident and five being like, I'm very confident, could you please type into the chat? how confident you are in your own ability to detect misinformation? Okay, I see, oh, it's interesting. I see a lot of threes and fours. And actually you guys are very much in line with the students that I was able to work with. So that's, that's, that's interesting [laughs]. allyson ota 4:48 So what did I design and how did I design it? allyson ota 4:50 I basically made an elearning course and I called it An Introduction to Information Literacy, and it consisted of three modules, Authority is Constructed and Contextual, All the CRAAP and More and Searching as Strategic Exploration. allyson ota 5:05 And then I really dug into the framework for information literacy for higher education. From the Association of College and Research Libraries. academic librarians use this framework as like a guide to help with our information literacy introduction. There are six frames or concepts within we were able to work with 10 faculty members last summer across multiple discipline areas and asked, you know, where are your student - read this framework - Where are your students struggling? allyson ota 5:32 Where do they need support, and overwhelmingly, they told us authority is constructed and contextual, and searching and strategic exploration. allyson ota 5:40 So the domains of learning were both cognitive and affective. And even though my terminal objective is affective, meaning I want to change their feelings about the importance of information literacy, they had to be able to actually perform analysis or otherwise they couldn't get there. So cognitively, they were tested through post pre and post tests to see if they could achieve the skills, and affectively just measuring their feelings through a retrospective survey was done, allyson ota 6:06 I utilized Keller's ARCS Model of Motivation and Mayers Cognitive Theory of Multimedia Learning. allyson ota 6:14 I had six different media types within the site, and 25 different embedded learning objects. And so predominantly videos made up 48% of the delivery of the instruction, followed by text explanation, Google Slides, infographics and only one ThingLink and Padlet element were within the site. allyson ota 6:34 So ARCS says, you know, I need to capture and maintain their attention and show that this is relevant so they'll actually want to learn, right? So I used a lot of video with really high quality audio, and good production. And I tried to keep it less than five minutes, if possible, you know, just to do some segmenting or chunking. And not to bombard them with too much at once. allyson ota 6:56 People learn better from words and pictures then from words alone. So there were, you know, lots of videos and Google slide presentations were in every module. And so this allowed me to do things like embed narration, so they could click the audio icon. And I could use redundancy and temporal contiguity. On top of narration and graphics. i And yeah, that was another thing I did. allyson ota 7:20 I gave a lot of examples and non examples of real items that you they could find online, through the tests and through the videos to show further relevance. allyson ota 7:31 The design was very minimalistic. The content was the star of the show this is you could think of this as a wireframe. All the pages kind of look similar. There's no navigation. So there would be no distractions at the bottom of the page, they go to the next page, they could use the back arrow. This was to make sure I could chunk information and reduce cognitive load and make them go through in the order I wanted them to. allyson ota 7:54 So another technique utilized was at the beginning of each module, allyson ota 7:59 I use pre training and signaling. So I would pan and give them a preview of like what they're going to learn. And then you know bolding and italicizing key terms and concepts that they would be applying. allyson ota 8:11 I used, you know, stuff like this to motivate them personalization principle. It was a young audience. And so I wanted to keep them in there. allyson ota 8:20 When they first get to the site, they would sign in with a username. So they were anonymous to me, but then the site would refer to them by their name the entire time. And also, they could choose between a light or dark mode option, just by toggling this button. allyson ota 8:35 And this is just a side by side to show you it's a really different experience. And most people have strong feelings about this. allyson ota 8:42 This is kind of like a blueprint of what I had, I used to build everything. So you don't have to read the text. But just know that there are three modules. And there are all these boxes. The blue boxes are cognitive objectives, I wanted them to meet the yellow are all motivational example like choosing to be more mindful when sharing information online or choosing to apply these skills in both academic and real world situations. And then my terminal objective, which was my goal to have them be confident and motivated to do these things. allyson ota 9:15 I utilized twine, which is self described as an open source tool for telling interactive nonlinear stories. So you could do like really cool stuff with branching scenarios and choose your own adventure. I didn't do that. I used it in the opposite way to allow scaffolding as I was kind of controlling a lot. That was a conscious decision. As you can see, all these black boxes are pages within the site. You don't have to read the text on this either, but it's forcing them through in the order I wanted to. So like Module One goes into Module Two goes into module three, and Oh, oops, sorry. When you click on one of these boxes, this is what it looked like for me on the back end. allyson ota 9:53 And I just had to use all this type of scripting and coding in order to make the website and then embed the multimedia learning objects. allyson ota 10:01 These are the media types. And the tools and technologies are used for them. I kind of already went over the top six, but I wanted to let you know I use Google Forms in order to collect assessment data. allyson ota 10:12 And let's just look at it. Module One was authorities constructed and contextual. And it dealt with topics like what is information literacy? And how do you identify authority? What are the different types? What is confirmation bias? And how should you approach information. allyson ota 10:28 I utilized explanatory text, and videos, as well as more videos and infographics. allyson ota 10:37 And then at the end, it had a summative to Google slide presentation that would like prepare them for the post test, there was always a pre and post test before and after each module. allyson ota 10:46 And then they would go into module two, which talks about all the stuff they would see in their everyday life online. It's called all the crop and more because it covers the crop test, which is a library strategy, lateral reading, which is used by fact checkers, and just examining evidence with many real world examples. allyson ota 11:03 And so media types included, you know, videos, infographics, lessons on how to perform crop with lateral reading, allyson ota 11:13 a Thinglink article exploration was in here, it's the only module that had Thinglink, they could mouse over or click on hotspots on this problematic article and see what's wrong with it. allyson ota 11:23 Padlet was in this it was an embedded test. It was the only place where they could see each other's work with the exception of the first student who posted but I had 17 examples from the web of social media images or articles that you could actually find online, and they had to analyze them and tell me if it's real or fake. allyson ota 11:43 Um an example assessment question was to look at this article and tell me whether or not they thought they could use it in their research and tell me why yes or no. allyson ota 11:54 And then of course, I closed it out with summary to help get them ready for the post test. allyson ota 12:01 Module Three pulls them back in from the internet, because we want them using specialized library resources. So there were four in particular, these were radiology students. So I focused on the library's catalog, PubMed Central Google Scholar, and EBSCOhost health databases. allyson ota 12:17 And then there were some, of course, slides. This one, I split into two presentations, just because I didn't want a long slideshow, but this had a lot of information. And one student did. You know, note that before this module, they didn't know how to use databases, so it was helpful to them. And this was probably the most relevant to this big paper their going to do at the end of the semester. allyson ota 12:37 So how did I evaluate this? I did a usability study and a learning assessment. allyson ota 12:42 The usability study was asking, like, is this usable, before giving it to these real students? Right. allyson ota 12:48 So I had three participants, a librarian with subject matter expertise and information literacy, an instructor of the course, who also has instructional design background, and a student who had recently gone from KCC to UH Manoa. allyson ota 13:02 And I made them all go through separately, and I tried to improve the site based on feedback received, I coded 97 comments, I got way more, but these are the ones that actually made me decide what to do with the design. And these are the categories I placed them in. And I assigned them positive or negative ratings positive honestly was helpful in the sense that it told me like, Okay, you're doing something, okay, good. And I got a lot of instructional content feedback, which was really helpful. Navigation, I had the most negative to positive, it was a problematic area. And I'll show you why. allyson ota 13:37 I had them complete tasks. And as you can see, the 100% is so beautiful. And it just means all three of the participants could do it all. I was testing the functionality. So the things I coded or the multimedia objects I embedded. And this is like such a glaring problem task 4 b It's called resume progress. And I'll explain just in a sec. allyson ota 13:57 I also asked them tell me how easy it is to use, please all these different tasks, how hard or easy was it I got lots of fives, which is really good, very easy. And here on Google Forms, and Google Slides I took feedback from the first round, and got fives again, but this one Resume Progress. Nobody gave me high marks on that. Really no fives. But um, so what it was is they had a week to do this instruction, I estimated it might take about an hour and a half to an hour to hour and a half. And I wanted to give them that flexibility because it's asynchronous to like stop, and like, go do something else and come back another day. And so I had a save progress link where they could actually save their space because there's no navigation, right? And then when they came back in, they're supposed to be able to resume progress but where I was displaying that resume progress Link was what was kind of difficult so I had to go beyond this test actually on my own and try to work with um, thank you to my colleagues and friends who helped me with that. allyson ota 14:57 Overall, they gave it a 5 for ease of use, though, and I got it fixed and ironed out before giving it to the students. allyson ota 15:05 And in the learning assessment we want to know is this effective instruction? allyson ota 15:08 I had 15 participants all from the target audience 14 in the radiology course, the same student from the usability study. And just to give you a glimpse, they were young 80%, or 12 of them were under the age of 30. Three were over the age of 30, kind of what I expected. And interesting, you know, and I shouldn't be surprised because it's Community College, bachelor's and master's degrees thrown in there. They're on their way to their A.S. many of them– this will be their first degree. allyson ota 15:36 And I asked what were your preferred multimedia types? you could choose three, and they said, videos, infographics and presentations, which was a relief, because that was how the majority of the instruction was delivered. And when I asked them, allyson ota 15:48 What did you like the most about the instruction? What was cool is 11 of them just said, mentioned the video in some way. And just some feedback was they liked the choice of videos, because they were short. And to the point, they didn't get bored. They like videos and pictures versus having to read a bunch of stuff. And, you know, a visual learner said it helped make you know it more fun. allyson ota 16:08 And with the coursework, someone said it's a good reference tool for their upcoming paper ... Asynchronous: we could go at our own pace and backtrack to help answer questions. And someone said they actually took the course before but doing this asynchronously was better for them and a refresher. allyson ota 16:23 What did they not like? What did they like the least? Four didn't have any comment. Three mentioned the length of the instruction to said Padlet. One did not like the long videos, because there were some long ones. And one said it could have been longer with more info. allyson ota 16:39 I asked them if they had ever attended a library orientation or instruction, the UH system, and only 27%, or four of them said Yes, three weren't sure. And 8 said no. allyson ota 16:51 And I wanted to show you that cognitively on the pretests and post tests, they all improved, especially in module two, there was 100% average on the post test. And I'm not exactly sure why. But I did think about the fact that module two was the only module that had all the multimedia types, and the most interactivity. So maybe I would want to put that in the other parts of the instruction. Something to consider. allyson ota 17:15 I asked if they thought misinformation was a problem. Overwhelmingly, yes, nobody said they 'no' or 'not sure,' which probably made the instruction more relevant to them. allyson ota 17:23 And for their attitudes, their affective domain, I'll just tell you that the ones that had the highest improvement was that they were more willing to seek assistance when they need help after the instruction. And, after the instruction, they agreed they would apply these skills with school and personal life scenarios. So that's great. allyson ota 17:43 My terminal objective, we'll zoom in, they started around where you folks were neutral, as far as being motivated and confident. But they got up there to agreeing that they were more motivated and more confident in their ability to practice these skills. allyson ota 17:58 So overall, I do believe that the goals were met, where they did achieve, motivate more motivation and confidence by the end of the instruction. allyson ota 18:07 And some additional feedback was "I would have preferred knowing this at the beginning of the program, or even before" uh, "the modules felt long." "I wish there was a progress bar to help me gauge how much further I need to go," which is great feedback and something I would work on. allyson ota 18:22 "Before this course, it wasn't a thought in my mind to check about the credibility of the source. Now I feel I'll be more aware of where I'm getting my information from." allyson ota 18:29 "I had no idea prior to this course, how many factors that could have checked." And now they feel confident in their ability to evaluate information. allyson ota 18:37 So for future work, I learned a ton this was the first time doing anything like this. And it was pretty insane and awesome at the same time. allyson ota 18:44 But I have a lot more work to do. I have a lot more literature review to do in life. So I this is just the first iteration. I guess. I definitely want to keep revising and refining and that progress bar, you know. And you know, just pointing out that more multimedia might be something to look into or more investigation as to why this was so successful. allyson ota 19:06 A takeaway, though, was I had open ended questions in all modules, so I didn't have automatic instantaneous feedback. And so I had to grade them. And it took me a long time, like a week or two to give it back to them. And so someone said, you know, they didn't like that the pre and post test questions were the same, but also they didn't know if their answers are correct or not. So I think that's something I have to really work on. And it would be super hard to scale up and give individual feedback to like 100 students it'd be like impossible. So I gotta work on that. allyson ota 19:38 And the fact that only 27% of them have you know, recall receiving library instruction at one said that they appreciated the refresher, lets me know that a lot of our students are not getting this kind of instruction, and they're... these students are graduating they're leaving, so, I think it's important, and I will be talking to people on my campus about my results. We are actually using my project to assess So the library's SLOs this year. allyson ota 20:03 But these are my references. allyson ota 20:05 Thank you to Kimberly the director of the radiology department Kelly and Tyler my awesome critical friends, Dr.s Fulford, Hoffman, and Patrick for this last year of intenseness. Dr. Peter and Dr. Paek for being awesome. At the library, Erica and Jessica and Joyce for all your help my friends, Rich and Justin, for having to endure me through this and my extended critical friends Chelby, Judy Byron, Kawika, Jessie and JR. And of course, my niece Karma and my sister Roxanne. allyson ota 20:36 And with that I leave you, mahalo. Dr. Dan 20:40 All right. Congratulations, Allyson, you're getting a shout out for your multimedia. Thank you slide in the comments, which I love. All right, let's open it up to questions. Feel free to raise your virtual hand or your physical hand or put something in the chat. Okay, we've got a few hands up. I'll start with Dr. Mike. I see him first. Dr. Mike. Dr. Mike 21:07 Thank you very much, Dan. Wonderful presentation to end TCC with. So thank you very much, Allyson, as I was one of the people last summer involved in the general education redesign, you might want to avail yourself of that particular group. I don't know if you're involved with that system wide Library group that kind of talks about these things. But this, this would be a perfect example of some of the interior materials that might be included in whatever happens with the gen ed, you know, we're, we're suggesting some kind of a lab or something like that, that would include information literacy with some modules. But certainly it's this is the kind of information that our students desperately need. And you're absolutely correct on that the students are not getting this info. So I'm just wondering, from your perspective, where do you think the major disconnect is as to why they're not being exposed to this? Because it really is supposed to be prevalent, but it doesn't seem to be the word doesn't seem to be getting out. So just curious, your thoughts? allyson ota 22:22 Well, right now, it's optional. So the only instructors who assign or ask for library instruction, like those are the only students we work with. If I were to just put this on my, on our library's website, I don't think many students would be motivated to like, do it. So we do need that faculty and librarian, you know, partnership, I think, and you know, students are motivated by passing in their classes, ultimately, when they're in college. So I think if it did become a Gen Ed requirement, though, then that would definitely be a great way to get it out there. Well, thank you. Dr. Dan 22:59 Thanks for the question. Dr. Mike. And interesting answer, Allyson, you know, what really strikes me is not only did you see gains in the cognitive outcomes, students clearly learned, but in some of the open ended comments you shared, it seems this really resonated with them. I was really impressed with how open they were like, yeah, these are valuable skills. I'm kind of glad I have them now. And I take that as a very good sign. Yeah, allyson ota 23:30 I'm pleased. Dr. Dan 23:31 Yes, sure. Sure. I see. Kelly also has her hand up, Kelly. Kelly 23:37 Hi, Allyson. Great job. Thank you. Okay, I'm gonna go. But you did wonderfully. And I'm so proud of you. Definitely, that you coded everything. That is just amazing work on your design. Congratulations. I have a question for you. You designed informational literacy for a specific group of RAD students. Do you think informational literacy could benefit and be utilized for a wider audience? And if so, would you use similar multimedia types? To reach this audience? allyson ota 24:15 It's such an interesting question. Because so so many audiences are so are so different. I don't know if this like GIFs. And stuff would resonate with some audiences, but I felt kind of safe doing it with my audience. But, you know, for the public, I initially was interested in making something for the public. So it would just have to be customized so that like in the case of module three, like all those databases are only accessible to our campus. So maybe like it would have to like look at the audience and see what they have available to them, and then show them free online databases they can use or public library databases that they can use. And just like by the subject area, there are different types of resources too. So that's why there's so much customization every time we get a request. To do instruction, actually, but I think it's done definitely possible like to to scale it to like larger or different audiences. I hope that answers your question. Yes. Thank you. Dr. Dan 25:12 Thanks for the question. Kelly. Allyson, actually, Dr. Ari's asking, Is this public? And are you to open to folks linking to it? So maybe you want to speak a little bit to that. allyson ota 25:23 It's not public... Well, I mean, it is public in that it's hosted on Laulima. And it's, you know, you type in a username, but it's not a password protected thing. Um, so like, you can just, it's just, you can just play around with that. If you're interested. I can put that link in the chat. But yeah, I guess, because I still feel like it's in this ... I'm still doing the assessment and everything. I don't consider it like a completed work, even though I students had to do it in Kimberly's class. But, um, but yeah, I definitely could make it available. I mean, kind of is actually. Dr. Dan 26:02 yeah, it's up to you, Dr. Ari is pointing out that there's part of our LTEC 112 class that we teach, which is technologies for academic success, which has a lot of overlap with, obviously, with information literacy. So if you are open to that, I think there is a place in that course where we might be able to fit it in. And, of course, we all understand that it's, this was a formative evaluation, and that you have plans to carry it forward. But that's a decision for you. i This will be our last question. I see. Tyler has his hand up. And Tyler. Hello. Tyler 26:40 Good job. So, Allyson, my question for you. Allyson do you see yourself using like asynchronous or multimedia approach again, like in the future, if you do other modules and stuff like that? allyson ota 26:53 Yes, I do. Um, I personally like the convenience of asynchronous instruction, or, you know, online instruction as a full time person who works. And I am going to be co teaching a class this summer, that's hybrid. So I might try to bring in some of these elements. But I don't have a whole lot of time to ramp up to that. So I'm not sure how crazy I'm going to go into that. But I won't have two semesters to work on it. I should say. But yeah, I definitely would be interested in I love all the theories I was reading, and I definitely want to read more. So yes, I'd like to learn more about it. Thank you. Dr. Dan 27:34 I know, a doctoral program that where you could read a lot more about it if you're interested. Just teasing. Tyler, thank you for the question. Allyson. There's a lot of comments and questions and kudos in the chat. So be sure to check those out. But it is 658, which means folks, we have reached the end of TCC 2022. And I'd like to give Allyson a round of applause for being our anchor student this year, closing out the entire worldwide conference. Congratulations Allyson. Thank you so much. And I'll just close with a formal few words on behalf of TCC Hawaii, Learning Times, and the Learning Design and Technology Department at the University of Hawaii at Manoa. Mahalo for your participation, and we hope to see you again next year. Thanks, everyone. Oh, and I'm, I'm sorry, I'm noticing that Dr. Fulford has her hand up. Dr. Fulford I, Dr. Fulford 28:39 I just wanted to say congratulations, everybody. You were awesome. So I'm so proud of all of you. And I can't wait to see you at the graduation ceremony. I hope you'll all be there. Dr. Mike 28:49 How about a screenshot? Dr. Paek 28:51 Yeah! Dr. Dan 28:52 Yes, yes. Okay. On three everybody. Look at your camera, fix your hair, and smile. 1,2,3 good. Let's do one more big smiles! It's the end of TCC! 123. All right, very good. Excellent. Thank you so much, everyone. It's been an absolute blast. working with all of you Transcribed by https://otter.ai