Exams

We’re a few weeks into the semester. Most courses are in full swing, and students likely have submitted their first assignments or will do so shortly. For those courses with exams, the exams are coming (if they haven’t already arrived!).

Here are a few good resources about creating multiple-choice tests. First, from the eLearning Coach Connie Malamed, 10 rules for writing multiple choice exams. I’m generally rule-averse, but I do think that these are wise guidelines. I’m not as opposed to “All of the above” and “None of the above” questions as she is, but I think this varies with your subject matter and learning objectives.

As an instructor, it’s hard to write multiple-choice questions where the right answer is clear and the incorrect answers are wrong, yet close enough to be plausible. Questions can’t be too easy, but no one wants a test where the possibility of misconstruing answers leaves students feeling as if the exam was intentionally misleading. Consider this quote from the article cited below:

The thing about multiple-choice questions is that the answer is right there on the screen. So the challenge as question-writers is to construct the question and its answer choices in such a way that the learner really has to master the objective in order to select the correct choice.

So how do you write really, really good questions? What do bad questions look like? And how do you use the data from students’ incorrect answers to help you build better questions in the future? This article is full of good examples, tips, and references.

This is also a great resource on writing multiple-choice questions for higher-order thinking. I find it exceptionally helpful to see both the “standard” question and the question that has been re-worked to stress higher-order thinking skills.

Think multiple-choice questions function best in subjects that put a premium on calculations? This is a great article detailing the mechanics of using comprehension-based multiple-choice questions in a communications course. I love the idea of letting students bring in a sheet of handwritten notes; the cynic in me would either keep these notes or perhaps mark them in some way in order to prevent re-use by future students.

Last, it feels fitting to offer a small reminder about resources for creating rubrics. Rubrics are useful for essay, short answer, or performance items. You might first write something that summarizes your own grading observations, then revise the rubric and share it with students as you communicate information about the exam. If you’re not distributing the question in advance, you can still share the rubric, removing any identifying details that would give away the question or skill to be demonstrated. You may find that using a rubric makes the writing or demonstration process more comprehensible for your students, and the reading or grading process easier and more consistent for you.

Of course, exams aren’t the only way to measure student learning (hardly!). We have posts coming on project-based learning, groupwork dynamics, and alternate research assignments. Stay tuned!

Mistakes and Feedforward

Recently, I’ve come across two interesting perspectives on making mistakes.

First, from the TED Radio Hour, this is a great radio show about making mistakes. In addition to being a really captivating human interest story, the radio broadcast also has a lot to say about learning. Views on mistakes come from a physician (“most of the greatest successes in medicine come from failures”), a noted psychology researcher (“if failure is not an option, then we just have a bunch of scared people hanging around loitering on the outside of the arena”), a jazz musician (“a mistake is an opportunity that was missed”), and a corporate coach (“a mistake offers the greatest amount of insight and the largest room for improvement”). This one is really worth a listen.

Second, I found this video in the course of researching something else, and the snippet below caught my attention.

Although the golf example may not be applicable to you, I suspect that many of us are guilty of folding up too quickly in the face of failure. It’s easy to shut down and turn away. When you know your efforts have gone awry, what do you do? What should you do? Letting the scenario play out with a dispassionate eye, observing what happens, and then reflecting on events and devising a new plan are all challenging skills on their own, never mind in the face of your own mistakes.

How can we help students turn their mistakes into valuable learning opportunities? Feedback is key, of course. I’m assuming that timely and individualized feedback is already part of your teaching practice. But what about the content of this feedback? When instructors give feedback, many naturally focus on the assignment in question. While valuable, reviewing the work the student has done is retrospective feedback. Students may be at a loss about how to translate your analysis into actionable steps for the next assignment. Using your feedback becomes more complicated if the next assignment has a different topic or format. What about offering prospective feedback? That is, feedback with specific attention to the work the student will do in the future? I’d like to share with you the idea of feedforward:

Feedforward is another interesting way to think about the feedback offered students. This more future-oriented feedback responds to what the student did, but in light of what needs to be done on the next assignment. Rather than isolated comments, it distills the feedback into three or four specific suggestions that target what the student should work on to improve the next assignment. . . .

Perhaps students would make better use of our feedback if they used that feedback to develop an action plan for the next assignment. “Based on the teacher’s feedback and my own assessment of this work, here’s the three things I plan to improve in the next assignment.” Maybe students don’t get the grade for the first assignment until the action plan has been submitted. And maybe that action plan then gets submitted along with the next assignment. 

Turning past performances into future successes is tricky. Recognizing mistakes and devising a plan for improving upon them requires both meta-cognitive skills and content-specific knowledge. This is the real work of learning and teaching – and where feedback and coaching can play such a crucial role.

How have you helped students to constructively use your feedback? Do you have any strategies that help students draw lessons from their own (or their classmates’) mistakes?

Happy New Semester!

For me, the start of a new school year is a much more significant marker of time than a new calendar year. I like a good New Year’s party, but, in the spirit of getting down to business, let’s talk about resolutions for the new semester. (I’m a super-fun party guest, I promise!)

Do you make resolutions for the start of the school year? Do you have some this year? We’d love to hear about them in the comments!

Or are you open to the idea of mixing things up, trying something new, collecting new data, etc., but aren’t quite sure where to begin?

Continue reading

The Students are Coming, the Students are Coming!

These may be the dog days of summer, but the start of school is just around the corner. I don’t mean to be a vacation killjoy: although the first few weeks of the term are busy and stressful, I actually like the excitement and novelty of it all.

In the coming weeks, we’ll have a handful of posts up about relevant back-to-school teaching and technology topics. You can also peruse our entries from last August for posts on everything from Pearson LearingStudio announcements and instructor bios to tips about Google use, uploading audio files, and your syllabus layout. It’s good reading, I promise!

In the meantime, let’s talk about the students.

First, it doesn’t look like the mindset list has been updated yet for the class of 2017, but last year’s list is still there. I promise an update when the new list goes up. As informative and entertaining as the mindset list is, I can’t help but feeling that it also tells us a fair amount about the authors’ own mindsets.

Moving right along, here’s a great graphic illustrating the twelve must-have skills of modern learners.

If students need these skills, this really means that they need to be provided opportunities to practice these skills. How do the big (and small) activities in your courses help students develop these mindsets? Are there ways you could tweak your course activities in order to include one or two of these elements on the way to meeting the course learning objectives?

For example, to give students the opportunity to develop empathy and global stewardship, you might ask them to think about the supply chain for products they are using or the products in their project proposals. What are the ramifications of those choices? Are changes desirable or feasible? To give students practice in developing effective oral and written communication skills, you might consider blogs, tweets, memos, or project pitches as alternatives to the standard papers and presentations.

This list purports to summarize what online students want. However, the key findings – more collaboration, multimedia feedback on a regular schedule, and clear waypoints and guidelines – need not be limited to online or blended learning students. In fact, I think these would be helpful for all students. Best, the brief article lists some concrete ways that instructors can incorporate the above practices.

With all the discussion of digital natives, it’s easy to assume that students are already experts in the learning technologies you are using in your course. This isn’t always the case. In fact, one the questions we are asked most frequently is how students view instructor comments in the Pearson LearningStudio gradebook. In general, without over-explaining or talking down to your students, it’s safest to assume that they may not be familiar with all the features of whatever digital resources you’re using; this goes for first-generation college students and others. As the instructor, you’ll want to provide a basic introduction, and then make sure that help and how-to documentation is available as students set out to work independently.

In closing, I want to share with you this intriguing TEDx video on the fallacy of designing courses with the average student in mind. The speaker is Todd Rose, faculty member at the Harvard Graduate School of Education. Rose reviews the experience of the U.S. Air Force in designing fighter planes: after intensive research, the Air Force discovered there was no such thing as an “average” fighter pilot. You see where this is going, right? Rose has some great insights about how damaging the notion of “average” is for learners and how we can best nurture the potential of every individual in the classroom.

If you want to think about Rose’s points further, David Hopkins has a nice slide comparing the average fighter pilot and the average student.

Case Studies

Case studies are a great way for your students to review material, practice authentic skills, and synthesize course content. A cleverly written (or ripped from the headlines) case study also provides a valuable active learning opportunity. You can use case studies to introduce a topic or to review material.

Depending on your format, content, and the complexity of the case study, students can work through case studies in a class discussion, a homework assignment, or group assignment. Case studies don’t have to take a lot of class time, although many instructors feel that some element of (class or group) discussion greatly enhances the case study learning experience.

Carefully structured student interaction ideally shifts student focus from racing to find the one right answer to instead reveling in the process of data analysis, applying context-specific knowledge, and weighing the relative importance of key factors. After all, these latter behaviors represent the transferable skills from this exercise. The case study solution is only a best outcome for one single scenario; the process of applying course content while in problem-solving mode is the gift your students will hopefully share with others down the road.

In planning for student interaction around a case study, this list of insights about the case study discussion process from a professor of strategic management at Harvard might be helpful. Likewise, this piece discusses how individuals might take on different roles in the decision-making process, a process that more closely mimics how many organizations make decisions. If you choose to employ the roles of leader, decision-maker, and advisers, a random assignment of roles is probably most efficient. My two favorite in-person ways to randomly assign roles are via playing cards (for example, the student who draws the ace is the leader, the student who draws the king is the decision-maker; and the rest of the students with numbered cards form the advisory panel) and picking numbers (student who has selected the highest number is the leader, the student with the next-highest number becomes the decision-maker, and all other students are advisers). For online courses, you might use a random number generator and post the results for your students to see.

In closing, I also love to throw a cognitive in wrench in things: once students have worked out what the best course of actions is for a scenario with x,y, and z; I then ask about w, x, and y (or a,b, and z). Does that change the outcome? If so, why? This wrap-up piece is great fodder for a class discussion or a written reflection on the activity.

Teaching with Simulations and Serious Games

Check out this list of online educational simulations / serious games. Grouped by topic, you can find simulations related to biology, leadership, politics, healthcare, city management, ethics, history, and marketing, just to name a few available topics. Summary information about each simulation is provided. While some entries link directly to the simulation, others contain a link to the developer charging educational licensing or individual user fees.

How might you use a simulation or serious game with your students? An activity like this is a wonderful way to start the course or introduce a specific unit, grabbing student attention and familiarizing them with the content. Likewise, you could use a simulation as a concluding activity, letting students demonstrate their knowledge by playing. Students can then reflect on the game itself and how well it captured reality. In a face-to-face class, you can use the classroom computer to play together as a class – or use student devices to have groups play together. Alternately, and for online courses, you can have students play on their own and then share their reactions in a discussion or short writing assignment.

Is the simulation at the right level for your students? Note that in the example use  scenarios described above, the level isn’t as important as what students do with the knowledge: get interested, evaluate, problematize, etc. Of course, the simulation can’t be impossibly simple or hopelessly intricate, but, barring those conditions, there is still a lot of learning to be done from a simulation that isn’t matched exactly to the level of the course.

Do you have an online simulation that works well with your students? What about a classroom-based simulation? Personally, I’m a big fan of teaching the tragedy of the commons with a classroom-based simulation. Please share your favorite simulations / serious games in the comments!

Bloom’s Taxonomy in the Smartphone and Tablet Era

Bloom’s Taxonomy, the prism through which many of us have been taught to evaluate learning outcomes, is changing. The general trend is to focus more applications of knowledge, rather than knowledge for its own sake. In the updated version, the act of remembering replaces “knowledge” at the base of the triangle. Progressively more intense higher-order thinking skills are enumerated moving upward toward the peak of the triangle, culminating with learners creating knowledge. The act of synthesis, absent in the revised taxonomy, likely has always found its way into the work of analysis, evaluation, and creation.

Of course, Bloom’s taxonomy is not the only way to graphically represent the cognitive processes associated with learning. Other revisions have represented the taxonomy as a series of interlocking gears, a blooming rose (hah!), and a feathered bird – just to name a few. Notably, Rex Heer at Iowa State’s Center for Excellence in Teaching has created a nice flash model of the way that the cognitive processes mesh with different levels of knowledge that learners will need to complete the task. It’s definitely worth clicking on the link in the previous sentence and then playing a little with the interactive graphic.

Recently, I’ve run across a few revisions of Bloom’s taxonomy focusing just on the apps available for tablets and smartphones. Check out the Padogogy Wheel:

I love the multitude of apps identified above – what better way to find something new that just might work for you? However, if you’re looking for something a little cleaner, here’s a nice distillation:

Of course, not all the apps are appropriate for your content, level, or course goals. You might use the above information to guide students when helping them select research, presentation, or study tools. If you’re currently using – or contemplating using – one of the apps identified above, this should give you a sense of the cognitive range of the app. Last, if you’re not yet sure how smartphones and tablets fit into the classroom and individual learning processes, hopefully these graphics have provided some food for thought.

If you’re looking for still more apps, the Koehler Center has a list of both discipline-specific and general study, writing, and document management apps. In addition, some of the web 2.0 tools identified on our website also have their own apps.

Apps on tablets and smartphones are more than just cool tools. In particular, some apps make it very easy for students to transition from sophisticated curators of knowledge of innovative creators of knowledge – that is, to ascend to the highest level in the revised Bloom’s taxonomy.

Do you have an app you love to use with your students? What impact does this app have on student learning? Alternately, is there an app that would benefit your students’ learning that you’d love to create?