Author(s): Lauren Kuryloski, Assistant Professor of Teaching, University at Buffalo
Macy McDonald, PhD Candidate, University at Buffalo
Role(s): Professor and Teaching Assistant
Data Visualization Grading Rubric
What did you want students to be able to do by completing this assignment?
The assignment has three primary objectives:
1) To provide students with practice in composing clear, accurate, and correct data visualizations;
2) To allow students to think through the process of how to tell ethical and persuasive stories with data and;
3) To give students an opportunity to strengthen their ability to develop data visualizations and written content that is accessible to a wide variety of users.
Was there anything this assignment taught students that you felt they wouldn't have been able to learn through other types of class assignments?
While students in the technical disciplines often have experience with creating data visualizations, they do not always have training in analyzing data displays as rhetorical artifacts. Our assignment encourages students to not only create visualizations that are accurate and clear (necessary features for an effective display), but also asks them to think about how persuasive arguments are made with numbers and visuals.
In addition, the assignment asks students to consider the accessibility needs of their potential audience(s) and create data displays that would be accessible for a wide variety of end users. Creating accessible content is given as a foundational element of effective technical communication in the very first chapter of the course textbook, and our assignment provides an opportunity to put theory into practice.
What is the course title and level?
EAS 580: Technical Communication for Engineers is a Master’s level technical communication course. The assignment would be appropriate for undergraduate and graduate-level courses.
What kind of prior knowledge is necessary to complete this assignment? How do students gain this knowledge?
Students should have a basic understanding of how to create data visualizations using tools like Excel or comparable data visualization programs. Given that this is a Master’s level course, most often populated by engineers working in industry, we assume that most students arrive in the course with this knowledge via their academic training and on-the-job experience.
The assignment has three distinct phrases: composing an initial draft, peer reviewing, and revising and resubmitting the final submission.
In the initial drafting stage, students are presented with a short case study assignment that places them in the position of mid-level managers at an engineering firm. They are told that senior management at the company is interested in creating a series of workshops focused on diversity, equity, and inclusion (DEI) in the workplace. If instituted, the workshops would be required for all employees and managers, and they would be aimed at increasing both recruitment and retention of diverse employees, including engineers with disabilities. The students, acting as middle management, are tasked with reviewing existing data on employment rates in the STEM professions and coming up with a presentation on the need for the workshops. Using employment data from the NSF, they are asked to create two data visualizations that can be used in the presentation.
In addition to the visualizations, students are asked to write one paragraph per visual justifying the choices they made. This component provided students with an opportunity to speak directly to the course professor/TA, explaining the technical, rhetorical, and accessibility choices that they made in the visualization. It should be noted, though, that while the written explanations accompanied the data visualizations, the visualizations were expected to function as stand-alone pieces. Truly effective visualizations make a clear and persuasive argument on their own and do not have to be overly explained to the audience. Put another way, a data visualization should be able to tell its own story.
After exchanging feedback with peers, students move into the final revision stage. They are asked to make revisions based on the feedback they received, as well as to revise with the goal of making the data displays more accessible. Students must make two changes to improve accessibility and are provided with options to assist them in the task. This helps reinforce that good design is accessible design.
How much time did you allot to this project?
Students were given a week to complete the assignment, which included the following components:
25 minute genre study video lecture.
Assigned reading of chapter four “Displaying Data” in Practical Strategies for Technical Communication.1
30 minute synchronous meeting:
Our class typically enrolls anywhere from 45-65 students, but we meet with all students weekly in small groups of 6-7 to discuss their assignments for the week.
20 minute “Just in Time” video in which the instructor anonymizes and then critiques a sampling of submissions, thus providing feedback to guide revisions.
Three-part process-based writing assignment:
Initial data visualizations with writing component.
Two peer reviews.
Final revised data visualizations.
We estimate this takes our students 4-6 hours over the course of the week.
What kinds of support or training did you provide to help students learn to use new techniques or specialized tools?
As our students are pursuing Master’s degrees in engineering management, we do not provide additional training with these tools, but instructors with less experienced student populations may need to demonstrate how to create graphs and charts in Microsoft Excel, RStudio, or comparable data visualization programs.
Did you need any specialized equipment, tools, or human resources to make this assignment feasible? If so, please describe.
Students will require access to Microsoft Excel, RStudio, or comparable data visualization programs.
Instructors will also need to supply an example data set. We drew our data set from the National Center for Science and Engineering Statistics, which is managed by the National Science Foundation. Pew Research Center is another excellent source for clean data sets.
How did you assess or grade this project?
The grading rubric is attached. In this assignment, we were most concerned with three of the core course concepts: accuracy, clarity, and correctness. We also assessed students based on their application of design principles, including fundamentals like whether or not they appropriately labeled their axes or provided descriptive titles. Students were trained in and required to incorporate at least two accessibility features into their final design. Accessibility features include, but are not limited to:
using patterns or distinctly saturated colors to ensure accessibility for different levels of color blindness;
We suggest students preview their visuals in grayscale or use this color blindness simulator to check their work.
providing direct labels;
including textual descriptions of images and graphics for screen reader programs;
resizing graphics and labels to ensure readability.
Finally, they were assessed on how thoroughly their visuals demonstrated their understanding of the relationship between data and persuasion. Students who produced clear, readable data visualizations without an apparent connection to the larger rhetorical situation received approximately half credit. Students are given an opportunity to articulate their understanding of course concepts in the justification paragraphs that accompany each data visual. These paragraphs help instructors better assess whether or not the visualization was achieving its aims as outlined by the author. Although we should note that their aims need to be in line with the assignment prompt. For example, it’s clear from the data set we supply that people with disabilities are statistically underrepresented in STEM fields. Occasionally, students will react to this information by creating data visuals focused on how the field they specialize in is more inclusive than another STEM field. These visuals may be well designed, but they rarely effectively respond to the prompt (which is that their workplace is creating DEI workshops). Inclusion isn’t a competition.
If you assigned this project again, would you change anything? If so, what?
In a previous version of this assignment, we had students design one data visualization to support the argument for DEI workshops and one to oppose the workshops. We did this because we wanted the students to see how data can be accurately used to support different arguments, if not correctly. However, given the number of skills we were also focused on building in that week, it was quite challenging for them and also rather unpopular as the vast majority of students didn’t want to oppose the workshops and disliked cherry-picking the data. Ultimately, we decided our students’ data literacy was sufficient for understanding how the same data points could be used in different arguments, but we would like to think of more ways to teach our students to spot cherry-picked or misused data just from the visualizations in a presentation and have considered ways to include that lesson in this assignment.
Describe any trouble spots or complications someone else might want to be aware of before trying a similar assignment in a course of their own.
We begin the week with data visualization principles that build upon previous weeks in document design. Thus, as written the current assignment doesn’t cover basic principles of design. Instructors looking to implement this assignment will likely need to take that into consideration when building the assignment into their schedules.
Additionally, in between the initial assignment and the final submission we provide them with a lesson in inclusive design principles, such as: including textual descriptions and direct labels for e-reading programs; running color-blindness checks; redesigning charts and graphs with textural or saturation differences rather than solely color; and resizing fonts and labels.
While these are necessary lessons, it can sometimes have the effect of encouraging students to revise only for accessibility rather than understand that they need to revise for accuracy and correctness as well.