In 2013, I wrote a Literacy Daily post about a new National Assessment of Educational Progress (NAEP) assessment then in development—the Technology and Engineering Literacy (TEL) assessment. I was excited about the TEL because, unlike the biannual NAEP reading assessment, the TEL promised to tell us something about U.S. students’ 21st-century digital literacies—such as their ability “to employ technologies and media to find, evaluate, analyze, organize, and synthesize information from different sources.”

The TEL was first administered nationwide in 2014, and then again four years later in 2018. The 2018 results are now available, and together with the 2014 results, and side-by-side with the NAEP reading assessment data, they make for interesting reading. Following are some highlights.

• In 2018, 46% of eighth-grade students performed at or above the Proficient level on the TEL assessment. (By comparison, in 2017, 36% of eighth-grade students performed at or above the Proficient level on the NAEP reading assessment.)
• From 2014 to 2018, eighth graders’ scores on the TEL assessment rose significantly—from 43% at or above Proficient in 2014 to 46% at or above Proficient in 2018. (By comparison, eighth graders’ achievement on the NAEP reading assessment has stalled—dropping from 36% at or above Proficient in 2013 to 34% in 2015 and then recovering to 36% in 2017.)
• On the TEL, as on the NAEP Reading assessment, female students outperform male students. Further, from 2014 to 2018, the gap between female and male students’ performance widened significantly, with the average score for male students increasing only slightly over that four-year period, and with the increase in the average score for female students accounting for most of the overall growth from 2014 to 2018.
• From 2014 to 2018, the percentage of black students scoring at or above Proficient on the TEL assessment jumped significantly from 18% to 23%. (By comparison, from 2013 to 2017, the percentage of black students scoring at or above Proficient on the NAEP reading assessment increased only slightly, from 17% to 18%.)
• On the TEL, the connection between parental education level and student performance appears to be weakening. In 2014, students whose parents finished high school significantly outperformed students whose parents did not finish high school. In 2018, that gap had closed. (By comparison, the NAEP reading results from 2017 show a significant gap between these groups—unchanged since 2013.)

What does it look like to score at or above Proficient on the TEL?

The TEL assesses eighth graders’ technology and engineering literacy through interactive scenario-based tasks. Students are challenged to enact “practices” that include “analyzing information,” “developing solutions,” and “communicating and collaborating.”

Examples of tasks on the TEL include the following:

• Creating website content to promote a teen recreation center.
• Evaluating provided information to explain how to fix the habitat of a classroom iguana.
• Selecting images to be used on a website advertising a television show about the Andromeda Galaxy and correctly citing an image’s source.
• Developing an online exhibit about Chicago's water pollution problem in the 1800s.

To do well on these tasks, students need to show they can perform the following tasks:

• Gather information through browsing and searching.
• Identify distortion, misinterpretation, or exaggeration of information.
• Analyze and evaluate information or data to solve a problem.
• Adjust a text’s content based on knowledge of audience and the communication method being used.

All these complex skills mobilize and build on more foundational reading comprehension skills and strategies. For example, to answer a question that asks students to read first-person statements from a variety of stakeholder perspectives about the pollution of the Chicago River and then match those statements to third-person “summary descriptions” for an online historical exhibit (see the screenshot below), students first need to comprehend the gist of each individual statement on its own before connecting it to a “summary description” that restates that gist in more general terms.

Similarly, for a question that asks students to choose and sequence audio clips to accompany an animation showing the stages of Chicago’s response to its water pollution crisis, students need to be able to put narrative segments in chronological order before choosing the particular segments that most clearly and helpfully describe what is visually depicted in each of the four consecutive parts of the animation.

To learn more about TEL assessment items, consider visiting the NAEP TEL website. Among other things, it provides a number of “live” sample tasks that you can attempt as if you were a student taking the TEL—with the difference that your submitted answers will be instantly scored and returned with feedback. For each scored item, you can also see the percentage of eighth graders who answered that item correctly.

Why do we need two NAEP assessments to inform us about how our students are reading?

As you explore the TEL and the insights it offers into how U.S. students are developing as 21st-century readers, you may find yourself asking why the NAEP reading assessment hasn’t evolved to keep up with the times. Why do we now need to consult two NAEP assessments to find out how U.S. students are reading?

One plausible answer is that, to maintain the value of the NAEP as a barometer of long-term trends in literacy achievement, it’s important to be cautious about making changes that could compromise this function. If the 2019 NAEP reading assessment were abruptly revamped, we could no longer compare the 2019 scores with scores from previous years.

This being the case, however, we might expect the NAEP reading assessment not to abandon more traditional genres, text formats, and reading tasks, but rather to expand its scope to include additional items focused on 21st-century digital texts, skills, and reading tasks.

There are indications in the the 2017 NAEP Reading Framework that, starting this year, we may finally begin to see some movement in this direction. There are plans to gradually shift to “digital administration” of the reading assessment on NAEP-provided tablets and to include some items reflecting “a wider range of texts, including those coming from digital sources that may involve dynamic features such as video, animation, or hyperlinks.”

In the meantime, I am grateful for the TEL assessment. It contains valuable data and insights. And the fact that it’s a separate NAEP assessment that now needs to be referenced whenever we talk about “the NAEP’s latest reading results” is, upon reflection, not a bad thing. It serves as a reminder of what has been true all along—that reading takes many forms, that we need to keep revisiting and expanding our definitions of literacy, and that single assessments tend to give only a partial picture of what our students know and can do.

Paul Morsink is an assistant professor in Reading and Language Arts at Oakland University in Michigan.

This article is part of a series from the International Literacy Association Technology in Literacy Education Special Interest Group (TILE-SIG).