University of California, Irvine
AC-6: Sustainability Literacy Assessment

Why are you taking this course? In a few sentences, please describe what sustainability means to you. Please indicate the extent to which you agree or disagree with the following statements: Please indicate below the extent to which you agree or disagree with the following statement. Some people describe the feeling of “awe" as a response to things perceived as vast and overwhelming that alters the way they understand the world. Please tell us your experience with feelings of “awe”. Select one answer for each row. Which of the following is an example of sustainable forest management? Which of the following is the most commonly used definition of sustainable development? The wealthiest 20% of people in the U.S. own approximately what percent of the nation’s privately held wealth? What is the most common cause of pollution of streams, rivers, and oceans? Ozone forms a protective layer in the earth’s upper atmosphere. What does ozone protect us from? Where does most of the garbage in the US end up? What is the name of the primary federal agency that works to protect the environment? In the U.S., what do we currently do with the nuclear waste generated by nuclear power plants? What is the primary benefit of wetlands? Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans? Higher levels of education generally lead to... Many economists argue that electricity prices in the U.S. are too low because… Which of the following countries has now passed the U.S. as the biggest emitter of the greenhouse gas carbon dioxide? Which of the following is a leading cause of the depletion of fish stocks in the Atlantic Ocean? Which of the following is the most commonly used definition of economic sustainability? Which of the following is the primary reason that gasoline prices have risen over the last several decades in the U.S.? What are the potential effects of global climate change? Imagine you are one of the many fishermen who rely on the fish you catch from Lake Erie as your main source of income. The Fisherman Council determined that each fisherman must limit his/her catch to 5 tons per year to maintain the fishery. You decide to catch 6 tons of fish this year. What could be the results of your decision? A.       You make more money this year than you would have if you caught 5 tons of fish B.       You make less money this year than you would have if you caught 5 tons of fish C.       The total number of fish that are available to catch each year could decrease D.       Fishermen, including you, could go out of business The most significant driver in the loss of species and ecosystems around the world is... Which of the following is the best example of environmental justice? Of the following, which would be considered living in the most environmentally sustainable way? What factors influence the human population's impact on Earth A.       Size of the population B.       Amount of materials used per person C.       Use of technology that lessens our impact Using resources, like fossil fuels, can create economic growth. However, future generations may be disadvantaged if the current generation overuses these resources. Which of the following principles can we follow if we do not want to disadvantage the next generation? The best way to support a local economy, such as the economy of Irvine, is to buy goods (groceries, clothing, toiletries, etc)… Which of the following statements about water is true? Imagine that we had to pay for all the costs associated with the goods we use every day. What would go into calculating the true costs of a product? Put the following list in order of the activities with the largest environmental impact to those with the smallest environmental impact: A.   Keeping a cell phone charger plugged into an electrical outlet for 12 hours B.   Producing one McDonald's quarter-pound hamburger C.   Producing one McDonald's chicken sandwich D.   Flying in a commercial airplane from Washington D.C. to China Workers around the world face a variety of social injustices, including low wages, poor working conditions, and lack of access to education. To help improve conditions for these workers you can: Which one of the following statements describes you best? Select one answer. Thinking about things you do in your day-to-day life, how often do you do each of the following? Thinking about things you do in your day-to-day life, how often do you do each of the following? Thinking about things you may do in your day-to-day life, when purchasing goods, how often do you do each of the following? Thinking about things you may do in your day-to-day life, when disposing of food and goods, how often do you do each of the following? Thinking about your typical behavior when you drive a car, how often do you do each of the following?

The Sustainability Literacy Assessment has been conducted repeatedly (2013-2017) as part of the Sustainability I course offered in winter quarter through the UCI School of Social Ecology's Urban Planning and Public Policy Department. The initial assessment survey was developed by teaching assistant Sally Geislar, Professor Richard Matthew, and researchers from the Center for Unconventional Security Affairs (CUSA), and was administered to Sustainability I students by teaching assistants. The survey was designed to capture changes in knowledge about sustainability challenges, as well as ways to create solutions to these challenges (ecoliteracy), attitudes, values, and personal behaviors as a result of participating in the course lectures, projects, extra credit events, and coursework. The surveys collected both quantitative and qualitative responses from students. Initial survey questions were compiled and developed using a series of existing questions and scales in the fields of environmental psychology and environmental education, as well as polling foundations and environmental non-profit organizations. The section on eco-literacy was based on a 17-question survey, developed by the National Environmental Education and Training Foundation and Roper Public Affairs, intended to measure Americans' knowledge, behavior and attitudes related to the environment. Questions on attitudes reflect an abbreviated version of the New Ecological Paradigm (NEP) developed by Riley Dunlap and William R. Catton Jr. in the 1970s and used widely since then. To capture values related to environmental concern, an abridged version of scales developed by Paul C. Stern was compiled. Finally, to assess personal behaviors related to the environment, questions were collected from Florian Kaiser's General Ecological Behavior scale as well as the sustainability behavior change program of a non-profit organization called Community Sustainability USA, Inc. In Winter Quarter 2016, the assessment was modified to reflect the components of the ASK sustainability assessment. The new assessment focuses more on sustainable development literacy than eco-literacy and includes a question on awe disposition (because research suggests awe opens people up to new information, i.e., learning), a connectedness to nature scale (because many of the research projects assigned this year have to do with the positive benefits of nature), and changing the eco-literacy EPA assessment to a more holistic sustainable development assessment, which is a better reflection of the course content, and which mimics the assessments done by other universities (i.e., Ohio State and University of Maryland).

The first Sustainability Literacy Assessment was administered in Winter Quarter 2013 to the Sustainability I student cohort; follow-up data were collected from the same cohort at the end of the Winter Quarter. Subsequent assessments and follow-ups were similarly conducted in Winter Quarters 2014 and 2015, using the same assessment tool. The Winter Quarter 2016 assessments and follow-ups were administered online, as was the Winter Quarter 2017 assessment. Using a pre-test, post-test methodology, students completed the pre-test survey before exposure to the treatment; in this case, the Sustainability I course itself is considered the treatment or intervention. To assess change over time, the responses of students who completed the survey prior to the first class were matched and compared to their post-test survey responses. Unfortunately, a control group was not able to be established for comparisons with the treatment group. Students enrolled in the Sustainability I Winter Quarter courses were asked to complete the online survey before attending the first class to capture baseline eco-literacy/sustainable development literacy, attitudes, values and personal behaviors prior to exposure to the course (or intervention). Credit was given for completion of both surveys, regardless of response content. Students received the post-survey on the final day of the 10-week course, and had one week to complete the survey for credit. The post-test survey reflects the questions in the pre-test to enable analysis of change over time.

Data on the size of the student cohort that participated in the initial administration of the sustainability literacy surveys are available for surveys from 2013 through 2017. Follow-up assessment data are available only for the 2013, 2014, and 2016 surveys. The percentage of students assessed for sustainability literacy and for whom a follow-up assessment was conducted is reported as an average of 2013, 2014, and 2016 data: 0.72%. The percentage of students assessed for sustainability literacy without a follow-up assessment is available for 2013-2016 and is reported as an average of the data for the five survey years: 0.64%. A brief summary of results from the assessments (2013 through 2017) is provided below. Winter 2017 In the Winter 2017 Quarter, a total of 236 students enrolled in the Sustainability I course participated in the sustainability literacy survey. This represents 0.79% of the total UCI population (Winter 2017). Data for the Winter 2017 post-survey is forthcoming. Winter 2016 In the Winter 2016 Quarter, a total of 236 students enrolled in the Sustainability I course and 230 participated in the sustainability literacy survey. This represents 0.81% of the total UCI population (Winter 2016). The 137 students who completed both pre-test and post-test survey in 2016 represent 0.48% of the total UCI student population (Winter Quarter 2016); the 203 students who completed only the pre-test survey in 2016 represent 0.72% of the total UCI student population. Winter 2015 In the 2015 Winter Quarter, a total of 198 students who were enrolled in the Sustainability I course participated in the sustainability literacy survey. This represents 0.72% of the total UCI population (Winter 2015). Follow-up assessment data for these students has not been reported. Winter 2014 In the 2014 Winter Quarter, 206 students enrolled in the Sustainability I course participated in the sustainability literacy survey. Of the 206 students who completed the pre-test, 166 students completed the post-test surveys. The 166 students who completed both pre-test and post-test survey in 2014 represent 0.63% of the total UCI student population (Winter Quarter 2014); the 40 students who completed only the pre-test survey in 2014 represent 0.15% of the total UCI student population. Analysis of the ecoliteracy section assessed the improvement of accurate responses by students before and after the course. The course average for the ecoliteracy section changed from 73% to 97% before and after the course, respectively. This represents a statistically significant change with a p-value p=0.000 on a standardized paired sample t-test. A scale was calculated after accounting for reverse coded questions for each of the sections: pro-environmental behavior and food, purchasing, waste, and driving behaviors. For each scale a standardized paired sample t-test was calculated to test the change from pre-test to post-test. Students revealed a statistically significant change in some of these sections. Students reported more sustainable behavior in food purchasing (p=.08) and waste practices (p=.005). Driving, sustainable purchasing, and pro-environmental behaviors did not show statistically significant changes. Winter 2013 In the 2013 Winter Quarter, a total of 210 Sustainability I students participated in the sustainability literacy assessment, completing both the pre-test and post-test surveys. This represents 0.84% of the total UCI student population (Winter 2013). Analysis of the ecoliteracy section assessed the improvement of accurate responses by students before and after the course. Of the values, attitudes, and environmental behavior questions asked of students, only the mean student scores on the ecoliteracy survey were significantly higher at the end of the course than before the course began in the Winter 2013 Quarter at UC Irvine. A standardized paired sample t-test was run to determine this result using SPSS20. The student average was .7402/1 before the quarter began and .7748/1 at the end of the quarter, with a p value of p = .002.