| Overall Rating | Silver - expired |
|---|---|
| Overall Score | 62.83 |
| Liaison | Danielle Smith |
| Submission Date | Aug. 11, 2020 |
University of New Brunswick, Fredericton
AC-6: Sustainability Literacy Assessment
| Status | Score | Responsible Party |
|---|---|---|
|
2.00 / 4.00 |
"---"
indicates that no data was submitted for this field
Does the institution conduct an assessment of the sustainability literacy of its students?:
Yes
Which of the following best describes the literacy assessment? The assessment is administered to::
The entire (or predominate) student body, directly or by representative sample
Which of the following best describes the structure of the assessment? The assessment is administered as a::
Standalone evaluation without a follow-up assessment of the same cohort or representative samples
A copy of the questions included in the sustainability literacy assessment(s):
A list or sample of the questions included in the sustainability literacy assessment or the website URL where the assessment tool may be found:
---
A brief description of how the literacy assessment was developed and/or when it was adopted:
Initial research was performed by utilizing the STARS “Campus Sustainability Hub”, which is a resource hub that provides AASHE members with access to toolkits and resource collection on all areas of sustainability in higher education. In addition, the STARS website also has a help center, which has an article for each section within STARS. Contained within the article about sustainability literacy assessments are answers to frequently asked questions, templates and tools, example responses from institutions that have taken varied approaches to conducting SLAs, common issues, and links to additional resources. The resources available through AASHE were an useful starting point for finding more information about the structure and content within SLAs. Another approach taken was to review completed SLAs that could also be accessed on the STARS website. The questions of from 5 institutions were compared, which included Arizona State University, the University of Michigan, the University of Alberta, Colorado State University, and Ohio State University. These universities were chosen because they were highlighted on the STARS website as exemplifying best practices. Moreover, the SLAs from these universities had questions that were very similar to one another and even some questions were even identical to one another.
The similarities found in many of those questions led to finding out that Ohio State University had originally created the SLA that many of these institutions were either using the same version or some sort of modified version. In 2013, the Environmental and Social Sustainability Lab (ESSL) at Ohio State University developed their initial Assessment of Sustainability Knowledge (ASK), which was designed to measure sustainability knowledge using environmental, economic, and social topics (ESSL, 2019). The original set of questions went through a rigorous development process, which consisted of gathering input from experts, organizing iterative focus group testing, pilot testing, distribution to nearly 2,000 undergraduates, and utilizing Item Response Theory (IRT) to select the most discriminating questions (Zwickle et al. 2014). In the end, the assessment was reduced to about half from an initial set of 30 questions to 16 questions. Furthermore, to refine their assessment the researchers at the ESS Lab set out to establish a broad baseline of data and compare their student population results with other universities (Zwickle et al. 2014), this led to a collaboration with the University of Maryland. In this second phase, they expanded the number of questions to 28 and had more than 3,000 students participate (ESSL, 2019). In the third phase, the 28-item assessment administered again, but to more than 4,000 students. Another IRT analysis was performed and the assessment was shortened and updated to just 12 questions, which included questions of varying difficulty and covered the environmental, economic, and social domains (Zwickle & Jones, 2018). In the vain of sharing knowledge and broadening baseline data, this version has been made available for other researchers and institutions to utilize. Therefore, both a representative from the ESS lab and the original researcher Dr. Adam Zwickle were contacted to obtain permission for the 12-item assessment.
After permission was granted from both parties, a modified SLA was created and consisted of 22 questions. The set of questions used for this assessment are presented in Appendix A. Some questions were either replaced or modified from the original set of questions by Zwickle and Jones (2018) for the purpose being more suitable for a Canadian audience. The first to be modified was question 15, where the original question referred to the wealth disparity among Americans and in this version, it asked about whether the gap between Canadian’s richest and poorest has changed over the last 3 decades (Yalnizyan 2010). Question 16 was originally about how many economists argue that electricity prices are too low in the U.S. because the prices do not reflect the cost of pollution from generating pollution. Instead, the new question is about the types of electricity in Canada and how their frequency of use will change over the next 20 years (NRC, 2019). Question 19 was only slightly adjusted where the word “Native American tribal lands” was changed to “First Nations traditional lands” for the purpose of making this question more suitable for a Canadian audience. Another small change was added to question 21, where the option about “flying in a commercial airplane from Washington, D.C to China” was changed to “flying in a commercial airplane from Fredericton, NB to Toronto, ON”. This change was made to provide some context for Canadians. Lastly, question 22 was created in response to the question about the country that has recently surpassed the U.S. in becoming the largest emitter of greenhouse gas carbon dioxide. The new question was about flooding in Fredericton and the types of impacts it may place on the community. The reason for replacing the original question was to add a question that had a local context and incorporated all three pillars of sustainability. In the end, the assessment still consisted of 12 knowledge-based questions that were either the same as or very similar to the original questions from Ohio State University. In total, there were 4 questions directly related to the environment (questions 10-13), 2 questions directly related to social concepts (questions 14-15), 2 questions directly related to the economy (questions 16-17). Moreover, there was 1 question related to both the environment and economy (question 18), 1 question about social issues and the economy (question 19), 1 question related to the environmental and social issues (question 20), and finally 1 question that incorporated all three pillars of sustainability (question 21). Additionally, questions about demographics, attitudes towards sustainability, and self-perceived perception of sustainability knowledge were added at the beginning of the assessment. Questions 1-5 were about demographics, questions 6-8 were about attitudes, and questions 9 and 22 were about the perception of sustainability knowledge before and after the assessment. Before distribution, the questions were peer-reviewed, and comments were provided by Dr. Tom Beckley on ways to improve the assessment and the appropriate adjustments were made.
The similarities found in many of those questions led to finding out that Ohio State University had originally created the SLA that many of these institutions were either using the same version or some sort of modified version. In 2013, the Environmental and Social Sustainability Lab (ESSL) at Ohio State University developed their initial Assessment of Sustainability Knowledge (ASK), which was designed to measure sustainability knowledge using environmental, economic, and social topics (ESSL, 2019). The original set of questions went through a rigorous development process, which consisted of gathering input from experts, organizing iterative focus group testing, pilot testing, distribution to nearly 2,000 undergraduates, and utilizing Item Response Theory (IRT) to select the most discriminating questions (Zwickle et al. 2014). In the end, the assessment was reduced to about half from an initial set of 30 questions to 16 questions. Furthermore, to refine their assessment the researchers at the ESS Lab set out to establish a broad baseline of data and compare their student population results with other universities (Zwickle et al. 2014), this led to a collaboration with the University of Maryland. In this second phase, they expanded the number of questions to 28 and had more than 3,000 students participate (ESSL, 2019). In the third phase, the 28-item assessment administered again, but to more than 4,000 students. Another IRT analysis was performed and the assessment was shortened and updated to just 12 questions, which included questions of varying difficulty and covered the environmental, economic, and social domains (Zwickle & Jones, 2018). In the vain of sharing knowledge and broadening baseline data, this version has been made available for other researchers and institutions to utilize. Therefore, both a representative from the ESS lab and the original researcher Dr. Adam Zwickle were contacted to obtain permission for the 12-item assessment.
After permission was granted from both parties, a modified SLA was created and consisted of 22 questions. The set of questions used for this assessment are presented in Appendix A. Some questions were either replaced or modified from the original set of questions by Zwickle and Jones (2018) for the purpose being more suitable for a Canadian audience. The first to be modified was question 15, where the original question referred to the wealth disparity among Americans and in this version, it asked about whether the gap between Canadian’s richest and poorest has changed over the last 3 decades (Yalnizyan 2010). Question 16 was originally about how many economists argue that electricity prices are too low in the U.S. because the prices do not reflect the cost of pollution from generating pollution. Instead, the new question is about the types of electricity in Canada and how their frequency of use will change over the next 20 years (NRC, 2019). Question 19 was only slightly adjusted where the word “Native American tribal lands” was changed to “First Nations traditional lands” for the purpose of making this question more suitable for a Canadian audience. Another small change was added to question 21, where the option about “flying in a commercial airplane from Washington, D.C to China” was changed to “flying in a commercial airplane from Fredericton, NB to Toronto, ON”. This change was made to provide some context for Canadians. Lastly, question 22 was created in response to the question about the country that has recently surpassed the U.S. in becoming the largest emitter of greenhouse gas carbon dioxide. The new question was about flooding in Fredericton and the types of impacts it may place on the community. The reason for replacing the original question was to add a question that had a local context and incorporated all three pillars of sustainability. In the end, the assessment still consisted of 12 knowledge-based questions that were either the same as or very similar to the original questions from Ohio State University. In total, there were 4 questions directly related to the environment (questions 10-13), 2 questions directly related to social concepts (questions 14-15), 2 questions directly related to the economy (questions 16-17). Moreover, there was 1 question related to both the environment and economy (question 18), 1 question about social issues and the economy (question 19), 1 question related to the environmental and social issues (question 20), and finally 1 question that incorporated all three pillars of sustainability (question 21). Additionally, questions about demographics, attitudes towards sustainability, and self-perceived perception of sustainability knowledge were added at the beginning of the assessment. Questions 1-5 were about demographics, questions 6-8 were about attitudes, and questions 9 and 22 were about the perception of sustainability knowledge before and after the assessment. Before distribution, the questions were peer-reviewed, and comments were provided by Dr. Tom Beckley on ways to improve the assessment and the appropriate adjustments were made.
A brief description of how a representative sample was reached (if applicable) and how the assessment(s) were administered :
A plan of distribution was developed once a final version of the questions were completed. The first step of this plan was to set up the SLA on Lime Survey, which is an online survey tool for open source surveys and is supported by UNB. Moreover, to incentivize students to complete the SLA, there was the chance to win a UNB swag prize pack, which included a UNB sweater, UNB Sustainability cutlery set, UNB Sustainability reusable metal straws, and a UNB reusable water bottle. Next, a total of 37 faculty admins from each department were contacted via email and asked to forward the SLA link along to their students. A short explanation of the project was included in the email and a poster outlining details about the assessment was attached. Another approach was to post the SLA on UNB Sustainability social media accounts (Facebook & Instagram), contact UNB student groups, and having the official UNB Fredericton Facebook page create a post about the SLA. Additionally, the coordinator from the UNB Co-Curricular Program was contacted to have the SLA as a permanent fixture within their program. Professors were also emailed and asked to share the SLA with their students. Furthermore, the SLA was also posted on the front page of UNB Sustainability’s website. Lastly, a total of 6 engagement sessions were held for 2hr periods to have students fill out the SLA in person. The assessment was available to be completed online on Lime Survey from October 16th, 2019 to November 22nd, 2019 and was open to current students at the University of New Brunswick (Fredericton).
A brief summary of results from the literacy assessment(s):
The knowledge questions of the assessment included questions 10-21. Figure 8 depicted the distribution of scores with the overall average being 60.10%. Moreover, the greatest number of responses fell between the range of 40.01% and 50%.
Optional Fields
---
Additional documentation to support the submission:
---
Data source(s) and notes about the submission:
---
The information presented here is self-reported. While AASHE staff review portions of all STARS reports and institutions are welcome to seek additional forms of review, the data in STARS reports are not verified by AASHE. If you believe any of this information is erroneous or inconsistent with credit criteria, please review the process for inquiring about the information reported by an institution or simply email your inquiry to stars@aashe.org.