Kathy Boyle's ID portfolio

This is a template for the ID portfolio page, where you will document your instructional design work in progress during this semester. This template is used to organize your own portfolio page. DO NOT EDIT the template page itself. See the video guide on how this project works and how to set up your ID portfolio page: Zoom Video If you need to see examples, see Gërt Arnold Portfolio Page; Sarah Stolberg's Portfolio Page.

Navigation links: ETAP 623 Fall 2024 | Link to my mini-course front page: Assessment for Curriculum Improvement

About Me

My name is Kathleen Boyle but people call me Kathy. I am both a professional marketer and college educator. I worked in the marketing and advertising industry for over 20 years. In my second career I became a professional lecturer at Marist College where I have worked for 15 years. I have developed and taught undergraduate and graduate courses in business, advertising and marketing. I have also worked in assessment at the college which is an area I am particularly interested in. For fun, I like to take my new puppy, Ruby, out and about town. She is an 8 month old pittie.

My Topic and Purpose

Having developed curriculum at the college level for the past 15 years, I have come to realize the importance assessment can have in developing good curriculum. In the most basic terms assessment is the practice of determining what students should know and be able to do at the end of a lesson, course or program and then making sure they can do it. It can also highlight areas where the desired learning is not happening so that improvements can be made to the instruction. Unfortunately, many faculty do not see assessment in this way. They see it as a compliance “thing”, that is, as something they have to do to maintain accreditation. Because of this common misconception, assessment is often inadequately done or not done at all. The purpose of this lesson is to show faculty the benefits of assessment that is designed to improve curriculum and provide them with a roadmap for how to do it.

Scope of Learning Outcomes and Content

For college faculty, this course will provide instruction on how assessment can help faculty improve their curriculum. It will also provide detail on how to develop an assessment plan. The main lessons are as follows:

Why Assessment is good for building curriculum
Writing effective student learning outcomes
Developing effective assessment rubrics
Developing Success Benchmarks
Developing an Effective Assessment Plan

Needs Assessment

Educating college faculty on how program-level assessment can help them improve instruction and the overall educational experience can increase faculty motivation to engage in authentic program assessment.

Program level assessment in higher education is designed to determine the effectiveness of the degree or certificate program in achieving the desired educational objectives. Put more simply, assessment determines whether the degree/certificate program taught students what it promised to teach them. It is an outcome driven process that determines if, upon completion of the program, students know what they are supposed to know and can do what they are supposed to be able to do as stated in the program’s mission statement and promised student learning outcomes. Program level assessment is used for several purposes, some of which include: 1) to provide evidence to accreditating bodies of program effectiveness, 2) to improve curriculum and teaching methods,3) to improve institutional effectiveness and 4) to further scholarly research on how well a field of study is understood by others and how that understanding develops. (Wang & Hurley, 2012)

One of the key best practices for effective and usable assessment is that it is faculty-led. However, it has been cited by many institutions, that faculty are often reticent to engage in assessment. “Academic administrators are sometimes critical of faculty members’ reluctance to be involved in program level assessment. Indeed, the need for faculty engagement and cooperation was recently at the top of the list for provosts in a survey of U.S. higher education institutions (Kuh and Ikenberry 2009 as cited in Andrade, 2010). In addition, “national studies (Ewell, 1996; Kuh & Ikenberry, 2009; Steele, 1996) have identified lack of faculty support as the most significant barrier to effective implementation of assessment (Wang & Hurley, 2012).

Two main reasons faculty do not want to participate in assessment is that 1) they view it as a compliance issue that administrators are “making them do” for accreditation purposes and 2) they are concerned that assessment may identify weakness in their teaching or program and as a result have negative consequences (Ewell 2002; Nichols and Nichols 2000 as cited in Andrade, 2010).

Those that view it as a compliance issue believe assessment takes them away from more important work, such as teaching and research (Wang & Hurley 2012). Those who fear negative consequences cite the possibility of “budget cuts, loss of positions, or program discontinuation. They also argue “that assessment process restricts academic freedom, increased workload, decreased time for scholarship (Cummings et al, 2008 as cited in Andrade, 2010), and a lack of incentive related to tenure and promotion can also discourage involvement in the process” (Andrade, 2010).

Fortunately, research has shown that professors are willing to engage in assessment if they perceive a positive benefit to teaching and learning. “Perceived benefit to teaching and learning has a significant, positive relationship to the willingness of faculty to engage in assessment” (Wang and Hurley, 2012). Unfortunately, many faculty think the main priority of assessment is accreditation compliance and not improved teaching. In a survey I conducted among faculty colleagues at the small liberal arts college where I work, 38% indicated that the primary reason for engaging in assessment was for accreditation purposes and only a little more than half (56%) stated that the primary reason for engaging in assessment is to improve curriculum and teaching (Boyle, 2024).

Educating faculty on best practices in assessment can facilitate more meaningful assessment results and analysis of the results which can be used to inform effective curriculum improvement plans and reinforce motivation for faculty engagement because the assessment results drove improved curriculum that increased student learning.

Another important best practice in effective assessment is that all faculty members understand assessment and are involved in the assessment process. Faculty drive curriculum development. Therefore, faculty need to understand what parts of their program are working in achieving the student learning outcomes, and what parts are not working so they can make improvements. A well-structured assessment will give faculty the crucial data they need to flag curricular issues that require revision. However, based on the primary survey referenced above almost 1/3 of faculty (32%) have little involvement in assessment and 38% have only a moderate involvement. In addition, 75% state they have only moderate to little knowledge on conducting effective program assessment (Boyle, 2024).

The learners/participants involved:

Since good assessment practices state that assessment should be faculty-driven and that all faculty should participate, the learners for this program are faculty in institutions of higher education. The participants are full-time tenured and non-tenured faculty who have moderate to little involvement in and knowledge of assessment. For the faculty who has limited motivation to participate in assessment, this mini course is designed to increase motivation to participate in their program’s assessment. For faculty who have limited knowledge in assessment best practices, this mini course will provide the necessary knowledge to conduct their program’s assessment effectively.

Analysis of gaps

- Motivation Gap: As stated above, faculty are more likely to engage in assessment if they perceive the benefit of assessment as being improvements to teaching and learning. However, a significant portion of faculty do not view the primary purpose of assessment as being the improvement of the educational experience. This instructional program should convince faculty that good assessment can help them improve their program’s curriculum which will lead to increased learning.

- Knowledge Gap: As stated above, a significant portion of faculty have only moderate to little knowledge on assessment. Specifically, in the primary research assessment survey faculty indicated that they would like to learn more about the following assessment activities:

- Developing student learning outcomes (57%)

- Developing assessment rubrics (57%)

- Developing success benchmarks (69%)

Existing efforts

Some research has been conducted on faculty conception about assessment, including the study Faculty and student conception of assessment in higher education, by R. Fletcher, L. Meyer, H. Anderson, P. Johnson, & M. Rees. As well as X. Wang, X and S. Hurley’s study entitled Assessment as a scholarly activity?

With regard to closing the learning gap, the American Association for Colleges & Universities (AAC&U) offers educational conferences and other resources to help faculty learn and be able to implement assessment best practices. AAC&U “… is a global membership organization dedicated to advancing the democratic purposes of higher education by promoting equity, innovation and excellence in liberal education” (AAC&U, 2024). One of their key initiatives was the development of VALUE rubrics. VALUE rubrics are rubrics that enable educators to assess students’ original work across 16 broad learning outcomes, like critical thinking, ethical reasoning and inquiry & analysis.

Intent statement: This mini course will motivate faculty to engage in program-level assessment and will provide them with instruction on assessment best practices so they can implement meaningful and useful assessment that will improve student learning.

Analysis of the Learner and Context

Learners: Higher education faculty with moderate to little involvement in and knowledge of assessment. All faculty should be involved in program assessment, however at most colleges participation is more opt-in than mandated by administration, especially in terms of using assessment to inform curriculum improvement. Therefore, it is critical to motivate inactive faculty members to want to engage in assessment. It is also critical to show them how to engage in assessment so that it is meaningful and useful. Otherwise, faculty will become de-motivated to continue participating in the assessment process.

Context: This will be an online, asynchronous mini course that faculty can engage in when it is convenient for them. The total course time will be 1 ½ to 2 hours. No additional resources will be required.

Performance-Based Objectives

After completing this mini-course, learners will be able to:

Explain what transformative program-level assessment is and how it can help improve program curriculum to result in increased student learning.
Develop program-level, student-centric student learning outcomes (SLOs) that are meaningful, realistic and actionable.
Develop rubrics to effectively assess program-level student learning outcomes.
Develop effective success benchmarks to analyze student data against
Develop and effective assessment plan that will provide useful and useable data to improve program curriculum.

Task and Content Analysis

Pre-requisite Knowledge & Skills

Before taking this course, the participant should:

Be able to use computer skills to navigate online directions

Understand how to create program curriculum to meet learning objectives
Be familiar with the concept of assessment at their institution

Unit 1: What is Transformative Assessment

After this unit students should be able to:

· Explain the concept of transformative program-level assessment for curriculum improvement and why it is important

· Explain how program level assessment can help with curriculum improvement

· Explain the assessment process and the steps to building an effective assessment plan

Unit 2: Creating Student Learning Outcomes

After this unit students should be able to:

· Explain the concept of student learning outcomes

· Explain the difference between learning objectives and student learning outcomes

· Create student learning outcomes that are meaningful, realistic and actionable

Unit 3: What, Where and How to Assess Student Learning Outcomes

After this unit students should be able to:

· Explain how to determine what work-product will be assessed, where in the program it will be assessed and how it will be assessed

· Create effective program-level rubrics to assess student learning

Unit 4: Developing Effective Success Benchmarks

After this unit students should be able to:

· Explain what success benchmarks are and why the important in assessment

· Explain the different methods for choosing comparison benchmarks

· Explain the difference between benchmarks for essential learning goals and aspirational goals

· Create useful and useable program-level benchmarks

Unit 5: Assessment Plan

After this unit students will be able to:

· Develop an effective assessment plan that will provide useful and useable data to improve program curriculum

==

References and Resources

Andrade, M. S. (2010). Managing change—engaging faculty in assessment opportunities. Innovative Higher Education, 36(4), 217–233. https://doi.org/10.1007/s10755-010-9169-1

Boyle, K. (2024). Assessment Survey. (Primary Research Survey).

Homepage. AAC&U. (n.d.). https://www.aacu.org/

Fletcher, R., Meyer, L., Anderson, H., Johnson, P., & Rees, M. (2012). Faculty and student conception of assessment in higher education. Higher Education, 64, 119–133.

Suskie, L. (2018). Assessing Student Learning: A Common Sense Guide: 3rd Edition. John Wiley & Sons.

Wang, X., & Hurley, S. (2012). Assessment as a scholarly activity? The Journal of General Education, 61(1), 1–15. https://doi.org/10.5325/jgeneeduc.61.1.0001