Assessment, Evaluation, and Research
Category: Exemplary
In this competency, I have developed in the following areas:
-
“Differentiate among assessment, program review, evaluation, planning, and research as well as the methods appropriate to each” (p. 20)
-
“Facilitate appropriate data collection for system/department-wide assessment and evaluation efforts using current technology and methods” (p. 20)
-
“Ensure all communications of AER results are accurate, responsible, and effective” (p. 20)
-
“Identify the political and educational sensitivity of raw and partially processed data and AER results, handling them with appropriate confidentiality and deference to organizational hierarchies” (p. 20)
-
“Design ongoing and periodic data collection efforts such that they are sustainable, rigorous, as unobtrusive as possible, and technologically current” (p. 20)
-
“Apply the concepts and procedures of qualitative research, evaluation, and assessment including creating appropriate sampling designs and interview protocols with consultation, participating in analysis teams, contributing to audit trails, participating in peer debrief, and using other techniques to ensure trustworthiness of qualitative designs” (p. 20)
-
“Demonstrate a working knowledge of additional methodological approaches to AER (e.g. mixed methods, historical or literary analysis, or comparative study) including elements of design, data collection, analysis, and reporting as well as strategies for ensuring the quality” (p. 20)
Assessment, Evaluation, and Research was never a skill that I had on my radar until my experience at CampusQuad this past summer. Going into fall 2015 I had anticipated taking Higher Education Law because I thought it would provide the most solid foundation for work in student affairs and eventually a PhD. After my summer experience, I realized that assessment is a skill that I can use in any career. I entered into the Assessment in Higher Education course with a foundation of business research methods and economics statistics. Despite this foundation, I had never really enjoyed research and had trouble differentiating between the types of research methodologies that existed. I had forgotten much of what I learned in statistics and research classes in college, and I was worried I would not be any good at assessment. Taking the Assessment in Higher Education course with Dr. Rho in her first semester was an amazing experience, because we had a small class that experienced her passion for assessment and research very personally. Through this class, we learned about the difference between assessment and research, something I had never considered before, and the different ways that student affairs practitioners can use assessment. One of my favorite things we did in this class was weekly presentations on how a different functional area uses assessment. Types of assessment covered included satisfaction, usage, outcomes, cost-benefit analysis, and many others. By doing these presentations, we were able to see the varied ways in which a department can use assessment. What was most beneficial to me was learning about the offices you would not think would traditionally do assessment, using assessment in an innovative way. This really contributed to my ability to critically think about how assessment can be used. My goal going into this class was to develop a foundation of assessment so that I could design assessment strategies for an education technology company that I work for in the future. Through the class, we saw an assessment project from start to finish, making this the perfect experience for what I wanted.
This assessment project that we did for the class gave us the opportunity to act as institutional assessment staff and meet with a department on campus to do an assessment for them. We essentially acted as consultants and went through every step of the assessment with our client. Because of the way this project is set up we were able to learn the skills directly related to assessment and the politics and institutional dynamics that are tied to seeing the process all the way through. Through this project I learned that assessment, evaluation, and research is so much more than collecting data and analyzing it to make it look like something you want it to be. I learned that assessment is an essential part of being a successful unit, and that being successful in assessment means knowing how to determine what really needs to be assessed, not what seems like it needs to be assessed. For this project, we took on the challenge of assessing a peer mentor program within the President’s Leadership Academy (PLA). Our goal was to determine the satisfaction of the mentors and mentees through a document analysis and focus groups. Ultimately, we ended up with this qualitative assessment, but not before going back and forth with the department we were working with, who wanted a quantitative assessment, and Dr. Rho who helped us understand why quantitative assessment was not appropriate for this study. I have included our final project as my artifact for this competency, because it shows every step of the process we went through, including the description of the program, literature review, assessment plan, assessment analysis, results, and future suggestions. This artifact demonstrate every aspect of the assessment class.
Taking this class alongside Outcomes was critical to developing my understanding of Outcomes based assessment. In both Assessment and Outcomes we learned about this type of assessment, but because I was able to learn about them side-by-side, I had an advantage when we had to design the assessment plan for our Implementation Plan. In the Outcomes course, I learned about rubrics in assessment work, and using pre-existing instruments to perform assessment. For my Implementation Plan I used the AACU Value rubrics to design an assessment plan that would assess how students were progressing in oral and written communication throughout college. I was able to tie this learning outcome to my general education requirement and use this assessment strategy to determine if students were meeting the outcome. Ideally, if a student were not achieving the outcome, there would be additional supports to help them do so.
I also use assessment, evaluation, and research on a consistent basis in my assistantship experience in the Division of Student Affairs. Coordinating a program like Ziggy Points means needing to do an assessment and evaluation every year to determine what we are doing correctly and what needs improvement. From the assessment we have implemented the past two years, we have changed our process around notifying students about the Ziggy Points program, how we communicate with partners, and our process for coordinating with offices on campus. Because of the assessment course I took, I have changed how I see assessment work on a daily basis. Instead of doing assessment at the end of the year for Ziggy Points, I would like to develop a culture where offices that host Ziggy Points events have access to the data we collect so they can do assessment with the students that attend the event. What I envision is these offices either using the data to do post-event surveys or collecting data over time to see who is and is not coming to events so they can develop strategies around how to reach different students.
Overall I think that assessment has become one of my strongest competencies both because of the theoretical foundation I acquired through the Assessment in Higher Education course and because I work in two offices that have strong cultures of assessment. Coming from a business and economics background it makes sense to me that I enjoy this work, but it took taking a class and learning about the subject from someone who is passionate about it for that spark to occur. Moving forward I would like to continue to develop this competency through gaining more exposure to quantitative and mixed methods assessment.
Reference
ACPA & NASPA (2015). Professional competency areas for student affairs practitioners. Washington, DC: Authors.
Assessment Project Team