Episode 57: Will Miller on Continuous Improvement through Institutional Research and Assessment
Listen On
How do create the culture, processes, and shared data you need to build a culture of continuous improvement? How can institutional research, assessment, and accreditation professionals lead the way in this? We discuss these questions with Will Miller, VP of Continuous Improvement and Institutional Performance at Embry-Riddle Aeronautical University
In the high-stakes world of higher education, the word "assessment" often conjures images of bureaucratic checkboxes and looming accreditation deadlines. Faculty and staff frequently view it as a hurdle to clear rather than a tool to use. However, as Will Miller, Associate Vice President of Continuous Improvement and Institutional Performance at Embry-Riddle Aeronautical University, explains, assessment is not the end goal—it is the means to an end. That end is a better, more connected student experience.
By shifting the focus from mere compliance to active improvement, institutions can break down silos and use data to proactively support students before they fall through the cracks. In this episode of the Connected College podcast, we explore how to rebrand assessment, redefine success from the student’s perspective, and use granular data to drive real institutional change.
Redefining Student Success Beyond the Numbers
Most higher education leaders define student success through a rigid set of metrics: first-year retention, four-year graduation rates, and post-grad earnings. While these are valid, Miller argues that we must allow students to define success for themselves.
For one student, success might mean transferring to a dream school after completing Gen Ed requirements. For another, it might mean landing a high-paying job in aviation maintenance after only two years of study. If an institution views these as "losses" because the student didn't graduate, they are missing the point. True student success is holistic. It includes mental health, a sense of affinity with the campus community, and the mastery of core skills like collaborative group work—even if that path doesn't follow a traditional four-year trajectory.
The Three Pillars of Institutional Performance
To effectively support this broader definition of success, Miller identifies three "legs of the stool" that every modern assessment office should lean on:
Accreditation and Compliance: Ensuring the institution meets the standards of its various governing bodies.
Goal Setting and Measurement: Working directly with faculty and staff to determine what they want to accomplish and how they will know if they’ve succeeded.
Data Integration: Bringing together disparate data points—from library check-ins to LMS engagement—to create a "data lake" that allows for predictive analytics.
At Embry-Riddle, this transition was so significant that the department was rebranded to "Continuous Improvement and Institutional Performance." This name change signals to the campus that the office is there to help them get better, not just to collect reports.
Breaking the "M&M" Cycle of Retention
In many colleges, retention is treated like a "Morbidity and Mortality" study in a hospital. Administrators sit around a boardroom table looking at the "cadavers"—the students who have already left—and try to figure out what went wrong. While this post-mortem is interesting, it does nothing to save the students who are currently struggling.
The future of assessment lies in proactive, predictive modeling. By using data from the moment a student applies, institutions can identify which "levers and dials" to activate. If a student shows signs of low resiliency or decreasing engagement in the first few weeks, the college can intervene before that student becomes another statistic in next year’s retention report.
The Power of the Curriculum Map
One of the most practical ways assessment enables student success is through curriculum mapping. Often, faculty create beautiful maps showing how Intro to Political Science leads to senior-level research, but they never share that map with the students.
Miller suggests a more collaborative approach: asking graduating students where they actually learned their program outcomes. When institutions compare student perceptions with faculty intentions, they often find gaps. Closing these gaps, by explicitly telling students why a certain course matters to their future career, increases engagement and ensures that every class, whether taught by a tenured professor or an adjunct, serves the broader mission of the degree.
Navigating the Demographic Cliff with Innovation
With the "demographic cliff" approaching, the competition for a shrinking pool of students will only intensify. Institutions cannot afford to stay in their comfort zones. This means being willing to A/B test new success initiatives and making "big bets" on innovative programs rather than spreading resources thin across dozens of unproven ideas.
Assessment provides the safety net for this innovation. It allows leaders to fail fast, learn what works, and double down on the initiatives that truly resonate with students. By moving from a culture of compliance to a culture of continuous improvement, colleges can ensure they aren't just surviving the coming changes, but thriving in them.
Episode 57 Transcript
-
Elliot Felix: Welcome to the Connected College Podcast. I'm your host, Elliot Felix. I've helped more than a hundred colleges and universities change what they offer, how they operate, and how they're organized to enable student success. And if you're a leader in higher ed, and you think that the silos and separations get in the way of student success, then this podcast and my upcoming book, The Connected College, are for you. We're here to learn and work together to bust silos, question tradition, and forge partnerships so that students feel connected to their college, their community, their coursework and their careers.
Elliot Felix: This is a lesson that Will Miller learned the hard way early in his career as a political scientist and pollster. He went on from that faculty role to leading accreditation and assessment functions at three different institutions and even providing software and consulting to hundreds of them. We have a great conversation about how assessment is the means and the ends is continuous improvement, which he believes so strongly in. He recently renamed his assessment and accreditation department at Embry Riddle into Continuous Improvement and Institutional Performance. I hope you'll enjoy geeking out on using data for student success as much as I did in this conversation. Welcome, Will Miller. Can you tell us a little bit about yourself and how you got started in assessment?
Will Miller: Yeah, absolutely. And you know, it's really fun to have these conversations and talk about assessment in a helpful way when so many people, including faculty and university folks view us as kind of a bureaucratic checkbox sometimes really helping to kind of combat that stigma and talk about the ability for assessment to really contribute to the student experience, to student success, to the student journey. I'm a political scientist by training. I was loving the faculty life. And then I was at a small liberal arts college in Florida, getting ready for an accreditation review. Our director of accreditation and assessment had a medical issue and had to take pretty much the entire prep time up to the visit off.
Will Miller: It was kind of all hands on deck. And with my backgrounds in political polling and data, they kind of asked me about getting involved in the institutional research side and then the bureaucratic side of my studies and public administration. I love policy. So it was, what can we do from a policy angle? So I agreed to, in a short period, kind of, in an interim capacity, lead these efforts and then absolutely fell in love with the work and the ability to work with faculty and help them have positive impacts on student learning and the student experience and all of these factors for student success.
Will Miller: I did that for about 3 and a half, 4 years at that first institution, which was Heidelberg College. Uh, and then I, I left higher ed for about four years, and I went in and worked with a higher ed tech and consulting company that focused on assessment accreditation, and that was Campus Labs, which is now part of Anthology. As my wife finished law school, I didn't love being on the road four days a week, so I found myself at Jacksonville University for almost five years, overseeing analytics, effectiveness, strategic planning, and then just in the last six, seven, eight months made my way a little further south in Florida to Embry Riddle Aeronautical University, which is a completely different institution type for me.
Elliot Felix: You hinted at some of these measures in terms of first year retention and on time graduation, but how would you define student success?
Will Miller: That's a question where I differ from a lot of my colleagues. I'm not an advocate that it's my job to define student success. I think a student defines what success looks like and we have a role in helping them elevate that expectation. I say that because your traditional student success measures, like you said, first year retention rate, getting them to stay in school, four year graduation rate, six year graduation rate, career placement, first job earnings, all of those are super valid measures.
Will Miller: But let's use student retention as an example. Let's say I have a student that's got a 1.2 GPA at the end of their first year. They're getting no institutional aid and they're paying 60,000 dollars through private loans or federal loans to attend. If we don't think that student has a chance to actually graduate in four years or ever complete, is it really success if they come back and start the next fall, they're another 30,000 dollars in debt, and then I fail them out at the end of the fall term? I have a hard time, from an institution perspective, sure, that's retention. For that student, would it have been more successful if we had had a conversation and said, I'm not sure this is the right fit for you right now?
Will Miller: There are times where students leaving are not the worst thing for the students. Our two best students we had at that institution in my program, I helped transfer out. I was like, if I can get you into Georgetown and you want to go into the Foreign Service, I'm doing you a far better job, even though this will look like a loss or an unsuccessful outcome for our institution. We succeeded in helping them meet what their actual goal was at their time at our institution.
Elliot Felix: Right, so imagine then you're a fan of thinking about persistence and retention holistically. It sounds like for you, the key is defining student success from the student perspective, right?
Will Miller: And I mean, I'm a big advocate for, use the user defined field. Ask the students when they enter, what's your goal? What does success look like for you and your interaction with this institution? It's not that I would just accept the student saying, I'm going to come for two years and leave. Obviously the goal is they fall in love with it. But I'm not going to be mad at myself if they do exactly what they said.
Will Miller: We have that at Embry Riddle with Aviation Maintenance. If you can work on plane engines, you can get a job wherever. We have students where as soon as they pass their federal certifications, companies show up and offer signing bonuses. It's hard to look at a student whose goal was to get a job working for Delta and they get offered that after their sophomore year. You've got your ambition. The hope is you had a good enough time with Embry Riddle that you remember us and you become a donor and an engaged alum, even though you're not a graduate.
Elliot Felix: So it's like retention, graduation, placement, salary—those are all great. It's just don't pursue them at the expense of success as students define it for themselves.
Will Miller: And I'd say, too, would I rather graduate the kid in four years who is now mentally drained and self-doubting, or if they would have taken an extra year, they're in a better place leaving? Part of the student success piece is we have to be thinking holistic about it. It's not just about performance. I was a terrible advisor to students. I would be the one where they'd come in struggling and I'd be like, stop going, just write that off as an F, you're done. Because otherwise you're going to put all your energy into making that D stay a D, and all of your B's are now going to be C's. You're going to be stress tired. Just drop it. It's probably a lot better for this kid's mental wellbeing. We have to be open to these conversations.
-
Elliot Felix: For folks that don't know what the Office of Institutional Assessment and Research or the Office of Assessment and Accreditation does, give us the overview of those functions in higher ed.
Will Miller: Yeah, in the last few weeks, we've actually rebranded my entire group. When I got to Riddle, we were the Office of Accreditation and Assessment. We were in charge of ensuring continued compliance with SACSCOC as our institutional accreditor and all of our various specialized accreditors like ABET for engineering or AACSB for business. It's making sure we're doing the things we're supposed to do, using data to make improvements, and following our mission.
Will Miller: On the assessment side, it's about working with faculty and staff to goal set. What are your goals? How are you going to know if you accomplish them? What do we do to keep moving forward? Then the IR side, the Institutional Research side, that's data aggregation. How do we bring together disparate systems all over campus collecting data on library check-ins, academic engagement, and LMS interaction into a massive data lake? We rebranded to the Office of Continuous Improvement and Institutional Performance because getting accredited or collecting data is not an end stage. It's about whether we are performing at a level that shows commitment to our mission and our students.
Elliot Felix: So it sounds like there's three legs: the accreditation piece, the goal-setting piece, and then the data wrangling to connect inputs and outcomes.
Will Miller: And that third leg is so crucial. 95 percent of the time with retention, we treat it like an M and M study in med school. We look at the cadaver, talk about everything that went wrong, and then we don't necessarily take the step to prevent other bodies from showing up. I spent early time in boardrooms talking about why a student didn't retain instead of identifying the 300 students who are going to be next for the same reason and stopping them. That proactive predictive piece is where assessment plays a role.
Elliot Felix: Paint a picture for the role that assessment can play in enabling student success.
Will Miller: Assessment is where we start to figure out holistically what's working and not working. Classroom life is not the majority of a student's day. If we learn through data that a student is super engaged socially but never goes to class, or a student is buried in books but has zero peer interaction, that's not success. Assessment helps us advise students to be well-rounded. It’s where we ask the big questions: What does success look like? Do we have different paths to get there? And if things are failing, how do we prevent it from happening again?
Will Miller: One of my biggest challenges is convincing people it's okay to say a goal wasn't met. It's even better to tell me it failed miserably so we can share warning signs and not waste resources. With retention, presidents often want numbers to go up quickly, so they throw 18 new initiatives at it. If retention goes up, we have no idea what caused it. We find ourselves trapped funding all 18 things, which ironically upping tuition and over-programming students. Assessment helps us untangle those threads.
-
Elliot Felix: I feel like we just identified another measure of student success, which is mental health. What other qualitative things are in the mix for you?
Will Miller: For me, a big one is affinity. I want students coming to Embry Riddle not just because we are a top-tier flight school, but because they enjoy their time. Something has to bring them joy on campus. Whether it's a relationship with one faculty member, a social club, or their major, that affinity is vital. Then there are core skills: written communication, public speaking, and now data analytics and coding.
Will Miller: One thing we noticed is that we have lots of group work because it's realistic for engineers and aviation, but we don't have many courses that teach you how to work in a group. We check a box for collaborative learning, but we never tell you what it's like to manage conflict.
Elliot Felix: In a recent national survey, the absolute lowest metric was group work at 42 percent satisfaction because everyone is expected to do it and no one is taught how.
Will Miller: 100 percent. And from a faculty angle, it's horrible. There is a perception that group work is easier because you have less to grade, but I would rather grade another thousand papers than spend hours mitigating inner-group conflict between students. We've never taught faculty how to do good group work.
Will Miller: Another trend is the move toward data desegregation and equity of learning. We can now look at intro composition courses at a community college and see that students from specific feeder high schools aren't as prepared. That allows us to go back to those high schools and say, "What can we do to help you prepare them better?" It's about using the data to take the conversation to a place that actually helps the student.
-
Elliot Felix: So what do you do? Is it piloting and randomized controlled trials to be able to distinguish what works?
Will Miller: That's the ideal world, but it requires patience. Higher ed today, with fiscal trouble, often doesn't have four years to wait. We need to ask students what they think had an impact, but also recognize they might only see the direct, student-facing thing. You have to take away the fear element. People get very protective of their programs to justify their jobs. You have to create a comfort level where they can say, "This didn't go the way it was supposed to," and know they won't get fired.
Elliot Felix: It strikes me that another strategy instead of 18 initiatives is to pick three or five that you can really focus on.
Will Miller: I think there is a way to do big bets. Students can help you think through what those should be through interviews and focus groups. If you ask them to reflect on what events they would have skipped if not required, or where they met their friends, you start to see what is core to their experience. I'm also an advocate for A/B testing. If I said that in some areas of campus, jaws would hit the floor—"we can't do that, that's not higher ed"—but we have to be nimble.
Will Miller: The demographic cliff is coming. We're in a happy honeymoon phase now with a steady incline, but soon the floor falls out. When there are fewer students to pick from, you have to make sure you retain them. Schools aren't going to be able to just offer scholarships forever. At my previous institution, the discount rate was likely over 80 percent. If I see five students walking together, I only have one tuition payment. We have to ask: am I better off giving 1,000 students an 80 percent discount or 500 students a 40 percent discount where I can actually focus on their experience? These conversations need to happen outside of just the presidential cabinet.
-
Elliot Felix: I would love to hear a few specific examples of how assessment enables student success.
Will Miller: I'll start with a purely academic one. Assessment has morphed. Average faculty feel expectations have risen exponentially for what needs to be reported. I still have faculty who say, "I already do grades, that's assessment." But grading is different than assessing. We have curriculum maps that show how courses match program outcomes, but we never tell the students what that map is.
Will Miller: One of my favorite examples is sitting down with graduating students, giving them the program outcomes, and asking, "Where did you learn this?" We then heat map their perceptions against the faculty's map. It leads to better conversations. Faculty start day one by telling students why the course is essential for what comes later.
Will Miller: This is especially important for adjuncts who might not know how their course fits into the broader scheme. If I'm teaching Research Methods, I'll teach it one way if it's just a Gen Ed, but a different way if I know every student has to do a research-based thesis their senior year. We need that granularity to see that students who had an adjunct for Intro to Chemistry might struggle four courses later in Organic Chemistry compared to those who had full-time faculty. Even if the adjunct was "easier," the student wasn't adequately prepared. That's actionable data.
Elliot Felix: This is awesome, Will. I really appreciate your insights on assessment and how these things are changing.
Will Miller: It's definitely something I'm passionate about. I appreciate you letting me on.
Elliot Felix: Thanks for listening to the Connected College podcast. Subscribe to my newsletter at ElliotFelix.com for insights, tools, and special offers. Let's create connected colleges where students will succeed.