Episode 78: Jeff Doyle on Unconventional Ways to Predict Student Success
Listen On
How can you use the data you already have (but aren't looking at) to predict student success? How can you create incentives to drive the adoption of high-impact practices? How can you build caring, mentoring relationships that increase student success? We dive into these questions with Jeff Doyle, Director of Fraternity and Sorority Life at Southwestern University and prolific blogger/poster of "Deep Thoughts on Higher Ed."
Moving Beyond Grades to Predict Student Success
For decades, higher education has relied on a standard set of metrics to determine if a student is "succeeding." We look at GPAs, credit hours, and retention rates. But as Jeff Doyle, Director of Fraternity and Sorority Life at Southwestern University, points out, these metrics often measure what is easy to track rather than what actually impacts a student's life. If we want to truly move the needle on student success, we have to start measuring what we actually care about.
The challenge today isn't a lack of data; it’s that universities are often twenty years behind industries like retail or social media in how they use that data. We have the information—from card swipes at the dining hall to login frequency on Learning Management Systems (LMS)—but we aren't organizing it in a way that allows for meaningful, real-time intervention.
The Secret Metric: The Five-Fold Power of Belonging
Through extensive research at institutions like Baylor and SMU, Jeff Doyle uncovered a single variable that outperformed every other predictor of student persistence. It wasn't a student’s high school SAT score or their first-semester midterm grades. It was a simple question asked in week four of their first year: "Do you feel like you belong at this university?"
Students who disagreed with that statement were five times more likely to leave the institution within a year. Belonging acts as the lens through which students view every other challenge. If a student feels they belong, they view a poor grade or a personal tragedy as a hurdle to overcome at "their" school. If they don't feel they belong, those same challenges become reasons to withdraw entirely.
Incentivizing Success through High-Impact Practices
One of the most innovative ways to move from outcome-based goals to process-based goals is linking financial aid to high-impact practices. Instead of merely requiring a certain GPA to keep a scholarship, some institutions are now rewarding students for engagement.
By providing stipends or scholarship increases for students who participate in internships, research, or learning communities, colleges create a "process-oriented" incentive. Research shows that setting a goal to "perform a specific action" (like attending ten tutoring sessions) is far more effective for student motivation than setting a goal to "get an A." When we tie behavior to financial support, we communicate exactly what we value.
Busting Silos for Holistic Advising
The modern university has become a collection of specialists. A student might see one person for financial aid, another for career advice, and a third for academic scheduling. The problem is that no one is looking at the "whole student."
To create a truly connected college, institutions must move toward holistic advising. This requires a student-centric platform where different departments—from student affairs to academics—actually communicate. When we connect the dots between a student's dining habits, their LMS engagement, and their sense of belonging, we can intervene before a student even realizes they are at risk of dropping out.
Conclusion: Learning, Sharing, and Taking Action
Predicting student success in a changing world requires a commitment to constant learning. Whether it's embracing AI or simply taking the time to listen to student stories, the goal is to turn data into action. As higher education enters an era where expansion is no longer the default, the focus must shift to better collaboration and the consolidation of services to put the student at the center of the experience.
Episode 78 Transcript
-
Jeff Doyle: I just think what we measure says a lot about what we care about in higher ed, we could measure things that have much greater impact on students' lives if we would just take the time to listen to them.
Elliot Felix: That was Jeff Doyle, director of Fraternity and sorority life at Southwestern University, and a real font of knowledge on LinkedIn, his deep thoughts on Higher Ed blog, and pretty much every place else. I literally learned something every day from Jeff online. It's amazing, and I know you're gonna learn a lot from this conversation. We talk about using the data you already have, but maybe aren't looking at to better predict student success, we talk about how to create incentives for high impact practices and how to build caring mentoring relationships. Let's dive in. Welcome to the Connected College Podcast. I'm your host, Elliot Felix. I've helped more than a hundred colleges and universities change what they offer, how they operate, and how they're organized to enable student success. And if you're a leader in higher ed, and you think that the silos and separations get in the way of student success, then this podcast and my upcoming book, The Connected College, are for you. We're here to learn and work together to bust silos, question tradition, and forge partnerships so that students feel connected to their college, their community. Their coursework and their careers. So welcome Jeff. I'm excited for today's conversation about student success.
Jeff Doyle: Yeah, me too, Elliot. When I saw your book and read some of it, I was like, holy cow, this guy is sharp and I would love to talk to him more.
Elliot Felix: That's a high bar. I feel the same way about you. I think you're a wealth of information, both real time taking the pulse of what's happening and where we're headed, but also looking back and understanding where we came from. I think we have a great conversation in front of us. I think a great way to start is understanding how you did. Tell us a little bit about your own higher ed journey and how you got to, to where you are today.
Jeff Doyle: Sure. Yeah. My father taught for 30 years. He was the first in his family to go to college and ended up getting a three degrees and teaching at a historically black school for 30 years in Maryland. My mother was a reference librarian, a grammar person, so educational learning was huge. I was a big science and math guy and pre-med, pre-vet until I realized I was allergic to a lot of animals. And did research in a medical school at UVA where you also went to college. And I was like, okay, this is not my cup of tea. I could do it, but I don't love it. I love the things I'm doing outside the classroom and how could I help people experience the meaningful nature of a life outside the classroom the same way I have where I've learned so much. That's where I discovered you could get a master's degree in this. Worked in it for four years and I discovered I could get a doctorate basically in how universities work. And I was like, okay, these are pretty unique instruments of societal impact. So basically I spent almost my whole life on a college campus growing up and learning and trying to create meaningful experiences for students.
Elliot Felix: Life on a college campus. Sounds pretty good to me. I think I first came across your prolific thinking when you were not thinking outside the classroom, but you were also in the classroom, I think you were teaching a class on the future of higher ed not too long ago as well. Do I have that right?
Jeff Doyle: Yeah. So our university has what they call free school in the summer where anybody that wants can teach a class. And so most people teach paddle pickleball or paddleboarding. And I was like I really the future of higher ed. So I created a course and that invited people from my LinkedIn, I think I've got like 12,000 followers, and ended up with a bunch of provosts and other people meeting weekly to discuss the future of higher education.
Elliot Felix: Did you get it all figured out?
Jeff Doyle: Yeah. Oh yeah, we solved it, nice. No, it was really about just discussion and thinking. And that's the enjoyment of connecting and having a space where you can talk about that. You might not solve it, but at least you have people that care about the topic.
Elliot Felix: And out of that, did you come away with a definition of student success?
Jeff Doyle: I've wrestled with that definition for decades and. It's always, an iterative process. Lately I've been focused on what do students bring with them, what do we do in relationship with students when they're here and what can we do behind the scenes to better understand what's happening? I used to talk a lot about the big three, which is that middle component of how do we interact with students, and that had to do with finances, academics, and sense of belonging. In particular, I've honed in on that one piece of belonging because we found through our research at several schools how important that was and how little it's included in predictive analyses because people don't know how to assess it, measure it, and impact that one.
-
Elliot Felix: Yeah and you have a post on this, the nine. And of the nine ingredients of persistence or for student success? The health, the purpose, the self-discipline, the kind of habits that they come in with. The belonging, the finances and the academics when they're there. And then the things that universities have to do behind the scenes. Assessment, outreach, technology. Three legs of the stool, and each leg has three sub legs. I think it's a good structure.
Jeff Doyle: The brain remembers things in small increments, so I created in a way to help me and other people remember it.
Elliot Felix: Yeah, I think threes is good. That's how I do all my talks. I there's always a three act structure and I try and pair each. Each thing with a kind of a barrier or a challenge, a story of getting over and around it or through it while a adapting to some kind of trend that's reshaping higher ed. So I'm a big fan of the rule of three, even if you have to have three sets of three. So if we're really trying to define student success, thinking about kind of the inputs, the outputs and the experience in between, and we're trying to think in particular about belonging, academics, and finances. I'd love to hear your take on how we predict student success. I know this is a, something you've put a lot of thought into and researched and. You've identified some of the kind of typical metrics and predictors, but also some of them more unconventional ones.
Jeff Doyle: A little aside that'll take us to where we wanna go is , I love TikTok. I have over 5,000 tiktoks and over 50 folders, and my TikTok feed gives me incredible insights. And why does it do that? It knows what I like to learn from and read and think about. Right. And we're on Amazon. Why does it always tell me the things that I think I probably do wanna get? Why can't we do that in college? Why are we 20 years behind? Because I can tell you right now, and you know this 'cause you and I have talked about it, that there are a lot of those little things. We can learn from students that tell us what they might most need. If a student has a meal plan and they never go in and eat it and use it at the dining hall, why is that? I don't know the answer, but at least we should know if they have a, if they live in a dorm and we have a card swipe and they haven't card swiped into the dorm for a month straight. No one's looking at that? Wouldn't it be nice to know why this student who's paying all this money has never gone into their dorm? Maybe someone's always letting them in illegally, or maybe they're living somewhere else, the same thing with the Learn Learning Management system. Like during COVID, I volunteered to take over liaising with all of our fully online students, and that's what I did. I worked with our online system and said, this student hasn't logged in. To our LMS for a week straight. What's going on? People were like, I dunno, that's not on anybody's radar. And I'm like I'm gonna call the student, call the students. Yeah, my, my mother just died from COVID. I hadn't told anyone. It's pretty traumatic. I don't know what to do. And so we were able to intervene and help that student at that point in time. But, universities, we have a lot of this data. We're not organizing it in a meaningful way to impact lives for the better.
Elliot Felix: Something I've been hearing a lot of lately is it used to be that, we didn't have the data, now we have it or we have too much of it and we're not looking at it or organizing it or making sense of it. And in terms of things that maybe have untapped potential, you just mentioned a few things where it's like we're not really looking at the usage of. Of resources, whether it's a meal plan or a access to a dorm. What are some of the other predictors of student success?
Jeff Doyle: Sure. One of the ones I'm most proud of was when you go to college and you get a scholarship, how do you keep your scholarship? It's usually related to your grades, right? But grades are, that's like measuring with the, one of the loosest. Abilities to study success. And KU and Kinsey and some other people put out these high impact practices 25 years ago, and they said thousands and thousands of studies show that these practices most impact student learning and success. So at the one of universities I was at 10 years ago, they were like, how could we better use financial aid? And I was like, why don't we link financial aid to high impact practices? That's a great idea. People on board. And so we started out with $500,000. By the time we finished, when I left that role, we had $6 million a year, which we were giving out to lower income students who were involved in a high impact practice. If you stayed involved in a high impact practice every semester you did, you got $1,500. So you get $12,000 scholarship if you were involved in high impact practices. And then we just went back and we said, okay, what about these students is different than these exact same group of students who didn't do these high impact practices? And suddenly we realized there's a statistically significant difference. We proved what we already knew, but no one's doing, which is linking behavior other than grades. To success in college. And so we ended up yeah, publishing an article in the Journal of College Student Development on why aren't more schools linking scholarships to high impact practices.
-
Elliot Felix: What I love about that is you're creating an incentive. That communicates the value of something, but you're also attaching it more to the process than the outcome. And one of the really interesting things I learned in the research for how to get the most out of college is the difference between process and outcome oriented goals. And it turns out that outcome or performance based goals. Are much less effective than process oriented goals for students. So this was a study of well over 5,000 students who set a process based goal, like, I'm gonna take 10 practice tests did way better than students who set a performance based goal. I'm gonna get a B in calculus. It's much more concrete, when you've done the 10 practice tests. And as long as that. Process is a useful activity. It's much more motivating. And it sounds like this is along those same lines, right? Let's get folks doing high impact practices and tie the aid to that more so than the outcome, which we know will result if we're so engaged. I think there's power in. Information that confirms intuition. We did a study not too long ago where we looked at sensor data from 38 different universities and we looked at usage of the library. And if you had student success functions in the library, it was used 25% more than when you didn't. And if you had more books in the library, it had no impact, virtually no impact on the gate count. You're nodding your head. Our listeners can't see that. But I think a lot of people are nodding along with us. It's intuitive, but it's nice to have the data to make the argument. So I appreciate creating an incentive around high impact practices. What are maybe one more, one other sort of family of unconventional student success metrics that have predictive value.
Jeff Doyle: Yeah. One of the things I was gonna mention was this weekend I had some time, I've had this project I wanted to do for a while. So I read this study outta University of Chicago, studying college ranking systems and issues with it, it was very robust, 50 page paper. And then I went and created a spreadsheet of all the different factors that different the top ranking systems use to evaluate a great college experience and the things that we're talking about. Hardly any of that is in there. And if it's in there, it's an outcome measure, first of all. But you've got things that are five to 10% of what's the number of times a faculty publication have been cited by a journal in the top 10 and 25%. What's the faculty to student ratio? We've been using that for over 60 years. And can anyone show me a study that conclusively proves that ratio is essential to student success? There's some intuitive sense, but to make it that a portion of one of 10 characteristics of measuring the success of a college the number of people. Gladwell had an entire chapter in a book about right. Once that number gets too small, you actually get more ineffective learning 'cause you're not learning from enough people. Yet we're, yeah. You're not in a diverse enough group with different points of view and perspectives and backgrounds. Yeah. Yes. Why aren't we really thinking about. Student success here. There was a podcast episode on dining halls and this one college spent all their money on fancy good dining food. And this other school was like, we want to feed as many students as possible. And yet the one that the former got much higher rankings and much more positive reviews. Because they served, filet mignon or the other one was meeting the needs of students. And so sometimes I just think what we measure says a lot about what we care about in higher ed, we could measure things that have much greater impact on students' lives if we would just take the time to listen to them. And that's something we had done. At the school I was at Baylor, we had a hundred variables, and every semester we looked at our retention rate and we looked at these variables and we said, which of these were most predictive forwards and backwards? Finally we came down to this one variable, which was five times more predictive than any other variable out of all the variables. And it was a question we asked in week four of their first year in college. We said, do you feel like you belong at Baylor University? And if a student said, I strongly disagree or disagree, they're five times more likely to not be there 10 and a half months later at that college. And you look at all the factors and you're like that one little question. How many schools are doing pulse surveys in, in weeks four to six? And yet that's when you are finally deciding if this new culture is a place where you, is it for you? Yeah. Yeah. And if it is, you're gonna make it. And if it's not, we gotta intervene. And so I started going to my staff saying, these people will not be here. Yes. And my staff were like, I've got a job to do. I don't know what you're talking about, belonging and connecting with these people. Like we don't know how to. Do that. We take care of, where, what the facility of their dorm is and if this student group as an officer, but I was like no, but this, we know this makes a difference. And so yeah, sometimes aligning our structures, which is what good to great. A book came out about 30 years ago, was all about was the more you align to what you say is a priority, the more successful you'll be. And if. We know a sense of belonging is important, then universities should be spending a lot of time thinking about how do we create that? I went, then we went and interviewed the people who most said they didn't belong, and many of them were African American women who didn't have a sense of community on campus for multiple reasons. So suddenly we had a segment of the population that we could provide a major leverage to if we just provided more resources and avenues for them to feel at home at our university.
-
Elliot Felix: Yeah, I think, good to Great is, it's an interesting reference because, part of what that centers around is finding your key metric, right? Finding your, the metric that drives your business success. Hedge your hedgehog concept. Yeah. How do you measure your head sog concept and, I think it really is, it is belonging, right? Because that's like the lens that you see everything through. And I think focusing folks on that is so important. It's interesting that you uncovered that had, five times the predictive value. I know when they first started adding belonging to the, to Nessie a few years ago, it showed. Similar predictive power. I think students who said they belonged were about 40% more likely to persist from first to second year.
Jeff Doyle: Just to let, just follow up with one other example. I went to SMU eventually as director of success and retention student success. We had students that would leave the university and we'd say, are you gonna come back? We have a leave of absence or a permanent withdrawal. And the difference between the people that said leave of absence and withdrawal was one thing. If they felt like they belonged at that institution, they were gonna take a leave of absence. If they didn't feel like they belonged, they were withdrawing. And so it was just another example of. How do we build connection? Because even if they have poor grades or they have a tragedy that happens, they still feel like that's their place and they're gonna come back. Yeah, I'm coming back because I still feel like I'm a part of that place.
Elliot Felix: So if belonging is is the key predictor for student success, why aren't more folks measuring it? I know there's, national college health assessment and there's a bit in Nessie, but why isn't it like the gold standard? Why isn't everybody measuring it and the contributors to it, do you think?
Jeff Doyle: It's like when you do a search on chat, GPT, it gets you an answer. With using as little power and energy as possible. That's basically what I've learned how to refine my searches, is you gotta push it and challenge it. And man, it is so easy to measure with grades and endowment and research productivity. We've got measures for all of that. You wanna tell me how to measure belonging? That's a whole different game and it's constantly changing. You and me ask us, how do I know when I belong? I don't even know how to articulate that. If you had to study, how do you build a meaningful friendship? The minutes that it takes, I've got two guys I'm trying to develop a friendship with. I meet with them weekly for an hour and a half for dinner, and I'm basically coaching them on, I want more friends my age, and this is how I read. We do this and it's very maybe weird to some people, but these guys have bought into it. And I'm like, we have to invest time. Okay, now put that on a college campus. How do you translate a time investment in relationships where people hear each other and connect with each other? It's very difficult to get to it. It doesn't mean we shouldn't do it, but if you're just trying to get a quick answer and someone wants a piece of data to present to somebody, this is not the easy way to go.
Elliot Felix: So it's hard. Because it's so nuanced and squishy and emergent. It's hard to measure. You might be able to measure it as a, as an outcome, but the contributors to it are harder. And I do think it's, it is really something we have to pay attention to the. The biggest survey I know that really looks at it is the American College Health Assessment or the National College Health Assessment. That shows it's only about 65% of students feel a sense of belonging and they ask it directly. It's just a single question, do you feel like you belong at your colleges uni or university? But there are so many factors and facets. Of that to know what's driving it and to know what lever to pull. Yeah. But it's so critical and it reminds me of the, the Gallup Q 12. Everybody's always skeptical or poking holes at the, I have a best friend at work, but the reason that question's been in there for decades is that it's a great predictor of employee engagement, right? Of whether, yeah, of whether people are enthusiastic about and involved in and committed to their work. And I think belonging is a similar thing in the college environment, in the learning environment.
Jeff Doyle: Yeah, look. Go back to the 2014 15 studies on Gallup Purdue studies that looked at over. Oh yeah. Those are great. 300,000 people who were meaningfully engaged in their work post-college. And they go back and said, what was it about that are different about these people than those who aren't having a good experience in their jobs? And they were like, there's three big things. Yeah. One of them was. People cared about me where I went to college. And another of the big three was someone took the time to meet with me one-on-one to get to know me. So when I read that, I started, and that experiential learning was the third leg of the stool, right? Capstones. Exactly. And internships and, yeah.
Jeff Doyle: Yep. So I started meeting I was like, okay, meeting with every student in my class is gonna be a major time suck, but I'm gonna try that. I met with every student in my class one-on-one and got to know them. My teaching vs grew sky high. I was asked to present to the department. The students had a wait list to get into my class. They're like, this guy works as hard, but man, he cares about us as people and. I was like, okay, that's worthwhile. How do we incentivize mentorship and creating a caring environment? Those are some questions I'm always asking myself.
Elliot Felix: And I think that tracks, it also tracks with the Google project Oxygen, right? With is now a while back, but when they did a massive survey of what makes a good manager at Google. And one of the key things was, the managers who receive. Good ratings from their team and whose teams perform well. They're the ones that, demonstrate an interest in the whole person, who ask you how things are going, who are aware of how things are going, who take the time to build the relationships, to meet with people one-on-one. It isn't just technical expertise or good communication or. Clarity of vision and strategy. It's also that, that relationship building.
Jeff Doyle: yeah, I mean I, when you look at great companies from a good to great or other publications, that's often what they do. They have core values that they live out in their day-to-day work. And so in some ways, that's what I would encourage colleges to do, is to put in place practices, behaviors, and processes that show that your values really are what you care about.
-
Elliot Felix: And so if you had a magic wand and you were gonna help a whole bunch of colleges and universities better measure and move the needle on belonging, what are some of the contributors that you'd recommend they look at? And what are some of the interventions you might recommend to, to make some progress.
Jeff Doyle: It starts with collecting the data. Then actually learning to look at it. Right now we have 25 different student affairs units. We have 10 different academic support areas. We've got administrative people all throughout campus. But they're not communicating in a meaningful way on a student-centric platform.
Elliot Felix: And this is any campus, right? This you're talking about?
Jeff Doyle: Yeah, this is, yeah. At any campus, this is not just ours. And so how do you put students at the center of what we're doing to understand what's happening to them so that we can communicate? One of the things I've been writing and thinking a lot about lately is we have academic advisors, we have career advisors, we have financial aid advisors, we have student involvement advisors, we have all these different folks, but who's doing the holistic advising? Who's taking the time instead of, I can help you with your classes, but that's it. I can help you with getting a job, but that's it. And so part of me is have we become like the medical field where you see a specialist for everything, but no one is like the internist that sees the full picture on who you are. Do we need a shift slightly back to looking at the full picture of who a student is and what would most benefit them. So I've been asking myself that is how do we collaborate more and use students as our fulcrum for understanding how to impact change.
Elliot Felix: Yeah. I think that's a good playbook, right? The center on the students collect the data, connect the dots, so you're seeing. The full picture. And I think that's also how folks meet the moment. I think higher ed has a tried and true way of how you meet new needs and it's to add staff and services and programs. And I think the era of Expansion is over for all but a handful of institutions. And it's really about consolidating these different functions, collaborating better, co-locating different services and programs and departments. Because, flat is the new up. And if, if we're not working together I don't, I just don't see it happening. I think that it's like the silos persist instead of our students.
Jeff Doyle: Yeah. That's what I really loved about your book was that you actually took the time to study that and write about it. And I think, I actually, in my classes, got away from having students read books 'cause they wouldn't read 'em. But I would like to, you didn't ask me to do this, but I would put a plug in for Elliot's book if you guys haven't read it or listen to it or some, I don't think he's recorded it yet. That's what I'm waiting for him to do. No, the audio book audio book will be out this summer, but.
Elliot Felix: Oh, that's awesome. Yeah. Good. You have a lot of great insights in that. Thank you for saying that. I appreciate it. I guess just to wrap up your final advice for folks who were trying to predict student success in a changing world, how should they adapt?
Jeff Doyle: Yeah, I'm 55 and I'm starting to feel a little old. I constantly learn, my father, every morning he got up, he would read the Washington Post at 5:00 AM and circle two or three articles he wanted me to have read before dinner that night. 'cause we'd be talking about them. There you go. And I started, when I started getting on LinkedIn five or six years ago, that's what I do is I read and learn and I share. And I think we just have to keep doing that. I wasn't interested in AI at first, and then I was like wake up and start doing this 'cause you're gonna need to understand it. And so every morning I ask myself, how can you challenge yourself today to learn something new and share that with someone else? And and so that's a good start, I would say for anyone is when's the last time you've read an article or listen to a podcast and what are you gonna do as a result of that? I often tell my students like. First, I wanna know that you enjoyed it, then I wanna know what you learned, then I wanna know what you did as a result of what you learned. Because we have to take action based on our learnings. There's a lot of things we're teaching, but if people don't act on it, then we haven't succeeded. So I try and build those habits, . How do I put something new into my life that I will now do for the rest of my life?
Elliot Felix: You're certainly modeling that and leading by example and doing the what, the, so what the, now what for sure. And I've certainly been learning from you and I'm sure our listeners will too. So Thanks Jeff.
Jeff Doyle: We both went to UVA, right? Yeah. We gotta put a plug in for alma mater here. Yeah. Not that, that's another, another way, and another place to, to learn for sure. But at UVA, they do give students maximal autonomy to learn and grow, and I think both of us benefited from that. So thank you for your time.
Elliot Felix: Thank you. Thanks for listening to the connected college podcast. Subscribe to my Connected College Newsletter at ElliotFelix. com for insights and excerpts from my upcoming book, tools you can download, and special offers. You can also find more information about talks I've given, articles I've written, and upcoming events there, and please support the podcast by rating and reviewing it wherever you're listening. Let's create connected colleges where students will succeed.