Impact of LPSOE series: Idea 1.

Pavan - Great idea!

Another conjecture is that the existence of L(P)SOE faculty generally increases the amount of attention other faculty pay to teaching matters. This might be measurable by things such as self-reported number of conversations about teaching practices, number of teaching related workshops attended, number of curriculum updates considered by departments, number of faculty mentoring undergraduate research students, etc. The toughest part with these will be establishing baseline and getting critical mass of measurements. It also may be challenging to directly attribute observed changes to our existence, as opposed to general trends in education attention based on technology.

Sarah and Carrie, you both make excellent points.

1. I agree that student expectations, etc. might be determined by demographic factors - however, what we would be measuring is a change in these numbers, with at least some of that change attributable to the LPSOE series.

2. Self-reported faculty measures of the number of teaching-related conversations might be hard to interpret; but again, I think it's a great idea to incorporate as part of the entire package. To my mind, no one measure of "success" will be independently sufficient. I think it will be the impact we can show in a large number of related ways that will be most convincing. So, this would be one additional data point. We could also add some aspect of "learning about teaching methods" to this self reported measure, which might help us tease apart general conversations about education from what LPSOEs bring to the table.

While I agree that we should ultimately look at student outcomes, it may be difficult to collect this information without doing something like the Northwestern study, which compared different groups of faculty.

Instead, as a first pass, I think that we can look at departmental teaching "climate" (loosely defined). The results would likely be in the range of better to no change. There are some departmental "climate" surveys that are being developed, and I can try and look them up.

Another way to look at this is using Pavan's point #2 and develop a network analysis for who talks to whom about teaching. If we have departmental buy-in, this could be an annual survey in which everyone is asked to check off from a list of names all people with whom they had teaching conversations with in the past year. The survey would take fewer than 5 minutes, but the data could be phenomenal using network analysis.

Here are some hypotheses. The first one deals with general impact, and the last two deal with the potential contribution of teaching faculty.
1. Does network density (number of connections) increase over time after hiring teaching faculty and/or in departments with more teaching faculty?
2. Are teaching faculty more likely to be hubs or central actors in the networks?
3. If there are sub-networks within departments, are teaching faculty more likely to be liaisons who bridge multiple sub-networks ?

We can also look at more nuanced analyses such as people who teach the same course(s), how often do people talk about teaching, or the purposes of these conversations. Also, depending on the number of departments and their associated the variations, there may be interesting research questions that we can develop from comparing different types of networks.

All great ideas. As far as establishing a baseline for the impact of LPSOE's, specifically in BIOSCI, UCD may be a good place to look to. The first 'cohort' of LPSOE's have just been hired so there is an opportunity to survey both students and faculty prior to having LPSOE faculty in the department and then track specific impacts over time, such as student experience, faculty attention to teaching, changes in teaching methods used within the classroom, faculty participation in pedagogical development, etc.

Ideally, I think that having longitudinal data (e.g. beginning of fall quarter every year) would allow us to examine changes over time in different departments and institutions with varying numbers or intensities of L(P)SOEs. Having a "pre" condition before the arrival of any L(P)SOEs is interesting, but we may not see meaningful changes until after a few years. We will probably have to collect data for a few years and then examine in a naturalistic way (kind of like natural history) to see how patterns and variations emerge. I think that multiple "post" points (e.g. annually) would be better than one "post" point.