Working Groups

Timeframe: February 2019 – March 2021

Chair: Kathy Miller

Members: Kitch Barnicle, Jeffrey Franke, Paula Kavathas, Lisa Kelly, Bob Mathieu, Shannon Patton

Charge:

Since 2003, CIRTL operations have been administered within the Wisconsin Center for Education Research at the University of Wisconsin – Madison (UW). This administration includes financial management, human resources, technical services, legal services, grant management, etc.

The origin for this operations arrangement lies in UW having been the host institution for the funding grants for CIRTL. However, as of 2019, CIRTL no longer relies on external funding for normal operations. Normal operations are supported by member dues (with WCER services provided through indirect charges on those dues). 

While the services provided by WCER remain excellent, the deep administrative connection to UW poses significant vulnerability for the CIRTL Network. Perhaps the most significant vulnerability of the current UW-based organization is the ongoing need for a UW administrative PI. 

The charge to the CIRTL Organization 2.0 Working Group is to analyze the advantages, challenges, and vulnerabilities of the current organizational structure, and to explore and develop several feasible plans for the next operations model for the CIRTL Network. These plans can be wide-ranging, but must:

• Be financially self-sufficient;
• Provide the basic services required to sustain the current operations of the Network; 
• Include basic infrastructures such as a website and online platform for Network and cross-Network activities;
• Remain viable with either growth or shrinkage of the Network;
• Permit straightforward leadership change in the CIRTL Network Director role.

The Working Group may want to begin with the findings of a preliminary study in 2015.

The Working Group report should present the Network with the advantages, challenges, benefits, and risks for each organization plan in order to provide a foundation for Network discussion.

The Working Group will regularly provide updates to the Leadership Team and to the CIRTL Network leaders. The Working Group will provide a report to the CIRTL Network leaders at the Fall 2019 In-Person meeting, with a formal first discussion then. 

The Working Group is not charged with the implementation of CIRTL Organization 2.0, although it is anticipated that some members of the Working Group will also participate in that process for continuity.

Timeframe: April 2019 – December 2019

Chair: Judy Milton

Members: Colleen McLinn, Gina Hurley, Robert Mathieu, Rachel Kennison (joined October 2019), Kathy Takayama (joined October 2019)

Charge: 

One goal of the CIRTL Spring 2019 in-person meeting was to take the first intentional steps toward becoming a multicultural organization. At the meeting, two working sessions provided facilitated brainstorming of CIRTL values. 

The charge of this working group is to draft value statements for CIRTL, using the group’s own wisdom and experience with CIRTL and drawing on data collected from CIRTL Network members during the Spring 2019 in-person meeting at the University of California, Irvine. The group will then present their draft value statement to the CIRTL Network, iteratively gathering feedback and refining the statements until a final set of statements is approved by the Network for adoption.

Timeframe: July 2018 – May 2019

Chairs: April Dukes, Lauren Woods

Members: Sarah Hokanson, Kate Diamond, Marisa Dietrich, David Lawrence, Chris Chen, Suzanne Young, Rachel Kennison, Kitch Barnicle, Katie Ingraham Dixie

Charge:

The CIRTL Network embraces the importance for the development of future faculty of engaging with the diversity of the Network partners and of each other. For this purpose, the CIRTL Network has invested and continues to invest substantially into the creation of a rich curriculum of development opportunities in the Cross-Network Learning Community.

While evaluation results show that participating future faculty value and recognize the impact of these activities on their development, the number of participating future faculty has been less than capacity and far less than desired by the Network.

At the March 2018 CIRTL In-Person Meeting at Yale University, there was a call for an intentional effort to increase participation, as well as a brainstorm of ideas for doing so. This call was repeated at the May 2018 CIRTL Administrative meeting.

To be intentional about increasing future faculty participation, the CIRTL Network will create the ad hoc Working Group on CNLC Future Faculty Participation. The charge to this Working Group is:

1. Develop and solicit ideas for interventions and actions to increase future faculty participation in the CNLC while achieving high-impact outcomes in future faculty development; this may include investigation of existing or new data of relevance;
2. Lead Network-wide discussion of these ideas over the course of the 2018-19 academic year;
3. Synthesize input and create a proposal of integrated actions for presentation to the Network by the Spring 2019 Network meeting, including evaluation metrics; revise for Network adoption.

Timeframe: December 2016 – October 2018

Chair: Jeffrey Franke

Members: Kathy Miller, Jim Grover, Adam Fontecchio, Kate Diamond, Sarah Hokanson, Bennett Goldberg, Bob Mathieu, Cora MacBeth

Charge: The DPWG is charged with recommending CIRTL-wide issues and processes for collective decision-making, and developing plans for implementation and metrics for success of the approach(es) that align(s) with CIRTL values and operational needs.

Timeline: The DPWG will provide a written status report to the community of CIRTL institutional leaders two weeks prior to the Spring 2017 in-person meeting. During the in-person meeting, members of the DPWG will facilitate a community discussion around any preliminary recommendations or issues identified by the working group. Following the in-person meeting, the working group will prepare final recommendations to be adopted following inclusive discussion and consensus of the institutional leaders. If the final recommendations are not approved, an action plan for considering alternative decision-making processes will be created by the working group. 

Process: The DPWG will publish its plans and procedures, as well as its meeting minutes. Guiding principles for DPWG operations will include: (1) a commitment to aligning the process and the proposal to CIRTL core values, (2) the use of evidence and scholarship to direct DPWG recommendations, (3) regularly engaging the community in discussions of network governance models and decision-making approaches as the process evolves, (4) developing and implementing methods that provide the opportunity for all members of the community to provide feedback at various points in the process and have such feedback be available to all.

Membership: The membership of DPWG will be representative of the CIRTL community, including the leadership, operations, and working groups, as well as representatives from second, third- and fourth-wave institutions. Specifically, the goal is that the DPWG will include a member from the CIRTL Leadership Team, a staff member from CIRTL Central, a representative from the Cross-Network Operations Group, and one representative from the leaders of each of the second, third, and fourth waves, not overlapping with any other membership classes.  Up to an additional three members may be added without constraint.

Timeframe: November 2016 – August 2018

Chair: Lucas Hill

Members: Henry Campa, Kelly Clark, Mark Connolly, Mark Graham, Robin Greenler, Jacqueline Fajardo, Denise Leonard, Xiufeng Liu, Cora MacBeth, Jacinta Mutambuki, Denise Pope, Jennifer Stanford, Devin Wixon, Lauren Woods

Charge:

To date, by design, CIRTL has purposefully allowed member institutions to develop their own approaches to evaluating their professional development programs. The rationale has been that CIRTL leaders should exemplify the Teaching-as-Research approach to incorporate inquiry and evaluation into their work. Furthermore, in developing the CIRTL Network, CIRTL leaders have been aware that institutional members prize their autonomy. Thus, while asking member institutions to commit to a common set of learning outcomes, CIRTL has honored the autonomy and variability of context at individual member institutions by leaving the details of local programming and its evaluation within the purview of institutional responsibility. However, as 25 new institutions have joined CIRTL, we are aware that assessment of impact on future faculty will be enhanced by a database of questions, measures, and instruments that institutions could draw on, if they so choose. Practically speaking, we also know that new institutional leaders are looking for guidance as they develop and engage in early assessment of the institutional programs they develop.

The Working Group will create a databank of questions and measures that can be used to create instruments for use in evaluation within member institutions. This will involve a review of questions, measures, and instruments already in use at CIRTL institution to identify the most promising of those questions, measures, and instruments for use in local-level evaluation. Questions and measures will be designed with explicit effort to relate to the CIRTL Learning Outcomes. While institutions will retain their right to select their own approaches to evaluation of local programming, over time, we hope that institutions may agree to the use of some common measures, questions, and instruments. The Working Group’s charge to create a databank of questions and measures is an important step toward the possibility of common evaluation questions and measures across many institutions in the Network.

The intention is to develop questions and measures that would be particularly useful for the following purposes:

• Pre/post surveys for use in gathering evaluative data from participants in short-term CIRTL program activities (e.g., workshops, seminars);
• A common rubric for use in assessing, providing feedback, and gathering comparative data on the Teaching-as-Research Projects;
• Interview protocols for use with participants in long-lasting (e.g., semester or year-long) programs to support TAR experiences;
• Assessment of the impact of CIRTL on member institutions. 

Timeline and Time Commitment: The group will meet regularly approximately every month for one year. Additional meetings, either with the entire group or sub-teams assigned to particular activities, will be called as needed, with the length of each meeting dependent on the work to be accomplished. There is the possibility of the working group continuing beyond the one year as progress and interest dictate.

Member Expectations: A committee led by a member of the research and evaluation team (Lucas Hill) and composed of several interested institutional leaders and colleagues from CIRTL member institutions will work collaboratively to identify and develop the questions, measures, and instruments described above. Members of the Working Group are expected to be actively involved in the activities of the group to meet the Charge and to be prepared to commit to the necessary time.

Invitation: We invite any institutional leader, administrative co-leader, and CIRTL central leader to participate. We also invite any campus colleagues who have particular experience or expertise in program evaluation to participate.

Outcomes of the Working Group

The Evaluation Instrument Working Group extensively reviewed a decade of current and prior evaluation instruments used for local and cross-network programming and compiled an instrument database. Given the variability and scope evaluations and approaches across the Network, the Working Group focused on developing: (1) an Evaluation Resource Guide to provide practical examples and resources for leading local program evaluation, (2) a rigorously-collected set of evaluation question stems that can be customized for local programmatic context, (3) a small set of instruments that could be used in common across the Network, (4) approaches for measuring CIRTL learning outcomes, and (5) templates for surveys for low- and high-engagement CIRTL programs that can be customized for local program context and contain key evaluation question types.

In the future, the Working Group recommends that the Network identifies a small set of question stems they seek to make common across the member institutions to track common outcomes and where possible, a set of items that match shared curriculum or concepts. Despite institutional variability, there was still much agreement about types of questions used in evaluation instruments, which supports great potential in developing common measures to better track the impact the CIRTL Network on participants.

Products:

• Local CIRTL Program Evaluation Resource Guide
• CIRTL Program Evaluation Questions
• Common Teaching-as-Research (TAR) Interview Protocol
• Common Teaching-As-Research (TAR) Rubric
• Common General Interview Protocol
• Measuring CIRTL Program Outcomes
• High Engagement CIRTL Program Pre-Post Survey
• Low Engagement CIRTL Program Post-Survey

Timeframe: July 2017 – February 2018

Chair: Jeff Engler

Members: Steve Ackerman, David Daleke, Kitch Barnicle, Shannon Patton  

Charge: 

1. The working group will develop an addendum to the network MOU covering policies for sharing non-human subjects data among Network members. The data-sharing agreement should be developed with input from all Network members and be reviewed by WCER’s attorney and perhaps other administrators at UW-Madison
2. Generate a list of the categories of data collected centrally and develop guidelines on who within and beyond the Network may have access to the data (e.g. IPs, cross-network data, Institutional leader survey data such as the needs assessment data, other operational materials).
3. Draft policies on how requests for data are to be processed. (Who does the request go to? What is the timeline for providing data? What is CIRTL Central’s responsibility to extract and ‘clean” the data? Does the Network need to be notified if someone party to the data-sharing agreement requests shared data).
4. Draft policies on how shared data may be used in publications and presentations with respect to attributions and authorship. [Presumably data cannot be used for research purposes without a prior IRB approval].