News & Events
Essential to improving teacher quality is ensuring that the right data are available to inform the policy and practice changes needed to continuously improve educator preparation program (EPP) quality, teacher effectiveness, and ultimately student learning. Unfortunately, that data are not uniformly available today.
This paper describes how data sharing among states, EPPs, and K-12 leaders can help ensure quality teaching and learning. The paper:
- Discusses challenges to using data to improve educator preparation
- Presents state policy recommendations
- Profiles policy in action in Massachusetts, Tennessee, and Missouri
WestEd’s Ellen Mandinach served as a member of the working group for this paper, developed collaboratively over one year with a group of national, EPP, and state-level experts.
Ellen B. Mandinach contributed a blog post, which can be found below, on data literacy in early education to the National Institute for Early Education Research (NIEER).
Data have been used in education for many years. Good teachers and administrators have been using data to inform their practice and make decisions. Why is data use important? Because it is no longer acceptable for educators to solely use anecdotes and gut feelings to make decisions. Educators need hard evidence. To that end, there has been a growing emphasis for the past 15 years to make education a more evidence-based and data-driven discipline.
Data have ranged from accountability and compliance data to data for continuous improvement at all levels of education, but one issue that has loomed large is the conflation of data literacy with assessment literacy. The two constructs have been confounded for many years (Mandinach & Gummer, 2016a; Mandinach & Kahl, 2014). When educators and the general public think about data, they typically think about test results and student performance. They fail to think about all the other sources of data that help educators to inform their practice. Until fairly recently, there has been no clear definition of data literacy and certainly no analyses of the skills, knowledge, and dispositions that are needed to use data effectively and responsibly (Data Quality Campaign, 2014; Gummer & Mandinach, 2015; Mandinach & Gummer 2016a, 2016b). The work of Mandinach and Gummer, based on several years of research, has attempted to lay out what it means for educators to be data literate.
A foundation of data literacy is the consideration and use of diverse sources of data, not just the limitation to only student performance data. For educators to have a comprehensive understanding of their students, they must look to behavioral, attitudinal, motivational, medical, attendance, home context, and other kinds of data. Even though measures of teacher readiness such as the edTPA (SCALE, 2013) contain an assessment rubric, it makes clear that teacher candidates must be able use contextual information, “assets,” to inform their understanding of student performance.
Diverse sources of data are particularly important in early childhood education where teachers often must look beyond student performance results to understand a child. As Dwyer (2015) notes, some of the data most relevant in early childhood settings other than assessments (formative, summative, and diagnostic) include screening results, informal check-ins, child characteristics and experiences, attendance, health information, family language/education experiences, family conditions for learning, classroom observations, participation, walkthroughs, and staff experience and education. In workshops conducted across Pennsylvania for early childhood educators, Dwyer, Mandinach, Nunnaley, and Saylor (2015) noted several purposes for data use in early learning:
- Improve child outcomes
- Improve teachers’ skills
- Identify gaps in achievement
- Realign resources
- Facilitate parental engagement
- Improve program quality
- Increase access to high quality programs
- Change adult behavior
Dwyer and colleagues (2015) reflected on why data use is important in early learning settings, recognizing that evidence is important. They noted that there needs to be realistic expectations for how data use can inform and improve daily practice. Through effective data use, educators can:
- Reflect on practice
- Check assumptions
- Get others’ views
- Commit to new actions
- Attend to the effects of changes in practice
- Make practice public
But how does this happen? More than eight years ago, the Institute for Education Sciences (IES) commissioned a comprehensive review of the literature that existed at the time, recognizing that data-driven decision making was only then emerging as a hot topic in educational research (Hamilton, Halverson, Jackson, Mandinach, Supovitz, & Wayman, 2009). Some 3,000 research and implementation studies were identified with only a handful meeting the strict criteria for rigorous research laid out by the What Works Clearinghouse. Five recommendations were noted. For there to be effective data use at any level of education, schools and districts must:
- Make data part of an ongoing cycle of instructional improvement
- Teach students to examine their own data and set learning goals
- Establish a clear vision for schoolwide data use
- Provide supports that foster a data-driven culture within the school
- Develop and maintain a districtwide data system (p. iii)
These five recommendations have stood the test of time. The growing research in data use further confirms the recommendations. Much of the work firmly espouses the need for the introduction of data teaming, leadership, and the creation of data cultures within schools and districts. Data systems have morphed from data warehouses to dashboards and apps that provide real-time data for instructional decision making.
Yet despite having much of the infrastructure in place, particularly the billions of dollars spent on technology at the federal, state, and local levels, attention to the human infrastructure remains problematic. Fulfilling all of the five recommendations from the IES practice guide is an important step forward. However, if educators do not know how to use data both effectively and responsibly, the investment in attaining the recommendations will go for naught. Even though the field recognizes the importance of data use, the delivery of consistent and comprehensive professional development is often lacking and falls below other competing priorities.
As Means, Padilla, and Gallagher (2010) noted, professional development for data use must be ongoing, not sporadic. As Mandinach and Gummer (2016a) note, waiting until educators are in practice to acquire data literacy skills is too late. They must begin to acquire such skills at the earliest stages of their professional careers, that is, during pre-service preparation. Because of this growing need, WestEd and its collaborator, Using Data Solutions, is working toward the development of curriculum materials that can be used in teacher preparation programs to teach data literacy. The objective is for teacher preparation programs across the country to begin to integrate the construct, data literacy for teachers (Mandinach & Gummer, 2016a, 2016b) into their curricula. The ultimate objective is to create a teaching corps that knows how to use data.
The AERA Data SIG Business Meeting will be help on Sunday, April 30 from 6:15 to 7:45pm at the Grand Hyatt San Antonio, Third Floor Bonham C.
Please come and join the session. The session will include local Texas educators talking about how they use data.
An interesting book is now on the best seller’s list about decision making and pertains to all fields, not just education. Michael Lewis of Moneyball fame has written The Undoing Project. It is the story of two psychologists who studied decision making in fields such as medicine and sports. The two psychologists, Daniel Kahneman and Amos Tversky wrote ground-breaking articles on how decisions are made, the assumptions, and fallacies. For example, the hot hand phenomenon in basketball is really a myth. Kahneman was awarded the Nobel Prize for Economic Sciences for this work. A good and interesting read for those involved in decision making.
The AERA Data Driven Decision Making in Education Special Interest Group (Data SIG) is pleased to announce a new resource and upcoming webinar.
REL Appalachia developed the Teacher Data Use Survey (TDUS) in collaboration with a panel of expert researchers to help education leaders understand the kinds of supports needed to promote effective data use to improve teaching and learning in their schools. The survey helps identify how teachers use different types of data; teacher attitudes toward data use; how teachers collaborate to use data; and supports to help teachers use data.
The TDUS is authored by Data SIG members – Jeff Wayman, Vincent Cho, Ellen Mandinach, Jonathan Supovitz, and Stephanie Wilkerson. The survey comes in three versions: one for teachers, one for administrators, and one for instructional support staff. Together, these survey versions provide a full picture of teachers’ data use. The survey can also be customized to gather information on specific types of data, such as state, periodic, and local data. Using REL Appalachia TDUS reporting tools, survey results can be presented in a full data report in Word and as a visual dashboard.
Jeff Wayman and Stephanie Wilkerson are doing a webinar series on using the TDUS. The next webinar is on Friday, October 21, 2016. If you are interested in joining, please register.
The survey, implementation manual, and tools are available for public use:
- Teacher Survey.
- Administrator Survey.
- Instructional Support Staff Survey.
- Guide to using the Teacher Data Use Survey.
- Teacher Data Use Survey Report Template.
- Sample dashboard report.
September 15, 2016Event
Dr. Ellen Mandinach (WestEd) and Edith Gummer (Ewing Marion Kauffman Foundation) gave the keynote address at the Council for the Accreditation of Educator Preparation (CAEP) Annual Conference on September 29th.
The keynote is entitled “Data and Educator Preparation Programs: Data for Programmatic Continuous Improvement and Data Literacy for Teachers.” Drs. Mandinach and Gummer discussed how educator preparation programs can use data to facilitate their process of continuous improvement. They also introduced a new construct, Data Literacy For Teachers, and discussed why it is important for educator preparation programs to include instruction on all sources of data use, not just assessment results.
Mandinach and Gummer were available for a book signing for their new book entitled Data Literacy For Educators: Making It Count in Teacher Preparation and Practice.
A new report to the Bill and Melinda Gates Foundation titled “Blended Learning and Data Use in Three Technology-Infused Charter Schools” examines the affordances of the technologies in three blended learning environments and their impacts for teaching and learning activities. A particular focus of the work was to examine whether the blended learning environments provided access to and more diverse data sources for teachers and students from which to make educational decisions.
Key findings include:
- Blended learning environments provide data to teachers and students that may not be readily available in more traditional classes;
- Blended learning environments provide for any time and any where virtual learning opportunities;
- Teachers were able to address the needs of particular students through various media and diverse learning experiences;
- Students were engaged through flexible and customizable learning activities; and
- The schools exhibited strong leadership, an explicit vision for the use of technology and data, the engagement of students in the teaching and learning process, the enculturation of data use through data teams and data coaches, and the provision of professional learning opportunities.
There is much that can be learned from these three schools about how the alignment and practice of research-based recommendations can create blended or personalized learning environments that have the potential to reach even the most challenged students and help them to succeed.
DDI is proud to announce the release of two articles as part of a special issue of the international journal Teaching and Teacher Education. The special issue’s theme is on how countries are improving the capacity of teachers to use data.
The first article is titled “What does it mean for teachers to be data literate: Laying out the skils, knowledge, and dispositions.” The authors, Dr. Ellen Manindach (WestEd) and Dr. Edith Gummer (Ewing Marion Kauffman Foundation), were invited to write an article on a framework for a construct called data literacy for teachers. The article lays out the framework, identifying the specific knowledge, skills, and dispositions teachers need to use data effectively and responsibly. It concludes with a call to schools of education and teacher preparation programs to begin to integrate data literacy into curricula and practical experiences.
The second article is titled “Teacher learning how to use data: A synthesis of the issues and what is known.” The authors, Dr. Ellen Manindach (WestEd) and Dr. Jo Beth Jimerson (Texas Christian University) were invited to synthesize the articles in this special issue on data use. The synthetic piece contextualizes how the articles contribute to the knowledge base of how teachers use data.
July 22, 2016Announcement
During a standing-room only presentation, participants of the National Center for Education Statistics (NCES) Summer Conference heard “What Data Parents Want: Using A Data Dashboard in Missouri.” The presenters included Dr. Ellen Mandinach and Dr. Ryan Miskell (WestEd), Dr. Edith Gummer and Mr. Christopher Laubenthal (Ewing Marion Kauffman Foudnation), and Mr. Jeff Falter (Missouri Department of Elementary and Secondary Education). With the Every Student Succeeds Act (ESSA) requiring districts to engage parents and families, NCES participants recognized the importance of communicating education data effectively to parents.
The presentation focused on the types of information parents use and search for as they make educational decisions. This information is vital as states and districts develop dashboards of information for various stakeholders, as the Ewing Marion Kauffman Foundation has done with EdWise. During the presentation, the audience asked several follow-up questions :
- Q: How do parents feel about the applicable processes, documentation, and guidelines associated with schools and districts near them?
- A: These topics frequently come up, but they highlight one of the obstacles of parents’ desire for data–frequently they want information not easily captured or shared.
- Q: All of your 118 interviewed parents came from Missouri; is this study generalizable to other states?
- A: Given the cross section of parents geographically and economically, we feel this study is a fairly representative sample that can be used in different states. Only the very affluent being under represented in the study.
- Q: On the topic of economically diverse parents, did the digital divide present itself in your work?
- A: Absolutely. We met a set of parents in St. Louis who did not have access to computers or the internet. When given the Amazon gift cards one of them asked, “how do I even use this?” People are making assumptions that all students are connected to the Internet and it’s not true.
- Q: How can data address the topics of bullying and peer pressure? How can we share that with parents?
- A: School climate surveys like the Five E’s work in Chicago as well as interviews and testimonials can capture some of this information. We are still challenged by how to best share this information, but we need to continue looking into the matter. Knowing about bullying and peer pressure can save lives.
- Q: Once you have a tool like EdWise, how do you promote its use amongst parents?
- A: Education data just by itself can be elite and so increasing use presents difficult challenges. These challenges, like those we addressed in St. Louis, are something we keep bumping into. The solution is to find ways to meet parents where they are, computers or no, and to develop a culture and an environment that says data use is okay.
- Q: How do parents feel about the applicable processes, documentation, and guidelines associated with schools and districts near them?
The Data for Decisions Initiative is proud to announce two upcoming presentations during the 2016 NCES Summer Forum and STATS-DC Data Conference. The conference will be held at The Mayflower Hotel in Washington, DC from Sunday, July 10th through Thursday, July 14th.
- On Tuesday, July 12th, Dr. Ellen Mandinach and Dr. Ryan Miskell (WestEd), Dr. Edith Gummer (Ewing Marion Kauffman Foundation), and Mr. Jeff Falter (Missouri Department of Elementary and Secondary Education) will give a presentation titled, “What Data Parents Want: Using a Data Dashboard in Missouri.” The presentation will be held from 2:30 PM to 3:20 PM in the Virginia Room on the second level.
- On Thursday, July 14th, Dr. Ellen Mandinach (WestEd), Dr. Jessica Mislevy (AIR), and Mr. Doug Paulson (Minnesota Department of Education) will give a presentation titled, “Measuring New Indicators for STEM Education: State-Level Opportunities and Learning Examples.” The presentation will be held from 10:15 AM to 11:00 AM in the Virginia Room on the second level.