January 3, 2018
The WestEd Data for Decisions Initiative is pleased to announce the publication of the Data Literacy Assessments Project. These resources are available to educational professionals, professors and students in college and graduate courses, and to researchers.
The Data Literacy Assessments Project at WestEd, in collaboration with Using Data Solutions, has produced four scenario-based exercises that can be used to measure data literacy. The materials can be used as assessments, as supplemental instructional materials to develop data literacy for educational professionals in professional development or college and graduate courses, and for research purposes. The scenarios are designed to depict typical instances when teachers need to use data to make decisions in the course of their practice or personal experiences. They will allow users to demonstrate a variety of data literacy skills as outlined in the construct Data Literacy for Teachers (DLFT) developed by Ellen Mandinach and Edith Gummer.
The resources are available through the DDI website or by typing the URL into your internet browser: https://datafordecisions.wested.org/data-use-in-action/data-literacy-assessments/
August 9, 2017
A new book focuses on data use according to the Data Team™ Procedure, which schools throughout the Netherlands have been implementing in a supervised manner since 2009.
August 9, 2017
Essential to improving teacher quality is ensuring that the right data are available to inform the policy and practice changes needed to continuously improve educator preparation program (EPP) quality, teacher effectiveness, and ultimately student learning. Unfortunately, that data are not uniformly available today.
This paper describes how data sharing among states, EPPs, and K-12 leaders can help ensure quality teaching and learning. The paper:
- Discusses challenges to using data to improve educator preparation
- Presents state policy recommendations
- Profiles policy in action in Massachusetts, Tennessee, and Missouri
WestEd’s Ellen Mandinach served as a member of the working group for this paper, developed collaboratively over one year with a group of national, EPP, and state-level experts.
April 15, 2017
Ellen B. Mandinach contributed a blog post, which can be found below, on data literacy in early education to the National Institute for Early Education Research (NIEER).
Data have been used in education for many years. Good teachers and administrators have been using data to inform their practice and make decisions. Why is data use important? Because it is no longer acceptable for educators to solely use anecdotes and gut feelings to make decisions. Educators need hard evidence. To that end, there has been a growing emphasis for the past 15 years to make education a more evidence-based and data-driven discipline.
Data have ranged from accountability and compliance data to data for continuous improvement at all levels of education, but one issue that has loomed large is the conflation of data literacy with assessment literacy. The two constructs have been confounded for many years (Mandinach & Gummer, 2016a; Mandinach & Kahl, 2014). When educators and the general public think about data, they typically think about test results and student performance. They fail to think about all the other sources of data that help educators to inform their practice. Until fairly recently, there has been no clear definition of data literacy and certainly no analyses of the skills, knowledge, and dispositions that are needed to use data effectively and responsibly (Data Quality Campaign, 2014; Gummer & Mandinach, 2015; Mandinach & Gummer 2016a, 2016b). The work of Mandinach and Gummer, based on several years of research, has attempted to lay out what it means for educators to be data literate.
A foundation of data literacy is the consideration and use of diverse sources of data, not just the limitation to only student performance data. For educators to have a comprehensive understanding of their students, they must look to behavioral, attitudinal, motivational, medical, attendance, home context, and other kinds of data. Even though measures of teacher readiness such as the edTPA (SCALE, 2013) contain an assessment rubric, it makes clear that teacher candidates must be able use contextual information, “assets,” to inform their understanding of student performance.
Diverse sources of data are particularly important in early childhood education where teachers often must look beyond student performance results to understand a child. As Dwyer (2015) notes, some of the data most relevant in early childhood settings other than assessments (formative, summative, and diagnostic) include screening results, informal check-ins, child characteristics and experiences, attendance, health information, family language/education experiences, family conditions for learning, classroom observations, participation, walkthroughs, and staff experience and education. In workshops conducted across Pennsylvania for early childhood educators, Dwyer, Mandinach, Nunnaley, and Saylor (2015) noted several purposes for data use in early learning:
- Improve child outcomes
- Improve teachers’ skills
- Identify gaps in achievement
- Realign resources
- Facilitate parental engagement
- Improve program quality
- Increase access to high quality programs
- Change adult behavior
Dwyer and colleagues (2015) reflected on why data use is important in early learning settings, recognizing that evidence is important. They noted that there needs to be realistic expectations for how data use can inform and improve daily practice. Through effective data use, educators can:
- Reflect on practice
- Check assumptions
- Get others’ views
- Commit to new actions
- Attend to the effects of changes in practice
- Make practice public
But how does this happen? More than eight years ago, the Institute for Education Sciences (IES) commissioned a comprehensive review of the literature that existed at the time, recognizing that data-driven decision making was only then emerging as a hot topic in educational research (Hamilton, Halverson, Jackson, Mandinach, Supovitz, & Wayman, 2009). Some 3,000 research and implementation studies were identified with only a handful meeting the strict criteria for rigorous research laid out by the What Works Clearinghouse. Five recommendations were noted. For there to be effective data use at any level of education, schools and districts must:
- Make data part of an ongoing cycle of instructional improvement
- Teach students to examine their own data and set learning goals
- Establish a clear vision for schoolwide data use
- Provide supports that foster a data-driven culture within the school
- Develop and maintain a districtwide data system (p. iii)
These five recommendations have stood the test of time. The growing research in data use further confirms the recommendations. Much of the work firmly espouses the need for the introduction of data teaming, leadership, and the creation of data cultures within schools and districts. Data systems have morphed from data warehouses to dashboards and apps that provide real-time data for instructional decision making.
Yet despite having much of the infrastructure in place, particularly the billions of dollars spent on technology at the federal, state, and local levels, attention to the human infrastructure remains problematic. Fulfilling all of the five recommendations from the IES practice guide is an important step forward. However, if educators do not know how to use data both effectively and responsibly, the investment in attaining the recommendations will go for naught. Even though the field recognizes the importance of data use, the delivery of consistent and comprehensive professional development is often lacking and falls below other competing priorities.
As Means, Padilla, and Gallagher (2010) noted, professional development for data use must be ongoing, not sporadic. As Mandinach and Gummer (2016a) note, waiting until educators are in practice to acquire data literacy skills is too late. They must begin to acquire such skills at the earliest stages of their professional careers, that is, during pre-service preparation. Because of this growing need, WestEd and its collaborator, Using Data Solutions, is working toward the development of curriculum materials that can be used in teacher preparation programs to teach data literacy. The objective is for teacher preparation programs across the country to begin to integrate the construct, data literacy for teachers (Mandinach & Gummer, 2016a, 2016b) into their curricula. The ultimate objective is to create a teaching corps that knows how to use data.
March 6, 2017
The AERA Data SIG Business Meeting will be help on Sunday, April 30 from 6:15 to 7:45pm at the Grand Hyatt San Antonio, Third Floor Bonham C.
Please come and join the session. The session will include local Texas educators talking about how they use data.
January 3, 2017
An interesting book is now on the best seller’s list about decision making and pertains to all fields, not just education. Michael Lewis of Moneyball fame has written The Undoing Project. It is the story of two psychologists who studied decision making in fields such as medicine and sports. The two psychologists, Daniel Kahneman and Amos Tversky wrote ground-breaking articles on how decisions are made, the assumptions, and fallacies. For example, the hot hand phenomenon in basketball is really a myth. Kahneman was awarded the Nobel Prize for Economic Sciences for this work. A good and interesting read for those involved in decision making.
October 18, 2016
The AERA Data Driven Decision Making in Education Special Interest Group (Data SIG) is pleased to announce a new resource and upcoming webinar.
REL Appalachia developed the Teacher Data Use Survey (TDUS) in collaboration with a panel of expert researchers to help education leaders understand the kinds of supports needed to promote effective data use to improve teaching and learning in their schools. The survey helps identify how teachers use different types of data; teacher attitudes toward data use; how teachers collaborate to use data; and supports to help teachers use data.
The TDUS is authored by Data SIG members – Jeff Wayman, Vincent Cho, Ellen Mandinach, Jonathan Supovitz, and Stephanie Wilkerson. The survey comes in three versions: one for teachers, one for administrators, and one for instructional support staff. Together, these survey versions provide a full picture of teachers’ data use. The survey can also be customized to gather information on specific types of data, such as state, periodic, and local data. Using REL Appalachia TDUS reporting tools, survey results can be presented in a full data report in Word and as a visual dashboard.
Jeff Wayman and Stephanie Wilkerson are doing a webinar series on using the TDUS. The next webinar is on Friday, October 21, 2016. If you are interested in joining, please register.
The survey, implementation manual, and tools are available for public use:
- Teacher Survey.
- Administrator Survey.
- Instructional Support Staff Survey.
- Guide to using the Teacher Data Use Survey.
- Teacher Data Use Survey Report Template.
- Sample dashboard report.
September 15, 2016
A new report to the Bill and Melinda Gates Foundation titled “Blended Learning and Data Use in Three Technology-Infused Charter Schools” examines the affordances of the technologies in three blended learning environments and their impacts for teaching and learning activities. A particular focus of the work was to examine whether the blended learning environments provided access to and more diverse data sources for teachers and students from which to make educational decisions.
Key findings include:
- Blended learning environments provide data to teachers and students that may not be readily available in more traditional classes;
- Blended learning environments provide for any time and any where virtual learning opportunities;
- Teachers were able to address the needs of particular students through various media and diverse learning experiences;
- Students were engaged through flexible and customizable learning activities; and
- The schools exhibited strong leadership, an explicit vision for the use of technology and data, the engagement of students in the teaching and learning process, the enculturation of data use through data teams and data coaches, and the provision of professional learning opportunities.
There is much that can be learned from these three schools about how the alignment and practice of research-based recommendations can create blended or personalized learning environments that have the potential to reach even the most challenged students and help them to succeed.
August 11, 2016
DDI is proud to announce the release of two articles as part of a special issue of the international journal Teaching and Teacher Education. The special issue’s theme is on how countries are improving the capacity of teachers to use data.
The first article is titled “What does it mean for teachers to be data literate: Laying out the skils, knowledge, and dispositions.” The authors, Dr. Ellen Manindach (WestEd) and Dr. Edith Gummer (Ewing Marion Kauffman Foundation), were invited to write an article on a framework for a construct called data literacy for teachers. The article lays out the framework, identifying the specific knowledge, skills, and dispositions teachers need to use data effectively and responsibly. It concludes with a call to schools of education and teacher preparation programs to begin to integrate data literacy into curricula and practical experiences.
The second article is titled “Teacher learning how to use data: A synthesis of the issues and what is known.” The authors, Dr. Ellen Manindach (WestEd) and Dr. Jo Beth Jimerson (Texas Christian University) were invited to synthesize the articles in this special issue on data use. The synthetic piece contextualizes how the articles contribute to the knowledge base of how teachers use data.
July 22, 2016
During a standing-room only presentation, participants of the National Center for Education Statistics (NCES) Summer Conference heard “What Data Parents Want: Using A Data Dashboard in Missouri.” The presenters included Dr. Ellen Mandinach and Dr. Ryan Miskell (WestEd), Dr. Edith Gummer and Mr. Christopher Laubenthal (Ewing Marion Kauffman Foudnation), and Mr. Jeff Falter (Missouri Department of Elementary and Secondary Education). With the Every Student Succeeds Act (ESSA) requiring districts to engage parents and families, NCES participants recognized the importance of communicating education data effectively to parents.
The presentation focused on the types of information parents use and search for as they make educational decisions. This information is vital as states and districts develop dashboards of information for various stakeholders, as the Ewing Marion Kauffman Foundation has done with EdWise. During the presentation, the audience asked several follow-up questions :
- Q: How do parents feel about the applicable processes, documentation, and guidelines associated with schools and districts near them?
- A: These topics frequently come up, but they highlight one of the obstacles of parents’ desire for data–frequently they want information not easily captured or shared.
- Q: All of your 118 interviewed parents came from Missouri; is this study generalizable to other states?
- A: Given the cross section of parents geographically and economically, we feel this study is a fairly representative sample that can be used in different states. Only the very affluent being under represented in the study.
- Q: On the topic of economically diverse parents, did the digital divide present itself in your work?
- A: Absolutely. We met a set of parents in St. Louis who did not have access to computers or the internet. When given the Amazon gift cards one of them asked, “how do I even use this?” People are making assumptions that all students are connected to the Internet and it’s not true.
- Q: How can data address the topics of bullying and peer pressure? How can we share that with parents?
- A: School climate surveys like the Five E’s work in Chicago as well as interviews and testimonials can capture some of this information. We are still challenged by how to best share this information, but we need to continue looking into the matter. Knowing about bullying and peer pressure can save lives.
- Q: Once you have a tool like EdWise, how do you promote its use amongst parents?
- A: Education data just by itself can be elite and so increasing use presents difficult challenges. These challenges, like those we addressed in St. Louis, are something we keep bumping into. The solution is to find ways to meet parents where they are, computers or no, and to develop a culture and an environment that says data use is okay.
- Q: How do parents feel about the applicable processes, documentation, and guidelines associated with schools and districts near them?
July 11, 2016
The Data for Decisions Initiative is proud to announce two upcoming presentations during the 2016 NCES Summer Forum and STATS-DC Data Conference. The conference will be held at The Mayflower Hotel in Washington, DC from Sunday, July 10th through Thursday, July 14th.
- On Tuesday, July 12th, Dr. Ellen Mandinach and Dr. Ryan Miskell (WestEd), Dr. Edith Gummer (Ewing Marion Kauffman Foundation), and Mr. Jeff Falter (Missouri Department of Elementary and Secondary Education) will give a presentation titled, “What Data Parents Want: Using a Data Dashboard in Missouri.” The presentation will be held from 2:30 PM to 3:20 PM in the Virginia Room on the second level.
- On Thursday, July 14th, Dr. Ellen Mandinach (WestEd), Dr. Jessica Mislevy (AIR), and Mr. Doug Paulson (Minnesota Department of Education) will give a presentation titled, “Measuring New Indicators for STEM Education: State-Level Opportunities and Learning Examples.” The presentation will be held from 10:15 AM to 11:00 AM in the Virginia Room on the second level.
April 18, 2016
Description: The Data Quality Campaign (DQC) will host a national summit, “Time to Act: Making Data Work for Students,” on Tuesday, April 26, in Washington, DC. Meeting our nation’s education goals is impossible without empowering those closest to students with the data they need to make decisions. At this summit, DQC will unveil a bold new vision for using data in the service of student learning and call the field to act with corresponding policy recommendations to make that vision a reality. The event will be open to the public with a reception to follow. It will also be webcast for those who can’t make it in person.
Location: The Mayflower Renaissance Hotel, 1127 Connecticut Ave. NW, Washington, DC 20036
Date: Tuesday, April 26
Time: 1:00 PM – 4:30 PM
- Aimee Rogstad Guidera, President and CEO, Data Quality Campaign
- Alyson Klein, Assistant Editor, Education Week
- Bev Perdue, Seventy-Third Governor of North Carolina; Founder and Chair, DigiLEARN: Digital Learning Institute; and Senior Advisor, Whiteboard Advisors
- Elizabeth Rorick, Deputy Executive Director of Government Affairs and Communication, National PTA
- Gail Pletnick, Superintendent, Dysart Unified School District, Arizona
- Jared Polis, US House of Representatives (D-CO)
- Joshua Parker, 2012 Maryland State Teacher of the Year; Instructional Coach, Paul Laurence Dunbar Senior High School, District of Columbia Public Schools
- Kara Anderson, Fairfax County, Virginia parent
- Karl Dean, Former Mayor of Nashville and Davidson County, Tennessee
- Luke Messer, US House of Representatives (R-IN)
- Mary Gwen Wheeler, Executive Director, 55,000 Degrees, Louisville, Kentucky
- Reggie Moore, Founder and Executive Director, Center for Youth Engagement, Milwaukee, Wisconsin
- Rich Crandall, Commissioner of Education, State of Colorado
- Terry Holliday, Former Commissioner of Education, State of Kentucky
- Tomeka Hart, Vice President of Strategic Partnerships and Institutional Advancement, Southern Education Foundation
March 21, 2016
DDI welcomes you to join the inaugural business meeting of an AERA Special Interest Group (SIG) on data-driven decision-making. The SIG will be launched at this year’s American Educational Research Association annual meeting in Washington, DC, Monday, April 11, 6:45-8:15pm (EST) at the Marriott Marquis, Level 4, Independence Salon G.
This SIG will provide interested researchers with an intellectual and collaborative home where theory, research, policy, and practice on data-driven decision-making can be shared with like-minded colleagues. Note: If you are unable to attend the SIG meeting and/or are interested in more information, please feel free to contact Ellen Mandinach at firstname.lastname@example.org or 202.674.9300.
Data-driven decision-making in education has become an increasingly emphasized topic by policymakers. Educators at all levels of the education system — from classrooms, schools, districts, state education agencies, and at the federal level — are expected to use data in effective and responsible ways. Having the right data and using them appropriately can provide credible and informative insights into what is really happening to students, teachers, administrators, and other stakeholders. For example, the decision to implement a new program, curriculum, or intervention should be based on data that support the particular situation.
Research on data use has paralleled this emphasis, but until now has lacked an intellectual home, despite undergirding many of the divisions of AERA. Teachers inform their classroom practices and make instructional decisions based on many sources of data. Education leaders at the building level help to create data cultures in schools. Superintendents and state commissioners must make informed decisions based on hard evidence. Policymakers rely on sound evidence from which to make decisions. It is safe to say that data should permeate all education practice.
As Director of the Data for Decisions Initiative at WestEd, Ellen Mandinach led the creation of this SIG. The Data for Decisions website provides resources to help educators use data for effective decision-making, and now the AERA SIG will further increase understanding and capacity on data for decision-making.
We look forward to seeing you on April 11, in Washington, DC, for a collaborative, engaging kick-off discussion on data-driven decision-making in education. As mentioned above, for more information, especially for those who will be unable to attend the session, please feel free to contact Ellen Mandianch at email@example.com or 202.674.9300.
March 23, 2016
WestEd’s Ellen Mandinach will present on data-driven decision-making at a pre-conference workshop of the Council for Accreditation of Educator Preparation Conference (CAEP),* Wednesday, March 23, Hilton San Diego Bayfront, San Diego, CA.
Participants attending Mandinach’s “Using Data for Programmatic Continuous Improvement and the Preparation of Data Literacy for Educators” workshop will learn about two emerging issues that impact teacher preparation programs:
- The use of data to inform the continuous improvement of programs
- Integration of data literacy as an essential skill set into teacher preparation
Rather than viewing data-driven decision-making as a heavy-handed accountability issue, schools of education can come to see that data can be helpful to facilitate improvement, according to Mandinach.
Mandinach is the Director of the Data for Decisions Initiative at WestEd and a leading expert in data-driven decision-making.
* CAEP advances educator preparation through evidence-based accreditation that assures quality and supports continuous improvement to strengthen P-12 student learning.
January 27, 2016
WestEd and the Michael & Susan Dell Foundation conducted four case studies of teacher preparation programs considered to be ’emerging ‘ in terms of incorporating data literacy into their programs. Dell and WestEd have just released the case studies. The four programs – Western Oregon University, Boston Teacher Residency, Relay Graduate School of Education, and Urban Teachers – help shed light on what constitutes a quality program and how each approaches data literacy.
The case studies can be downloaded together or as individual files: 1) Training Data-Literate Teachers – Insights from Pioneer Programs; 2) Boston Teacher Residency; 3) Relay Graduate School of Education; 4) Urban Teachers; and 5) Western Oregon University.
August 19, 2015
Ellen Mandinach and Andrea Lash of WestEd have written a chapter on assessment and data entitled “Assessment Illuminating Pathways to Learning.” This chapter describes trends in assessment and how different types of data can be used to inform practice. The chapter was released in the third edition of the Handbook of Educational Psychology.
September 17, 2015
On September 17, Ellen Mandinach (WestEd and the DDI) and Edith Gummer (Kauffman Foundation) will conduct an invited workshop at the CAEP Conference. The workshop, entitled Using Data For Programmatic Continuous Improvement and the Preparation of Data Literacy for Educators, will engage deans of schools of education and other administrators about the use of data. It will help the administrators to understand why data literacy is so important and issue around how the construct can be integrated into teacher preparation programs. The workshop also will help the institutions to use data for programmatic improvement.
August 19, 2015
Professional development is an important facilitator of effective data use. WestEd recognizes TERC Using Data as a proven source of high-quality and effective professional learning aligned with its Data For Decisions Initiative (DDI).
According to a recent survey,* “Educators articulated professional learning needs related to data use in six main areas.” Four of these areas are the focus of TERC Using Data’s new online course Using Data For Meaningful Classroom Change:
- asking appropriate questions of data (to guide analysis and use)
- data literacy/interpretation
- fitting data use with day-do-day practice
- sharing information via collaboration.
Details about TERC Using Data’s online course and other professional development offerings can be found at: http://usingdata.terc.edu/workshops/online_courses.cfm
*Jimerson, J.B. & Wayman, J.C. (2015). Professional learning for using data: Examining teacher needs and supports. Teachers College Record. 117(4), 1-36.
March 11, 2015
Ellen Mandinach and Edith Gummer edited a special issue of Teachers College Record on data-driven decision making. The PDFs of the articles are now available on the TCR website to subscribers. The print version will be published in April. Below is a list of the contributors and articles in the issue.
List of Articles
- Data-Driven Decision Making: Components of the Enculturation of Data Use in Education by Ellen Mandinach (WestEd) and Edith Gummer (formerly of WestEd and the National Science Foundation and now the Kauffman Foundation)
- Teachers’ Use of Assessment Data to Inform Instruction: Lessons from the Past and Prospects for the Future by Amanda Datnow (UCSD) and Lea Hubbard (University of San Diego)
- How Can Schools of Education Help to Build Educators’ Capacity to Use Data? A Systematic View of the Issue by Ellen Mandinach (WestEd), Jeremy Friedman (formerly of WestEd and now at the Harlem Children’s Zone), and Edith Gummer (formerly of WestEd and the National Science Foundation and now the Kauffman Foundation)
- Teaching Educators Habits of Mind for Using Data Wisely by Candice Bocala (WestEd) and Kathryn Boudett (Harvard University)
- Building a Conceptual Framework for Data Literacy by Edith Gummer (formerly of WestEd and the National Science Foundation and now the Kauffman Foundation) and Ellen Mandinach (WestEd)
- Professional Learning for Using Data: Examining Teacher Needs and Supports by Jo Beth Jimerson (Texas Christian University) and Jeffery Wayman (Wayman Services)
- Structuring Professional Learning to Develop a Culture of Data Use: Aligning Knowledge from the Field and Research Findings by Nancy Gerzon (WestEd)
- Using Data to Alter Instructional Practice: The Mediating Role of Coaches and Professional Learning Communities by Julie Marsh (USC), Melanie Bertrand (Arizona State University), and Alice Huguet (USC)
- The Development of Capacity for Data Use: The Role of Teacher Networks in an Elementary School by Elizabeth Farley-Ripple (University of Delaware) and Joan Buttram (University of Delaware)
- Factors Influencing the Functioning of Data Teams by Kim Schildkamp (University of Twente) and Cindy Poortman (University of Twente)
- Research and Policy Perspectives on Data-Based Decision Making in Education by Martin Orland (WestEd)
October 30, 2014
If you have been wondering exactly what practices a data literate teacher employs in the classroom, look no further than this post by the DQC. The post includes an infographic on how a teacher uses data, but also a video of a real data literate teacher in action.
April 8, 2014
We will soon be applying to form a Data for Decision Making Special Interest Group (SIG) within AERA. Contact Kellie Kim at firstname.lastname@example.org or fill out this form if you are interested in joining or if you would like more information.
Special Interest Groups (SIGs) provide a forum within AERA for the involvement of individuals drawn together by a common interest in a field of study, teaching, or research when the existing divisional structure may not directly facilitate such activity. The Association provides SIGs program time at the Annual Meeting, publicity, scheduling, staff support, viability, and the prestige of AERA affiliation.