Below is a list of frequently asked questions related to data for decision making. Have a question not addressed on this page? Please contact us at DDI@wested.org.
Components of Data Use
What are data?
Data are pieces of information given meaning by context that ultimately need to be transformed into actionable knowledge if they are to guide decision making. Data can be qualitative or quantitative.
What are the essential components of data literacy?
Work at WestEd has yielded information about the components of data literacy. Categories of data literacy skills include: inquiry processes, habits of mind, general data use skills, understanding of data quality, understanding of data properties, data use procedural skills, the ability to transform data to information, the ability to transform data to implementation, and collaboration.
To use data effectively, teachers should combine knowledge of their content specialty (often called content knowledge) and how to teach it (often called pedagogical content knowledge).
What are the components of a data culture?
A number of data culture components have been noted in the research literature, including: strong leadership; an explicit vision for data use; the technological infrastructure to support data use; human capacity—data literacy among the educators; the provision for data teams; the provision for data coaches; time for common planning; and the creation of a belief structure in which data can be used without fear of retribution.
What are the advantages of education agencies having a vision for data use?
The vision makes explicit why data use is important and how data are to be used in an education agency. This pertains to education at all levels—classrooms, schools, districts, states, and the federal level. Without an explicit vision, educators may not know to use data or why they should be using data.
What is the role of leadership to enhance the use of data for decision making?
The importance of effective leadership around data use is one of the most pervasive findings in the research literature. Leaders set a vision. They provide the necessary resources, structures, and processes to make data use a reality in their schools or districts.
An explicit message that data use is important and expected goes a long way in helping to enculturate data use. Teachers take their cues from building leadership. If they get the message that data use is not important, they will be less likely to use data in their practice. In contrast, if principals make it known that data use is expected and model that behavior, teachers are more likely to follow the model.
What are the benefits of data teams to schools and districts?
Data teams serve as a vehicle within a school or district to take the lead on accessing, examining, and interpreting data, and then implementing a course of action based on the discussions that result from the collaborative inquiry process.
Data teams can be content-based, grade-based, or cross-grade and cross-discipline. The teams can help others in the school use data effectively. The common meeting time provides a venue for educators to come together and discuss education issues to which data can be applied as a means toward a solution.
What are roles of a data coach?
Data coaches lead data teams. They take responsibility for the functioning of the data team, although a distributed model of leadership is always preferred. Data coaches lead the collaborative inquiry process, and help members of the data team and others in the school to use data effectively.
What are the benefits of data systems and technologies to education agencies?
From the smallest to the largest, schools, districts, and state departments of education need technology to support data use. With the proliferation of data, there are simply too much data for educators to store in their heads. The available technologies provide storage, analytic, and reporting functions that are beyond the capacity of humans. Whether a simple spread sheet or a complex integrated data system, the technologies make possible efficient uses of data.
Data systems should be aligned to the needs of the education agency, but they also often serve multiple purposes, including for accountability purposes, as well as administrative and instructional uses.
Using Data for Decisions
Who should use data?
All educators should use data. This includes leaders at all levels, teachers, teacher aides, and even the data clerks in a school. The types of data, however, depend on one’s role in the education process.
Why should educators use data?
Data provide educators with hard evidence of students’ strengths and weaknesses. However, to be effective data users, educators should know what specific data to examine and should look beyond only assessment or student performance data. To obtain a complete picture of a student, teachers should triangulate among diverse sources of data to understand how and why students are performing in specific ways.
Quality teachers have been using data for many years. And now, due to the proliferation of data sources and the increasing availability of technologies to support data use in recent years, education data is now receiving increasing attention from many sources.
What roles should schools of education and professional development providers play in data use?
Both schools of education and professional development providers are essential for helping educators become data literate. Professional development has been the vehicle for training the current cohort of educators for many years. The key is finding the right provider who can align data use training to the needs of the school or district.
There is an increasing recognition not just that current educators need to be data literate, but that data literacy preparation should begin early in educators’ careers. To that end, experts are now looking to schools of education as an essential preparation vehicle.
To date, schools of education generally have not had courses on data-driven decision making nor have they integrated data use concepts into existing curricula in a comprehensive way. More courses for administrators exist than for teachers. Experts in the field are working to ameliorate this deficit so that schools of education and other preservice teacher training providers can provide the needed preparation.
Is data use a passing fad?
Data use is here to stay. Like other professional fields, education is becoming an increasingly evidence-based practice. Surely educators must rely on their experience. But it is no longer acceptable for them to rely solely on anecdotes and gut feelings.
Instead, they must begin to examine data and use those data to inform their practice. To use a medical metaphor, people do not want their doctors making diagnostic decisions based on gut feelings. There is a need for hard evidence. The same rules should apply to education. All educators should be informed by data, supplemented by experience.
What conditions support data use?
Certain conditions should exist in order to make data use a focus and maximize its positive impacts. The supportive conditions include:
- Strong leadership
- An explicit vision for data use
- Technological infrastructure, including technologies that are aligned with education objectives
- Adequate opportunities to build human capacity around data literacy (professional development, in-service workshops, course credit for courses on data use)
- Provisions for data teams, data coaches, and other areas of support
- Common planning time
- Structuring of daily and weekly meeting times
- A belief system that incorporates trust and openness around the use of data and in collaborative inquiry
- An evaluative system that promotes data use
- A value system that promotes communication with parents, students, other educators, and relevant stakeholders
What are some of the barriers to data use?
Primary barriers include:
- Insufficient time, technological infrastructure, tools
- Data illiteracy
- Inadequate training for educators
- Leadership that does not promote data use and a lack of vision for data use
- Lack of data expertise in coaches or teams
- No provision for data teaming
- A belief system that does not include data use or a lack of alignment of data use and an education objective, or provides conflicting messages about data use
- A focus on data use solely for accountability and compliance or a harsh and punitive evaluation system
- Data systems that are not easily accessible or user-friendly including firewalls, or systems that lead to redundant data collection, overburdened data collection, bad data (dead on arrival data), or a lack of data quality
- Fear of recrimination
- Lack of willingness to collaborate
- Lack of evidence that data make a difference and impact practice
- Lack of a supportive data culture
Why do some educators fear data?
There are many reasons for fearing education data, some ill-founded and some more rational. Many educators believe that they can function on anecdotes, gut feelings, and experience without the use of data, in part because they have not seen data pay off in terms of an impact on their students. Educators need to see that using data helps them do their jobs better.
That said, there has been a long history in education of data being used for compliance and accountability, rather than for continuous improvement. Teachers might feel that they get punished for poor student performance data and that the data have not been used constructively to find more effective ways of reaching students.
Continuous improvement is the current mantra, but the emphasis on linking student performance (data) results to teacher evaluations sends mixed messages. Again, some educators fear that these data will be used to evaluate them negatively and unfairly.
Who are “bubble kids” and what are some issues associated with focusing only on them?
“Bubble kids” are those students on either side of the cusp of passing or failing a test. There is sometimes a tendency for educators to focus disproportionate attention on these students because relatively small changes in their test scores can affect the percentage of students in a school, district, or state, who are deemed “proficient” on high-stakes tests.
Education is about providing opportunities to all students, and educators should recognize that students at both the positive and negative extremes of the performance continuum also need attention and support.
What are some different purposes of data for decision making?
Data use ideally pertains to informing practice through continuous improvement. That means that the data are used constructively to help improve instructional or administrative activities.
Teachers use a myriad of data to improve their classroom practices, instruction, and meet students’ learning needs. Administrators at all levels of the education system use diverse data from which to make personnel, financial, curricular, programmatic, and other kinds of decisions. Other uses serve accountability, compliance, and evaluative functions.
The U.S. Department of Education, through the state departments of education, requires districts to supply a huge amount of data that are used for monitoring, accountability, and compliance. Other data are being used for evaluations, such as program evaluations, to determine whether new interventions, curricula, or activities have positive impacts.
One current and contentious use of data is to link student performance and teachers in order to evaluate teachers. Similarly, data are being used to evaluate the effectiveness of schools of education by linking performance data of their recent graduates to program effectiveness.
Types of Data
What is the difference between data literacy and assessment literacy?
Assessment literacy pertains to the capacity to understand assessment (i.e., test) results, including some fundamental principles of measurement, such as reliability, validity, and measurement error.
Data literacy, by contrast, is a broader concept, taking in many sources of data in addition to assessment data such as data on student attitudes, motivation, behaviors, perception, attendance, and demographic characteristics.
What is the proper place of test data in conjunction with other important data sources?
Test data loom large in the education process, regardless of the role of the educator. But it is critical for educators to understand that test data are only one source of information about students.
Other relevant data may need to be brought into the inquiry process in order to provide a comprehensive picture of students’ strengths and weaknesses. Such data may include classroom performance, health, attendance, behavioral, motivation, and attitudinal data. Even data on student transportation may be relevant.
Each inquiry will require different data to be examined. Educators must determine what data are important. And they should remember an essential principle of measurement and data use—the use of multiple measures.
Why is it important to use data from multiple measures?
It is rare for one data point to provide enough information to successfully lead to changes which benefit a student or students. For example, a student’s score on a test is an important piece of data. However, without also examining other data points such as other assessments (formative or summative), attendance data, and behavioral data, it may be too difficult to get valuable information on what kind of instructional changes would most benefit that student.
Furthermore, it is not practical to evaluate a teacher or a student only on one piece of data (like a test). Multiple measures that yield valid and reliable data should be used in order to fairly and accurately assess students and teachers.
Communicating about Data
What types of questions should school boards be asking to create a system-wide data culture?
School boards need to consider the necessary structural components and infrastructure to enculturate data use within the district. Questions might include the following:
- Why are data and evidence important to the functioning of the district?
- How do we enculturate data use in the district?
- What resources are needed?
- What kind of leadership is needed?
- Is there an explicit vision for data use?
- Is the technological infrastructure sufficient to support data use? Is it aligned with the education objectives of the district? What other technologies might be introduced?
- Are the district’s educators data literate? How can we support the improvement of data literacy throughout the district?
- What structural changes might be made within the district to support data use? For example, common planning time, the provision for data coaches, the provision for data teams, changes to the daily and weekly schedules to accommodate common planning time and data teaming, or financial rewards?
- How does the evaluation and reward structure reward or punish data use?
- Is there a belief system around data use?
- What else can we as the school board do to support data use within the district?
How do educators communicate with and involve parents in the data for decision-making process?
The key to communicating with and involving parents is for educators to be armed with data-based evidence of students’ performance. The data provide hard evidence of how the students are doing. Essential to the communication process is to be able to explain to parents what the data mean in lay person’s terms.
Having parents involved in the data-driven decision making processes engages them in the education of their children. It allows them to help take responsibility for their learning. Parents will have a better understanding of students’ strengths and weaknesses, and in discussion with the teachers, what steps in school and at home need to be taken to assist the students.
What are some strategies for communicating the importance of other data besides test data?
Educators should understand that test results alone provide an incomplete picture of their students. Other data can inform them about the whole student. Educators should be encouraged to think out of the box and ask questions. For example:
- What is the attendance pattern like?
- Has the student missed a lot of school?
- Might there be some health issues?
- Are there patterns of behavioral problems?
- What is happening at home with the child?
- Are there motivational or attitudinal issues?
- What other student performance data other than test results (e.g., participation data, formative assessments) might provide insights into the student’s performance?
Data for Decisions Resources
What technological tools exist to support data for decision making?
Many individual applications have been developed as well as suites of tools. Because the technology field is constantly evolving, new applications are appearing on a regular basis.
Some primary tools and applications include: data warehouses, student information systems, assessment systems, instructional management systems, statewide longitudinal data systems, handheld devices, data dashboards, diagnostic and progress monitoring tools, interactive whiteboards, classroom response systems, vodcasting, and visual data analysis tools.
Who are some of the researchers in the area of data-driven decision making?
Many scholars have been working in this field. Key names of those with significant published work in this area are listed below in alphabetical order. Apologies to anyone who we have mistakenly omitted.
- Kathy Boudett
- Cynthia Coburn
- Alan Daly
- Elizabeth Farley-Ripple
- Edith Gummer
- Jessica Heppen
- Jo Beth Jimerson
- Judith Warren Little
- Julia Marsh
- Pamela Moss
- Kim Schildkamp
- Joan Talbert
- Alex Bowers
- Jere Confrey
- Amanda Datnow
- Kara Finnegan
- Rich Halverson
- Joan Herman
- Michael Knapp
- Karen Seashore Louis
- Barbara Means
- Chris Padilla
- Sam Stringfield
- Erica Turner
- Vicky Young
- Vincent Cho
- Michael Copland
- Ann-Marie Faria
- William Firestone
- Laura Hamilton
- Meredith Honig
- Keith Leithwood
- Ellen Mandinach
- Shazia Miller
- Vicky Park
- Jon Supovitz
- Arie van der Ploeg
- Jeff Wayman