How do I identify educators’ greatest needs related to data use?
Last month I responded to a question about using educators’ time effectively when providing professional learning opportunities related to data use. In response to the feedback I provided on last month’s questions (access here), I received another question asking for additional clarification from an educational consultant in North Dakota.
“How do I identify the greatest needs of educators relevant to data utilization to ensure I most effectively use educators’ time when providing data use professional learning?
Could you please share practical examples of tools (surveys, checklists, etc.) and documented processes that have been or could be implemented to identify the greatest needs relevant to data use professional learning?”
This is an excellent pair of questions! When it comes to identifying the greatest needs for professional learning (for any topic), several different methods may be utilized depending on the resources, people and time available prior to the session. These methods include, but are not limited to:
- Focus Groups
- Assessments/Tests of Competency
- Samples of Work
- Existing Reports or Data
There are pros and cons for using each of the methods listed above, which are beyond the scope of the current article (see here for one breakdown of the pros/cons).
It is strongly recommended that before you provide a professional learning opportunity, you identify the needs of the audience and their level of familiarity/competence with the content. However, in some situations, being able to identify the needs and competence level of the audience before the session may not be feasible. In a situation like this, the facilitator must conduct their own, in-person screening assessment using feedback from the live audience. Fortunately, many of the methods identified above can be adapted for use with the audience on a smaller scale.
Regarding the posed question about specific tools or process for identifying the greatest needs, there is one framework that I have found to be particularly useful. It is known as the A+ Inquiry framework which was developed in 2014 by a pair of North Dakota data specialists and an assistant professor in teacher education (Anderson, Brockel, & Kana, 2014). Their model is intended to guide data users through the process of disciplined inquiry as a means to prevent unnecessary data hoarding and to promote effective data-based decision-making. The A+ Inquiry framework represents an eight-stage cycle with a central foundation of awareness throughout the inquiry process (see image below). Each stage of the cycle conveniently starts with the letter “A” representing each step in the process: Absorb; Ask; Accumulate; Access; Analyze; Answer; Announce; Apply. Guiding questions for each stage of the A+ Inquiry framework are included under the image below.
|ABSORB||What do you know about the context? What is not known which should be known?|
|ASK||What question can you ask that, if answered, will give you a better understanding of what you need to know?|
|ACCUMULATE||What data have already been collected? Who and what do existing data represent? What data need to be collected? From whom, where, and how will new data be collected?|
|ACCESS||Where and how do you access accumulated data?|
|ANALYZE||What analysis will you conduct? What software or other tools are required? How will you conduct the analysis?|
|ANSWER||Based on the analysis findings what is the answer to the question? Does the answer respond directly to the question?|
|ANNOUNCE||Which stakeholders should hear the answer? What are the implications for each stakeholder (including you)? What limitations should be shared? How will you communicate with each individual/group?|
|APPLY||Should stakeholders continue doing what they have been doing, stop what they have been doing, or make changes? How will stakeholders take action based on the findings?|
|AWARENESS||Ensures the right context is absorbed, the right questions are asked, the right data are accumulated, accessed, and analyzed, the right answers are derived, the right announcements are communicated, and the right applications are made.|
Oftentimes, when individuals seek out professional learning opportunities, they are in search of learning about a specific piece of knowledge or skills. In these types of situations, the professional learning provider has the luxury of being able to focus the content of their training to respond to these specific needs. However, when providing professional learning pertaining to data use in general, the specific knowledge or skills to be focused on may become less clear given the variability in the level of competence for data use amongst the participants involved. Without conducting a proper needs assessment beforehand, the professional learning provider will need to assess in-person where the majority of participants are lacking in competence related to data use. One way they could do this is by a simple poll of the audience based on some guiding questions. For example, the professional learning provider could formatively assess the audience using these probing questions that address each stage of the A+ Inquiry framework:
ABSORB: Raise your hand if this statement applies to you: “I am uncomfortable using data; I am somewhat comfortable using data; I am very comfortable using data.” Based on the number of hands raised for each response you should get a general sense of the level of the comfortability using data for the entire group. The next probing questions should start to identify which specific skills or knowledge the audience needs assistance with. You may follow-up with a more specific question such as “Are there any general questions/situations that are currently occurring that you might want to explore with data?” to help narrow the topic/scope. If some questions or situations are identified this should help narrow the focus as you address each of the upcoming stages of the framework. The next step would be to probe into whether they are able to ask questions that are capable of being answered using data.
ASK: Can anyone provide me with examples of questions that your school/district has addressed or could address using data? What data was used/would be used to address these questions? Ask for some specific examples to see if the audience can formulate questions that are objective, specific and appropriately measurable using the data sources identified. If the audience has difficulty coming up with answerable questions, then the focus of the session should be on identifying components of effective, answerable questions. If participants are able to provide multiple examples of answerable questions, then probe into whether the data they currently collect is able to address the questions/situations that currently exist.
ACCUMULATE: Who is unaware of the data currently collected in your school/district? Does the data currently collected by your school/district help to address the specific question/situation of interest? If many people are unaware of the data collected by the school/district, this opens up the floor for a discussion about the data that is collected, and possibly action planning for ensuring that this information is communicated out – such as through a data management plan (example). If participants are aware of the data that is collected by the school/district, but the data does not help to address the question/situation of interest, then this may lead into a discussion about how/what data is currently collected, and a discussion of what needs to be changed regarding what/how data is collected. If the appropriate data is collected and people are aware of it being collected, then there may be problems with accessing the data.
ACCESS: Is there anyone who is unaware of how/where to access specific pieces of data collected by the school/district? If many people are unaware of how/where to access data collected by the school/district, this may lead to a larger discussion or training session on the steps needed to access the data of interest. If participants are aware of how/where to access specific data, then it may be an issue of not knowing how to analyze the data that is collected.
ANALYZE: Do you feel comfortable analyzing data collected by the school/district to assist in decision-making? Is the collected data presented in an analyzable manner to address the question of interest? If people are not comfortable analyzing the data that is collected by the school/district, then the session may be focused on identifying the different types of data analysis methods and identifying which methods are appropriate for the types of data that they currently collect. If most participants feel comfortable analyzing the data, but the data is in an unusable format, then the conversation may steer towards identifying changes to be made in the school/district data system, or changes in the type of data that is collected to be more aligned with the appropriate analysis methods. Otherwise, if a majority of participants feel comfortable analyzing the data and the data that is collected is in an analyzable format (and accessible to participants), then the issue may be with interpreting the analysis.
ANSWER: Does anyone have difficulty interpreting the analysis of data collected by your school/district? If many people indicate having difficulty interpreting the analysis of the data, then the conversation may be focused on discussing examples of charts, tables and other data output and how these may address a question of interest. If participants do not indicate having difficulty interpreting the analysis of the data, then there might be an issue with communicating the results out to others.
ANNOUNCE: After data has been collected and analyzed, is there difficulty with communicating the findings from the data? If many participants indicate problems with the communication of the data findings, then the session may focus on the current procedures that exist for communicating data findings within the school/district. It could very well be that there is a breakdown in the communication chain, the information is being communicated to the wrong stakeholders, or there may not be any communication at all. The session would be focused on identifying what problems exist related to the communication of the data, and possibly in the development of a communication plan as part of the data management plan. If participants indicated there is little or no difficulty in communicating the findings of their data, then the issue may be in the application of the findings.
APPLY: After the data findings have been communicated to the appropriate stakeholders or leadership, are the decisions followed-through? If a majority of participants indicated that decisions are not followed-through, then the training session may be focused on identifying the barriers that exist to making the changes (and possibly inviting leadership or stakeholders in charge of making these changes into the discussion). The lack of follow-through with the proposed changes may be due to a variety of different factors that could be in or outside of their control. For instance, if data analyzed on a specific reading intervention pilot program indicated it was successful at getting struggling students back to benchmark, the proposed change may be to increase funding for this program to be expanded district-wide. Although the effectiveness of this program merits increased funding, budget cuts may not allow for more money to be devoted to expanding this program district-wide. School/district revenue sources may be outside of their control; however, they may attempt to explore alternative funding options for expansion of this program. The A+ Inquiry process can be repeated to address each of the barriers that exist.
Although the example above is general and simplistic in nature, it is crucial that each of the steps in the process is operating successfully to ensure that data is being used effectively. Simply put, if you are not asking the right questions, accumulating the right data, accessing the data, analyzing, answering, and announcing the data appropriately you (and your team) will not be in a good position to apply the data to make an effective data-based decision. Click here to download a blank A+ Inquiry Framework wheel and list of guiding questions to fill out on your own.
In addition to using A+ Inquiry framework as a guide for conducting a needs assessment relevant to data use professional learning, the framework can also be used as a guide for other educational data purposes. If you would like to learn more about the A+ Inquiry Framework and how it can be used in education, check out the FREE online course “Develop Your Data Mindset: Essentials of Educational Data Use” which recently launched and is available to all educators across North Dakota. The course is self-paced and includes 13 hour-long modules geared towards learning about data literacy, inquiry methods, and interactive simulations for using data in education. Simulations are targeted on the following topics:
- Universal screening
- Classroom-level goal setting
- Student level goal setting
- Progress monitoring
- Periodic assessment for differentiating instruction
- Classroom-level goal monitoring
- Student level goal monitoring
- Classroom-level goal evaluation
- Student level goal evaluation
Please post feedback or tips/resources related to professional learning regarding data use in the comments section at the end of the article!
If you have any data-related questions for me (or the group of readers as a whole), please complete this short survey, and we will do our best to answer them!
If anything in this article was confusing or unclear please, please, PLEASE let me know by emailing me at Chris.Thompson@k12.nd.us with the subject line “Data Champions” and I will do my best to clarify my unclarity!
 Anderson, N., Brockel, M. & Kana, T. E. (2014). Disciplined inquiry: Using the A+ inquiry framework as a tool for eliminating data hoarding, mindless decision-making, and other barriers to effective ESA programming. Perspectives: A Journal of Research and Opinion About Educational Service Agencies, 20(3).