Data Review – A Little Less Talk, a Lot More Action

Data.  It’s a four letter word.  Especially in education.

It has been argued by many to play a pivotal role in increasing student growth.  The four key PLC questions from DuFour are centered around it.  Even Danielson includes it as part of Domains, 1, 3 & 4 of the teacher evaluation rubric.

And yet, when many educators hear that it’s time for a data review meeting they either cringe, cry, or circle up their friends to plan out who’s bringing what treats to get through the agonizing process.

So what’s wrong with data?  Specifically, what’s wrong with data review and why does it get such a bad rap?  More importantly, what can we do about it?

Problem #1 We Don’t Engage All Stakeholders

When I was a teacher I remember being invited to countless data review meetings where the reading specialists would project graphs of student growth (or lack of) using something called AIMSWEB.  We would painstakingly go through each student with the specialists all sharing their insights. Periodically I would be asked for feedback on my students, but in the end the decision would be made to change some sort of intervention that I was not involved in by the other “experts” in the room. 

I sincerely hated those meetings.  I became an expert head nodder at those meetings.  Most of the time I was dreaming about what I would eat for, let’s be honest, any meal, or thinking about which member I would take from my favorite boy bands to form the greatest band of all time.  (This is way trickier than you think.  You need at least one bad boy which means you can’t just take ALL the cute ones.)

From start to finish, each stakeholder, from educator to specialist needs to be both an active creator and participant in the process and beyond.  The reason why data review has gotten such a bad name is that many educators, like myself, have experienced it as something being done to them as opposed to the collaborative process that it should be.  No one ever explained to me WHY we were having these meetings or what the expected outcomes were.  No one ever asked me what data I thought would be meaningful to look at or even to bring data with me.  I was simply told I had a substitute and was to show up to these monthly data meetings.  They were supposed to be these all important events, but I usually left them wanting two hours of my life back and needing a coffee.

Problem #2 We Never Get Anywhere

When I was an instructional coach one of the big parts of my role was Data Coach.  I ran data review meetings, helped teachers to look at formative classroom data, and facilitated discussions in PLC’s.  DATA was a four-letter word regularly used in my vocabulary.  And (gulp), I liked it.

“Here’s What, So What, Now What” was my jam.  We used it to evaluate everything from exit slips to Fountas & Pinnell assessments to reading responses and everything in between.  The feedback that I got was positive from the teachers I worked with.  It was a well-organized way to present the data (Here’s What), talk about causes (So What) and then come up with a plan of what we were going to do about it (Now What).

Unfortunately, as I have been reflecting upon this protocol in preparation for some data review with my current staff I have come to realize that there are some serious flaws in the way we used this protocol. 

Wait, what?  Did I just say the mack daddy of data protocols is all wrong?

Yep.  I did.

Here’s why.  I’m a control freak.  And I broke it.

Yep.  I’m a control freak.  Ok, recovering control freak.  Back in the hard core CF days I thought we needed a list of guiding questions, as well as categories of students, to look at when talking about data.  I needed a predictable structure that would get us from point A to point B to point C each time.  I mean how would teachers know what to talk about if I (God of Data) didn’t guide them step by step each time through the process? 

Recovering CF me realizes how incredibly idiotic this was for two reasons:  

1.  See Problem #1
2. There are so many questions and levels of students to talk about that we rarely made it to the Now What (THE MOST IMPORTANT PART) in a 45 minute PLC time and would have to continue to the next meeting

Although we had some GREAT conversations about students using this protocol I have to imagine that my staff walked away feeling frustrated when we had to wait a week to get to the action plan or meet at another time after school to finish it.  Without action, data review is just a pretty little template with some glorious notes about our thoughts, but no real impact on student learning.

Here’s What, So What, Now What is still a great protocol, it just needs to be simplified.  Don’t try to look at all the subgroups at one time.  Select the group that is most meaningful for your team to talk about and only use it for that group.  Select a few questions to focus on during your conversation.  Doing “All the Things” is not productive when discussing data.

Problem #3 Meaningless Data

Problem number three could be argued to be a large part of number one.  

Many times we are asked to analyze data that is not very meaningful because the data has gone well past its expiration date.  Standardized tests like PARCC or IAR or whatever it’s being called this year are thought to be important data to analyze because often our school success is judged by this benchmark.  However, when it comes six months into the following year it’s hard to find any correlation between the results and current teaching practices.  The kids have grown.  Our teaching has changed.  Nothing is the same.  It’s hard to have buy-in to discuss something that is related to something so far in the past.  

Another way that data can be meaningless is when it doesn’t match our strategic outcomes for students.  If we are saying that as a school we are trying to foster collaboration, creativity, communication and any of the other six C’s, then it is difficult to make the argument that we should spend hours analyzing a multiple choice test or any other form of assessment that doesn’t show evidence of those indicators.   If data is going to be meaningful for analysis it has to match with our intended outcomes.

Other times it really does go back to Problem #1.  If we don’t explain the why or ask for the feedback of everyone involved in selecting data to analyze then there ultimately will be little impact on students. We have to move beyond the idea that data review has to involve fancy charts, graphs or percentages.  Coming from a business background, I love me a big fancy spreadsheet with a pie chart or bar graph involved, but if we never move beyond simply looking at numbers data review is going to continue to lack meaning for many.

My So What

In order to combat the full fledged groans that usually commence at the mention of the word data we have to simplify the process.  Let’s stop making it this mystical thing that requires elaborate templates and official numbers. The whole point of looking at data is to cause growth in students.  The best way to do this is to select meaningful evidence that will help us to make instructional decisions that we can act on. 

That being said, I don’t have all the answers (yet), but here’s where I’m currently at:

1.  Select a facilitator.  Have this person engage all stakeholders prior to the meeting about an area they see a need to talk about. 
2. Decide on some evidence (data) that would demonstrate this need. (exit slip, writing sample, conferring notes etc.) 
3.  Decide on how you will be assessing the data prior to the meeting as a team and come to the meeting with it already “graded.” (Note:  This is not extra work.  This is simply assessing something you would already authentically be doing or have done.)
4. At the meeting answer the following questions:
   1. What does this evidence tell us about our students?  What did they do well? What did they struggle with? 
   2. How could we build on their strengths to create success? 
   3. What action steps do we need to take so that each student will grow?
   4. What questions do we still have?
   5. Create a plan of action with a follow-up date included.

Albert Einstein allegedly once said, “Everything should be made as simple as possible, but not simpler.” To me this means that we need to simplify the process, but not the thinking involved in looking at student data.  It is my hope that through several iterations and feedback from my team we are able to further refine these processes and get to the heart of what will move all students forward.

Christina

 

 

A Good Question Can Solve Any Problem

Audio version of this post.

Judge a Man By His Questions Rather Than By His Answers

Seven years ago I would have seriously struggled with this quote.  Coming out of the classroom into the role of an instructional coach, I thought my sole purpose was to solve any problem that was brought my way.  In some ways this perception was accurate. When I first started at Mill St., a big part of my focus in building relationships was finding out what my staff needed and being the solution to whatever that was.  This built both trust and credibility, but as I would later learn, was not the most impactful way to be a coach.

I’ll never forget my first 1:1 meeting with Melissa Hampton, Director of Professional Learning.  She had come to visit my building to find out how coaching was going and what she could do to support me.  I was so excited to share with her all of the work I had been doing with staff members on everything from learning targets to reading and math workshop.  She too shared my enthusiasm, but then asked me a simple question that has impacted my practice to this day, “What do you think would happen if you were no longer here at Mill?  Would the instructional practices continue?”  I paused for a minute, thought about it, and honestly wasn’t sure.  I knew I had done a good job of modeling, providing resources and coming up with ideas, but what I realized in my earnestness for solving problems was that I was leaving teachers out of the most important part of the process, the thinking behind the planning and creating.

Melissa continued the conversation with me, letting me share my thoughts, pausing (for sometimes what felt like an eternity), and asking questions that moved my thinking forward.  She never once told me, “this is what I would do” unless I asked explicitly for her opinion.  By the end of the meeting I had many ideas and plans for how I would move forward with my coaching that I was genuinely excited about.  She was clearly a wealth of knowledge, but never made me feel like I wasn’t capable.  In short, she had empowered me to reach my own goals.

Long after Melissa left I continued to think about what coaching moves she used and how I could get better at building the capacity of those whom I served.  I realized that what impacted me most were the questions that she asked.  Open-ended and reflective, she genuinely just seemed curious about me and my thoughts.   As a result, I started working daily to improve in my ability to ask meaningful and reflective questions.  The first time I heard someone say, “you ask really good questions” was truly one of the best compliments I had ever received.  By the end of my coaching tenure, I left feeling like the work would last, not because of my expertise, but because through thoughtful questioning & reflection, I helped build upon the talent that already existed within my staff.

Asking Meaningful Questions

A few months into the new school year, I have started thinking again about the art of asking questions, a skill that is often touted as important, but seldom developed as the complex process that it really is.  My ability to ask good questions has really come down to four things:

• Committed Listening
• Curiosity
• Believing the answer is in the room
• Practice, Practice, Practice

Committed Listening

First, the obvious.  It is nearly impossible to ask a good question if you haven’t been listening.  At my first coaching training with the fabulous Cindy Harrison back in 2012, I learned about something called, “Committed Listening.” (Not to be confused with Active Listening which I will forever associate with this episode from Everybody Loves Raymond) I’m not sure where it came from, but basically you are listening without:

1. Thinking about what you will say back
2. Finding fault
3. Piggybacking
4. Formulating a piece of advice
5. Giving a solution

My experience is that people have a tendency towards one of these when they are in a conversation.  For example, my mother is a notorious “Piggybacker” or “One Upper,” if you are an SNL fan.  Every time I share a story with her she has to tell me something related that happened to her.  These connections usually starting with the phrase, “If you think that’s bad, listen to what happened to me,” or “I know exactly how you feel” followed by a lengthy story.  The person you’re talking to assumes that it’s making you feel better by commiserating, but often it just makes the other person feel like you weren’t listening. (And you weren’t right? You were formulating your own story as the person was talking.) You probably could have guessed this from my prior story, but numbers 4 and 5 have been my personal struggle.

So how do you move forward if you know you struggle with one or more of these?  The first step is recognizing that you have a problem.  (Thanks AA)  Now that you are aware that you have a problem, you can recognize when your habit is creeping up and direct your mind back to the person talking.  A strategy that helped me as a coach with this was taking notes while the person was talking.  It was impossible for my mind to both type on the computer and also formulate a piece of advice or solution.  Whatever strategy you use it is going to take practice.  It’s okay to have setbacks, but keep reminding yourself that in the long run being a better listener leads to better questions which leads to increased ownership of the solution.

Curiosity 

In his book Curious: The Desire to Know and Why Your Future Depends on It, Ian Leslie makes the argument that curiosity creates more insight and innovation, but thanks to the information rich world we now live in, people are losing their ability to ask good questions.  Further, he posits if we more regularly flex our curiosity muscles we become more empathetic towards others resulting in a positive effect on society as a whole.

When I stopped thinking about how I would respond or formulating solutions while the other person was talking, it opened up my mind to be genuinely curious about what the other person was saying.  I started picking up on little details that in the past I would have missed due to the internal dialogue happening in my head.  My ability to ask better questions improved immensely because I was fully present in the conversation.

A great example of this is a conversation I had with a teacher about a boy in her classroom who was frequently misbehaving.  From first glance it appeared that he simply didn’t care about school.  He didn’t want to do the work, and would run out of the room often with other staff members chasing him down. This happened most frequently at the start of each day.  As the teacher told me stories about how he would spend the day disrupting the class I felt her frustration.  It was curious because he was only in first grade and appeared to already despise school.

Instead of coming up with some sort of behavior modification plan, I started asking her what she knew about him.  What were his strengths?  What was important to him?  Was there anything missing that we might not be considering?  The more we talked I could see the wheels turning in this teacher’s mind.  She started sharing that this little boy was very close to his mom.  He had a difficult home life and coming to school was a sense of comfort, but also anxiety because he was missing feeling that connection.  What she decided to do was giving him extra positive attention and share things with him about herself that he could connect to like superheroes and her family.   He loved spelling so that became his first activity of the day.  The change in this little boy was huge.  I’m not going to say he became a perfect angel, but the disruptive behaviors dramatically lessened and he started demonstrating a desire for leadership in her classroom.

The Answer is Always in the Room

“The answer is always in the room” is a classic coaching tenet.  What it really means is that you completely believe in the strengths and talents of everyone around you.  Because of this, you know that both you and the person you are working with are fully capable of coming up with a solution to any problem or puzzle that arises.

The implication for this belief is that it increases both the rigor and tenacity of the questions being asked in the conversation.  More open-ended questions are offered, because with enough reflection, I know we will be able to come up with a solution to whatever we are pondering together.  Additionally, if we reach an impasse, I will continue to probe for more information or ask questions in a different way until we come up with a plan that we both believe in.

In retrospect, I wish I would have approached my students with this philosophy when I was a classroom teacher, particularly my struggling students. Instead of telling them what to do when I saw them get stuck, I could have thought about a question that would have gotten them to find the solution themselves.  I thought by rescuing them I was doing my job, but I now realize what I was really doing was enabling them to always depend on me.

It’s not always easy to do in the moment though.  When I’ve worked with teachers on this issue, we first examine the learning target and talk about what we want students to get out of the lesson.  Next we brainstorm what students might struggle with.  Finally, we create a list of 2-4 questions that the teacher can ask to help a student to get themselves unstuck.  Sometimes, as we went through this reflective process we even ended up devising better ways to teach the students during the lesson, which prevented the struggle in the first place. 🙂

Practice, Practice, Practice

The biggest piece of advice I can give about asking questions is that you simply need to practice.  When I was a coach we met every Friday and got to practice skills related to coaching in a “critical friends” group.  Besides practicing daily with the conversations I was having with teachers, this was the best because I actually had someone listening and giving me feedback on my process.  It helped me to create goals for myself and refine my practice.

Another helpful way to improve is to videotape yourself teaching or talking with a staff member.  Once you’ve gotten past the excruciating experience of watching yourself on video (so hard to not nitpick every little thing), it can be a very powerful way to look for patterns in the types of questions you are asking and the effect you see happening on the person/group you are speaking with.

Here are some things to ask yourself as you reflect:

• (Pattern Seeking) Am I asking more open-ended or closed questions?  What is the ratio of questions that probe for more information vs. move the learner forward for action vs. result in reflection?
• (Positive Presuppositions)  How am I phrasing my questions in a way that show the other person I believe in their capacity and ability?  (For example:  As you examine the student work, what are some of your findings? vs. Did you look at the student work? Click here for more examples.)
• (Type of Questions & Impact)  What questions am I asking that seem to have the greatest impact on the other person’s ability to:
  • Create their own solutions
  • Move forward to action
  • Reflect
  • Delve deeper
  • Come back to the target if the conversation gets off task

A Change in Perspective

The other day I was having a conversation with the instructional coach in one of my buildings about some ideas that she had.   As we were finishing our talk she said, “Christina you ask really good questions.  People should call you the Questionnaire Extroardinnaire.”  I smiled, thanked her for the compliment and felt genuine gratitude for what seemed like my own “Melissa Hampton” moment.

For most of my life I believed my greatest asset was my intellect and ability to create solutions.  Although these are both strengths to be valued, I would much rather be known as someone who asks great questions and builds the strengths and capacity of others.  If we are going to create learning environments where impactful and innovative practices flourish, as opposed to new fads that die with the next leader, we need to make sure that the ability to ask good questions is cultivated in every learner in the school.