Putting Data to Use: Match Making part 1

When I hear conversations about edtech, just about everyone is talking about teacher and student facing tools and systems. Wonderful – they absolutely deserve the best. But there’s a meaty world of sticky problems that’s hidden a layer behind the classroom that ripe for a data scientist with a bent toward education. Layer is the wrong word – it’s more akin to an entire ecosystem of interconnected problems and relationships that all revolve around one word: fit.

Fit with school. Fit with classroom. Fit with principal. Fit with classmates. Fit with teacher prep program. Fit with district and community. Fit with grade level. There are so many “fit” questions that it’s a wonder we’re only beginning to see a few organizations dedicate themselves to this (myEDmatch and Haystack are two most prominent). And behind all the critique Cami Anderson is getting in Newark for One Newark, there’s an intriguing story of a unified application to help “fit” kids to schools.

Let’s look deeper at the questions themselves and how they’re managed now:

  • Kid – School: parents choose where they want to live, and zip code is by far the largest determinant of where kids go to school, even when competitive schools are options.
  • Kid – Classroom: almost entirely random. some manual smoothing by administrators and teachers.
  • Teacher – License Area: wholly teacher choice, but usually have to decide before they begin (unlike doctors or lawyers).
  • Teacher – License Approach: alt-cert v traditional, it’s all the buzz, but what if it weren’t so OR and there were more options like residencies, 5th year programs, etc.?
  • Teacher – Subject: wholly teacher choice, but also have to decide where they begin, and sometimes dependent on what they’ve already learned (e.g. to teach HS math you need a solid background in math. At least in most states…)
  • Teacher – Neighborhood/District: this has changed as markets have changed, but nowadays it’s primarily supply of jobs.
  • Teacher – School: varies by district, but many principals don’t have full control over whom they hire. This is one space where companies like MyEdMatch have brought a new approach although yet to see long term results because they’re new.
  • Teacher – Classroom: combination of where there are vacancies, supply for teachers (e.g. STEM subjects have more openings, as do SPED and ELL license areas), and some teacher choice.
  • Principal – School: varies by district, but at least in NY neither the community nor teachers have any say in the matter. Sometimes the principals themselves don’t either – they’re essentially assigned.

I won’t even pretend I have the solution, but when I look at all the randomness, chance, and early-phase choices that need to be made it can’t help but beget the question: is there a better way to build better experiences for everyone involved?

The answer is unquestionably yes, but what do you see as the most essential levers to push on first?

Data in a distributed organization

Data can be wonderful. That is, if you can get the right data, keep it clean, and coax it to offer powerful insights. If Big Data is the farm factory equivalent hormone-pumped chickens, gaining true insights into your work and telling powerful stories is more akin to the rare kobe beef (with their own personal masseurs).

You might think by the conversations happening on fivethrityeight (and other’s responses to it) that we live in a world where data is lifeblood of orgs small and large, from small start ups to the biggest corporations. And if you weren’t listening closely, you might have missed that in all the data the NSA grabbed from your cell and router that only a tiny tiny fraction of it amounted to anything more than really loud noise (I’m not condoning it – just saying it).

But ask anyone who actually works with data at an org where the data is messy, often focused on relationships, and sometimes locked behind state bureaucracies and all of a sudden you’re talking about a different world. This, my dear reader, is the life of a data wonk in a large non profit.

Interestingly, we’re coming to a point where three things matter most when it comes to data:

  • Our ability to make meaning from it. This ranges from powerful stories to predictive tools that help us do our job better. This isn’t a surprise, but sometimes a lack of focus on this element means spending money, time, and energy on things that matter less.
  • Everyone is an end user. It used to be that just a few people had the “real” tech in their hands, reviewed reports and that made the IT world simpler. Today, everyone is a knowledge worker, tries to be data driven, and quite frequently is the end user of multiple important systems that help them do their work more effectively and efficiently. This means more emphasis on design, experience, satisfaction, and change management.
  • Governance. If your data changes every few years (or more frequently), there’s a lot less you can do with it. That’s not to say if you have a good reason, you shouldn’t change it (we have lots of good reasons), but it does mean that those changes need to be documented, coordinated, and incorporated into the future of how we work. This has recently become a top priority for our org.

I’ll let you in on a secret. I like my job. You might misread this post as a judgment of what’s not going so well in our world with data, but it’s just about the opposite. We’ve turned a corner and are taking ownership of the fact that we’re not as data driven as we want to be because there are parts of our data ecosystem we need to tighten up. And that it’s going to take an organization-wide effort to get there, even as we move to a world where all the parts of our org become more autonomous and independent.

Convergence: data, schools, edtech, and entrepreneurs

Living a digital life means leaving trace elements of data behind. Just as businesses are using the strategies and technologies of “big data” to scour that data for new product ideas, marketing pitches, and custom ads, “edtech” entrepreneurs are are trying to figure out what value they can squeeze out of data flowing from our school doors.

There is some good that is coming of this.

A personalized test that challenges and adapts reinforces the right level of rigor. Access to knowledge anytime and anywhere lowers the barriers to works of literature, art, history and more that were once reserved for the “elite.”

And yet there’s a lot of bleh too.

MOOCs were promising, but are now better understood to only be valuable to a very small segment of highly motivated students who learn best through independent study. Crunching the data from EdX, Udacity, and Coursera will be interesting if they ever open those flood gates. Similarly, flipped classrooms, smart boards, and even power points rarely actually help teachers do their job better. Take your pick of flops (from a pedagogical standpoint): iPads, Chromebooks, e-textbooks, Amplify and so forth.

Certainly some of these technologies will have a role to play in the future, but for now they’ve collectively made an expensive surface scratch.

The dry erase marker is probably the best thing to hit classrooms in the last 15 years.

It’s fair to ask why all of these edtech startups (and not so “start”-ups) are struggling. They’re not struggling for cash, thanks to cozy deals with publishing companies, investors, and other not so disinterested parties. But they are failing nonetheless in delivering something meaningful to the people that could benefit from them most.

So why?

  1. It’s harder to fail fast. There is a simple truth behind this: the only losers in a failed start-up are the people who tried to make more money than they had to start and their deep pocketed investors. The losers in a failed ed-intervention are principals, teachers, students, parents, and communities who have given the public their trust. Risk, therefore, is significant and in short supply.
  2. It’s harder to measure what you’re learning. Lean methods and Lean Accounting are great when it’s easy to get your data. But when you want to look at longitudinal data that’s impacted by so many different factors, it’s safer and more accurate to turn to multi-year, multi-million dollar independent studies than week-over-week classroom data. And yet, that’s what’s expected. This leads to a lot of noisy data and even noisier sales people overconfident with their own data sets.
  3. Outsiders are making a lot of the decisions. While there is a growing cohort of teacher-preneurs, they’re still in the minority. This isn’t anyone’s fault directly, but good solutions require diverse teams that should be built from the ground up. Some of the best already are (more on this another time).

One way to look at the relative lack of innovation in education is to say “we got it mostly right.” Excluding the uninitiated, it’s hard to accept that line of thought when you step inside your average classroom.

Ultimately technology has a critical role to play and I’m all for encouraging innovative thoughts and experiments. What I remain critical of, and caution others on, is the “disruption” talk that emphasizes tech over student-focused solutions. Where I’ve seen technology most effective is in sharing content, videos, and lessons, assisting in feedback, driving adaptive assessments, and helping teachers crunch interim and formative assessment data.

Time to shake things up in the classroom, right?

There are a lot of people outside of education who think that with all that’s “wrong” with education, what we need to do is test lots of new models.

Andrew McAfee added to the clamor earlier this week, urging for change: “It’s past time to start trying experimenting with new approaches, from flipped classrooms and mastery learning to MOOCs, doing solid research to understand what works, then scaling up these better answers.”

Andrew clearly knows something about education – he’s part of the MIT staff and is known world-wide. That said, the contrast between his post and Thomas Friedman’s article about what’s working in Shanghai couldn’t be stronger.

China still has many mediocre schools that need fixing. But the good news is that in just doing the things that American and Chinese educators know work — but doing them systematically and relentlessly — Shanghai has in a decade lifted some of its schools to the global heights in reading, science and math skills.

Flipping classrooms, mastery learning, and MOOCs might be good for some people in some circumstances. But more importantly, the issue is not that we don’t know what works. The issue is doing that effectively and at scale. Sometimes it helps to be in a centrally planned economy. That, above all, may be one of the greatest differences between us, China, South Korea, Finland, and other high performing educators.