Student survey to evaluate teachers

http://www.nytimes.com/2014/09/04/technology/students-grade-teachers-and-a-start-up-harnesses-the-data.html
A third REALLY important article – this one about how a company is developing sophisticated surveys of students to evaluate their teachers. Not surprisingly, they know exactly who the good and bad teachers are – and can give teachers valuable feedback to improve.

Panorama is trying to assess how well teachers are doing by conducting scientifically valid student questionnaires that collect data about a variety of factors that might affect a teacher’s performance, from how well she conveys the material and whether she encourages interest in a subject to whether a school fosters a sense of belonging for students.

The company, which is run by two 23-year-old Yale grads with a penchant for computers and data crunching, has run surveys in more than 5,000 schools, and it has been adopted by some of the largest school systems in the nation, including the Los Angeles Unified School District and schools in Connecticut.

Panorama has followed the model of Uber and Airbnb in using the unconventional methods of tech start-ups to reinvent industries that have long been seen as tech backwaters. And its increasing popularity suggests that techniques pioneered by the tech industry — including the collection and analysis of large troves of data — may help address problems in American education.

The firm’s techniques have been widely praised by education experts, and it has won prominent supporters in the tech industry. Mark Zuckerberg, the co-founder of Facebook, and Google Ventures, the search company’s investment arm, are among its largest backers.

Some of its innovations sound small, but they have been instrumental in making its surveys more widely accessible than older educational survey methods. For instance, to reduce the costs of its surveys, Panorama created its own scanning system, which allows it to print and collect students’ answers on regular paper, rather than the expensive bubble-scan sheets more commonly used for collecting responses. The firm says its surveys and analytics services are about half the price of older survey methods.

Panorama also hired a team of software engineers and statistics experts to create a kind of analytics “dashboard” for schools — an interactive panel of graphs and charts that presents practical information teachers need in a comprehensible, rather than overwhelming, user interface.

Teachers can dig into how they performed on a question-by-question basis, and they can monitor their performance by subgroup. The survey reports allow teachers to see if they’re connecting better with boys than with girls, or if students who have trouble with English are having more difficulty in a classroom than those who are native English speakers.

Panorama has also invested heavily in improving survey science. It has sponsored research into the best way to ask questions of students to get the most accurate assessment of what’s going on in the classroom, including one recent study by Hunter Gehlbach, a professor of education at Harvard. Last week, again borrowing from the tech industry, the company announced that it would make the survey open-source, meaning schools can use and amend it free.

Seeing how students think about teachers, and how that perception is affecting what they learn, is an unusual development in public education. Today, schools assess the effectiveness of teachers primarily through standardized test scores and observations by administrators, but both measures have been criticized as too narrow, unable to shed light on the complex interplay between teachers and students on a day-to-day basis.

“Education is just starting to figure out what measurement actually means,” said Aaron Feuer, Panorama’s co-founder and chief executive. “Five years ago we thought test scores were the answer to everything. We’re offering a way to focus on the right metrics.”

But in some ways, what the company is doing isn’t new. Student surveys have long been seen as a potential third metric for education. In 2012, the Measures of Effective Teaching Project, a three-year study sponsored by the Bill and Melinda Gates Foundation, found that when combined with test scores and observations, student surveys made for a more reliable and consistent way to measure how teachers were performing.

“A lot of people are unhappy with an overreliance on test scores, but I don’t think it’s an option to drop test scores and go to nothing,” said Thomas J. Kane, a professor of education at Harvard who directed that study. “Student surveys are the most obvious place to add some other measures that aren’t based solely on test scores.”