Penn’s Quattrone Center for the Fair Administration of Justice is pioneering a systemic, data-driven approach to criminal justice reform. Its executive director, John Hollway, started with the idea that the law should function more like science—less argument, more truth seeking.
By Julia M. Klein | Illustration by Melinda Beck
Even after graduating from law school, John F. Hollway C’92 LPS’18 recalls, “If they had said on the radio that they arrested someone, I would think, ‘Oh, they got the guy.’” Then he encountered the saga of John Thompson, a Black man convicted, in two separate cases, of carjacking and murder, and sent to Louisiana’s death row.
Two of Hollway’s former colleagues at the Philadelphia law firm of Morgan, Lewis & Bockius spent nearly two decades, pro bono, on the capital case, working all the legal angles. Years into the laborious appeals process, with hope for a reprieve dimming, they uncovered evidence suggesting that their client hadn’t simply been denied a fair trial. He was actually innocent of both crimes.
“That really grabbed me,” says Hollway, executive director of the Quattrone Center for the Fair Administration of Justice and associate dean at Penn’s Carey Law School. “It redefined my view of the criminal justice system. In 1988, 12 independent people had heard all the evidence and come to the conclusion that this guy was absolutely guilty. And then it turned out they hadn’t heard all the evidence, and 12 other people, when hearing the evidence told in a different way, were absolutely certain he was innocent. It shook my foundations of what I thought the law should be doing.”
He began researching a book on the case in 2004. Soon afterward, he became vice president for business development at a South San Francisco biotech startup, Achaogen Inc., tackling the problem of bacterial resistance to antibiotics. “It was 10 PhDs and me, basically,” Hollway says, “and what was clear to me was that the way that scientists resolved disputes was very different from the way I had learned to resolve disputes as a lawyer.”
As he wrote Killing Time: An 18-Year-Odyssey from Death Row to Freedom (Skyhorse Publishing), Hollway kept thinking about how much happier Thompson’s fate might have been “if we’d had a collaborative process of truth-finding instead of an adversarial process.” While shopping the book (co-authored with Ronald M. Gauthier) to publishers, he got another jolt: “I kept getting told, ‘Oh yeah, this is a great story, but we get way too many of these—two or three a month.’”
In 2010, the year the book was published, Hollway joined the advisory board of the Northern California Innocence Project, devoted to overturning wrongful convictions. But maybe, he was starting to think, the best solution wasn’t litigating individual cases. That process was riddled with uncertainties, consumed years, and cost millions in lawyers’ fees (or, in pro bono cases, unbilled time). What if it was possible to step back and examine the whole system, to see how such devastating mistakes could have occurred in the first place? And, by diagnosing the problems, fix them?
Launched in 2013 with a $15 million gift from the Frank [W’77] and Denise [SAMP ’78] Quattrone Foundation and Hollway at its helm, the Quattrone Center for the Fair Administration of Justice is pioneeringa systemic, data-driven approach to criminal justice reform. Its methods derive, in part, from W. Edwards Deming’s notion of a feedback loop of “continuous quality improvement.” Deming, an engineer and physicist turned management guru, introduced his techniques in Japan in the 1950s. They were adopted in the 1980s by the American automotive and aviation industries and, more recently, by the US health care sector.
Hollway likes to point out that the initials “QC,” for Quattrone Center, also stand for “quality control.” The center’s tagline, he says, is “a systems approach to preventing errors in criminal justice.” Quattrone melds policy and practice, using one to inform the other. The center’s research on issues such as cash bail, pretrial detention, and the controversial police practice of “stop and frisk” have fueled litigation, legislation, and administrative reforms nationwide.
“Data can be connective,” Hollway says, bridging professional and ideological divides and fostering consensus.
Former Philadelphia Police Commissioner Charles H. Ramsey, a Penn Law Distinguished Policy Fellow who cotaught the seminar “Policing in the 21st Century” with Hollway last spring, echoes that idea. Quattrone “can be a convener, they can bring a diverse group of people together to have real dialogue—not just shouting at each other,” he says.
Pamela R. Metzger, the inaugural director of the Deason Family Criminal Justice Reform Center at Southern Methodist University’s Dedman School of Law, sees Quattrone as a model. Hollway “established the first major criminal justice reform center that had both national ambition and national reach,” says Metzger. “And he did that by bringing people together around issues that they had in common. John went out and got people to the table.”
In recent months, the Minneapolis police killing of George Floyd, mass protests, and growing demands for a racial reckoning have transformed the social and cultural landscape. The movement has boosted Quattrone’s profile and demand for its services. Given its longstanding concerns with racial bias in criminal justice, “we’re precisely placed for this moment in time,” Hollway says.
As a result, the center has stepped up its activities, cosponsoring webinars on police reform and applying its expertise to an increasing number of projects nationally, in Philadelphia, and at Penn itself. At the request of President Amy Gutmann, Quattrone is conducting the University’s Public Safety Review and Outreach Initiative, designed to evaluate Penn’s success in creating “a physically and emotionally safe environment on campus and in the surrounding community,” while promoting “anti-racism, racial equality, and justice.”
The center’s approach hasn’t changed, Hollway says, even if his vocabulary has shifted to reflect a new sense of urgency. Introducing a July 8 webinar on structural impediments to police reform, Hollway talked about Penn Law’s commitment “to support all communities in their anti-racism advocacy.” The panel featured Ramsey, two other former city officials now affiliated with the law school, and a policy analyst for the Philadelphia Police Advisory Commission. Focusing on such nuts-and-bolts obstacles as Philadelphia’s grievance arbitration process and the powerful Fraternal Order of Police union, it was a view from the trenches—in Ramsey’s case, from the streets.
An earlier, June 24 panel, “Beyond Reform: Re-envisioning the Role of Police,” showcased both more radical politics and loftier aims. Its participants were five like-minded “abolitionist” academics, all highly skeptical of policing and worried that incremental reforms—the sort of fixes that Quattrone has often recommended—could backfire by legitimizing or reinforcing the system.
“To be perfectly honest, my view is not the same as the way the Quattrone Center’s mission is framed,” says Dorothy E. Roberts, the George A. Weiss University Professor of Law and Sociology, the Raymond Pace and Sadie Tanner Mossell Alexander Professor of Civil Rights, and professor of Africana Studies [“Dangerous Ideas,” Jul|Aug 2016]. Roberts, a Quattrone affiliate, a Penn Integrates Knowledge Professor, and director of Penn’s Program on Race, Science, and Society, moderated and assembled the anti-policing panel. “It is so rare that our more radical viewpoint gets aired,” she says.
Roberts, who is among those advising Penn’s new public safety initiative, favors the abolition of not just police, but prisons, which she views as a manifestation of this country’s legacy of slavery, segregation, and racism. “I believe that the US criminal justice system is designed in a way that produces injustice. So the problem isn’t a malfunction that has to be corrected,” she says. “The problem is the system itself.”
Most of Quattrone’s interdisciplinary group of affiliated faculty and fellows—with degrees in medicine, nursing, law, economics, sociology, criminology, political science, and psychology—share a more reformist bent.
The center’s hands-on involvement generally takes the form of after-the-fact probes of undesirable criminal justice outcomes, variously known as “sentinel event reviews,” “root cause analyses” or “just culture event reviews,” which Hollway leads. (“They’re all basically the same thing—it’s just branding,” he says.)He is also an expert on setting up “conviction integrity units” that reexamine questionable cases with an eye toward possible exoneration.
In Austin, Texas, Quattrone helped to investigate deficiencies at the Austin Police Department’s DNA lab, closed in 2016. The release of its report was pending as of late July. In Tucson, Arizona, it is reviewing two incidents in which Latinx individuals died in police custody. In 2015, the center worked with the Montgomery County District Attorney’s Office, in the Philadelphia suburbs, to analyze how a rape case involving a prominent political figure and a misread lab report went awry.
Kevin A. Steele, the current Montgomery County District Attorney, says that the review identified “a lack of independent oversight” as “an underlying root cause” of the debacle. Thanks to Quattrone, the county commissioners funded a new position, a “deputy for professional standards” who reviews “hundreds of criminal investigations and case dispositions each year in an effort to minimize mistakes,” Steele says.
Quattrone also coordinates the ongoing Philadelphia Event Review Team, described in an April 2019 report as a “first-of-its-kind initiative in the United States, dedicated to a culture of learning from error.” The Philadelphia District Attorney’s Office, the Philadelphia Police Department, the First Judicial District Courts of Pennsylvania, and the Defender Association of Philadelphia are all participants. The team investigated the case of George Cortez, convicted of murder in 2012 on the basis of inaccurate eyewitness testimony; sentenced to life without parole; exonerated in 2016; and shot to death on the streets of North Philadelphia two months after his release. Among other recommendations, the report urged caution in charging defendants whenever eyewitness testimony remained uncorroborated.
In a June 23 webinar sponsored by the National Association for Civilian Oversight of Law Enforcement (NACOLE), Hollway said that “at this moment in time” the review process can help soothe anger and “improve the relationship between police departments and their communities.
“I don’t think anybody wakes up in the morning thinking, ‘Today’s the day that I’m going to be involved in an excessive use of force and lose my job and cost somebody his life,’” Hollway says. “The question we ask at the Quattrone Center then is, ‘If that’s the case, why do these tragedies keep happening? And what can we do to prevent the next tragedy?’ We all want to figure out how we’re going to move forward from here in a way that can begin to reunite and heal some of the wounds in our society.”
Paul Heaton, a University of Chicago-trained economist who is Quattrone’s academic director and a senior fellow, describes Hollway as “enormously capable at handling and adapting to new and unfamiliar situations.” He adds: “If there was some game where you’re just going to drop somewhere in the world and you have to survive and thrive and pick a teammate, John would be the guy.”
“He’s a little bit of a polymath or a Renaissance man,” SMU’s Metzger says of Hollway. “He’s clearly very skilled at developing partnerships, both financial and intellectual. He’s really good at institution building—which is, I have discovered, much harder to do than it appears.”
Hollway grew up mostly in Maryland, outside Annapolis. His father was a US Air Force officer, his mother a homemaker and, later, a nurse. His younger brother is a helicopter paramedic. At Penn, Hollway majored in diplomatic history and minored in East Asian studies, polishing his fluency in Japanese during a summer abroad. He attended George Washington University Law School, where he excelled at moot court competitions.
Back in Philadelphia, Hollway signed on as an associate with Morgan Lewis, where he met his wife, Jami McKeon, now the firm’s chair. (Morgan Lewis describes itself as “the largest law firm in the world led by a woman.”) But Hollway, though admiring of the firm’s pro bono commitments and challenged by some of his assignments, wasn’t sure he wanted a corporate law career. He cites the aphorism that working at a law firm resembles “a 10-year pie eating contest,” for which “the reward is more pie.”
He soon had other options. After two years, “I got hired away by a client who basically said, ‘You managed this litigation well for me, and I saw you think strategically. Why don’t you come be my chief of staff and you can help me build these hospital software products?’” The offer, from Shared Medical Systems, included stock and the chance to learn new skills. His title was general manager for wireless solutions.
Just before SMS was purchased by Siemens, in 2000,Hollway left to become vice president and chief privacy officer for Acurian Inc., which recruits patients for clinical trials. In 2004, he accompanied his wife to San Francisco, where she took charge of Morgan Lewis’s California operations. (The couple has two daughters and two sons, including Payton Hollway C’22.)
That’s when Hollway landed at Achaogen, whose name, he says, was meant to suggest “taking the chaos out of genetics.” Working with scientists was revelatory. “As a lawyer, you’re resolving disputes by controlling data, controlling the debate, and arguing,” Hollway says. By contrast, the scientists “would put all the data together on a table and ask each other questions about what the data meant, and then they would reach a consensus about what they agreed on and what they didn’t agree on. And then they would figure out how to answer the questions about what they didn’t agree on.” The process, he says, “was based on ‘What’s the truth?’ and not, ‘What can I prove?’”
In 2011, Hollway gave a talk at Penn Law about Killing Time, which had become a stirring tale of dedicated lawyers and a man both wrongfully incarcerated and redeemed by prison. As he recalls the event, “Al Russo, on the development team of the law school, grabbed me as I came off the podium, and said, ‘I have some interested alums who want to know why we don’t have an Innocence Project.’”
Hollway, who had just launched his own criminal justice reform consulting firm, told Russo that he had two answers to the question. The first was, “The Pennsylvania Innocence Project already exists, and Penn students already participate—it just happens to be housed at Temple. And you don’t need a second one.”
The other answer was really a sales pitch. “It costs millions of dollars to litigate these cases,” Hollway told Russo. “For the amount of money it would take to litigate one of these cases properly, you could create a center that would aggregate data and use data to advocate for policy reforms in ways that would bring people together. Because suing people is divisive. But data can be connective.”
That, says Hollway, “ended up being the germ of the idea—something that both the law school and the donors could get behind.”
Hollway and the Quattrones already knew one another from their involvement with the Northern California Innocence Project. A technology investment banker, Frank Quattrone had had his own unpleasant run-in with the criminal justice system several years earlier, fighting obstruction of justice charges against him that were eventually dropped. Hollway says his thoughts “resonated with the Quattrones’ vision of a center they could support.”
Hollway began with “all this expertise right there in the [law school] building.” Aided by Penn’s interdisciplinary culture and compact urban campus, he reached out to the medical and nursing schools and the University’s social science departments to assemble a roster of affiliated faculty.
Steven E. Raper L’12, vice chair for quality and risk management and associate professor of Surgery at the Perelman School of Medicine, played a key role in early discussions of how post-mortem analyses of medical errors might translate to a criminal justice setting.
Quattrone’s advisory board boasts the expected array of legal luminaries, but also, since 2017, Grammy Award-winning singer-songwriter and reform advocate John Legend C’99. The Research Fellows Program attracts young talent, seeding its ideas nationally. Sandra G. Mayson, one of the inaugural Research Fellows in 2015-17, says her Quattrone stint has “shaped my career ever since,” exposing her to interdisciplinary criminal justice scholarship and helping her forge new academic relationships. (Based at the University of Georgia School of Law, she is returning to Penn Law for the 2020-21 academic year as a visiting assistant professor.)
Since 2016, Quattrone also has had an Exoneree Fellow Program. Its 2018 fellow, Keith Harward, was convicted of murder, robbery, burglary, and rape and served 33 years before being cleared in 2016 by DNA evidence. Saved by science, he participated in a discussion of what Hollway called “science that’s gone off the rails”—in Harward’s case, bite mark evidence. The panel was titled, “Once Bitten, Twice Shy.”
In its seven years of existence, Quattrone has spawned an impressive body of research. In a study of Harris County, Texas, Heaton and his collaborators, including Mayson, showed that cash bail and pretrial detention contributed to wrongful convictions by encouraging guilty pleas in less serious cases. These practices disproportionately disadvantaged African American defendants, they found, and cost taxpayers unnecessary money. With the county’s cooperation, Quattrone is now studying the impact of cash bail reforms instituted partly as a result of the study—a prime example of the Deming feedback loop in action.
Heaton’s research already has demonstrated that relatively modest institutional fixes—such as broadening defense teams to include social workers and other professionals, and using bail advocates—can reduce sentences without endangering public safety.
John M. MacDonald, professor of criminology and sociology, has been studying racial and ethnic inequities in the system for years. In 2018, with Ellen A. Donnelly, he published a paper, based on Delaware data, about the downstream consequences of cash bail and pretrial detention. After controlling for factors such as “severity of charges” and “criminal histories,” they found that bail and pretrial detention contributed to 30 to 47 percent of Black–white disparities in conviction and sentencing. Other MacDonald research has explored “stop, question, and frisk” practices in New York and the impacts of California sentencing reforms, which downgraded some lesser felonies to misdemeanors.
Like Heaton’s, MacDonald’s work challenges the notion that addressing systemic inequities in criminal justice requires implementing sweeping social change. Just altering criminal justice policies and practices can be effective, he argues, as well as far easier to accomplish. “We’re starting to see policies that can produce better equity with no effect on public safety,” he says.
Another Quattrone affiliate is Cary Coglianese, Edward B. Shils Professor of Law, professor of political science, and director of the Penn Program on Regulation. Coglianese, who helped launch the center, sees “tremendous opportunities for agreement, tremendous overlaps and consensus points, within our polarized society.” He has lately become intrigued by the ways in which artificial intelligence might counter human bias and other errors.
Regina Austin L’73, William A. Schnader Professor of Law and director of the Penn Program on Documentaries & the Law, has approached Quattrone’s mission from a different angle: students in her “Visual Legal Advocacy” class [“Legal Zoom In,” Nov|Dec 2016] compose short documentary videos that make the case for reforms, sometimes dovetailing with the center’s research. “Our approach tends not to be quantitative—we’re more into storytelling,” she says.
On occasion, the center, despite generally eschewing adversarial tactics, does play a role in litigation. Civil rights and criminal defense lawyer David Rudovsky, a Quattrone-affiliated senior fellow, has been involved for a decade in a legal challenge to Philadelphia’s “stop and frisk” practices on racial disparity and Fourth Amendment grounds. He says that statistical analysis by his Quattrone colleague David S. Abrams, professor of law, business economics and public policy, has been integral to the case. “We’ve learned a lot from the data,” Rudovsky says, including how rarely police stops resulted in the discovery of weapons.
Hollway piloted his “sentinel event review” concept in 2015 by examining the “Lex Street Massacre,” a 2000 mass murder in West Philadelphia in which 10 people were shot and seven killed. In that case, the wrong suspects were held for 18 months before police, relying on a competing confession and ballistics evidence, finally identified four other men as the perpetrators.
“We did a review in which we tried to figure out why, in the most scrutinized case in Philadelphia history, we had arrested four incorrect guys and not learned about it for 18 months,” Hollway recalls. “It’s never just one thing. It’s not just the original arrest: it’s the prosecutors accepting it, the defense lawyers not being able to fix it even if they know about it, the judge, etc. The checks and balances that are supposed to catch that error also failed.”
The Lex Street case showed Hollway the importance of establishing trust. “Success was going to be based on creating an environment of psychological safety where people were going to be comfortable sharing the true stories of these traumatic events,” he says. Participants in these cases “have lived through something that they do not believe should happen on their watch.” Most, he says, were “hardworking, dedicated professionals, and they pride themselves on getting it right.” When they don’t, “that is a very difficult thing for them to process,” he says, especially in “an environment where it’s all about blame and zero tolerance for error.”
Winning their cooperation was a challenge. Meanwhile, he was parenting teenagers—another sort of challenge—and hearing stories from his wife about high rates of depression, anxiety, substance abuse, and suicide in the legal profession. These converging circumstances led Hollway someplace unexpected: Penn’s master’s degree program in applied positive psychology, known as MAPP [“Degrees of Happiness,” May|Jun 2010], which he completed in 2018.
The program’s “biggest takeaway,” Hollway says, “has been that the way you think about things affects not only the way you react to things, but the outcomes that you get.” Another insight was that “the desire to get better does not imply that you’re not already doing something well,” he says. “I apply that in my Quattrone Center work all the time.” (McKeon has applied Hollway’s positive psychology insights, too: she created two unconventional positions at Morgan Lewis—“chief engagement officer” and “director of well-being,” the latter filled by one of Hollway’s MAPP classmates.)
Hollway sees his work in pragmatic rather than ideological terms. “Criminal justice is really a place that has emerged as a nonpartisan area for reform, for the most part,” he says. “We have disagreements about what ought to be a crime, and we may have disagreements about what we do with someone who’s committed a crime. But almost nobody disagrees with the idea that we should probably only punish people who have actually committed crimes, and that we should hold people who have committed crimes accountable. So where the Quattrone Center starts is, ‘Let’s make sure we get the right person in the right way.’”
While Hollway specializes in deep dives on individual cases, he says that, as the academic director, Heaton has brought the center “this unbelievable ability to do large-scale data analytics projects” on the impacts of various practices. “That changes the conversation,” Hollway says, making it possible to ask: “Is this the result that you want, or should we think about doing it a different way?”
This approach helps to forge alliances, he says, with those on both the right and the left. Sure, Hollway says, he and Dorothy Roberts come at problems from different historical perspectives. “But we can both agree,” he says, “that we don’t want a cash bail system if it’s going to add to crime.”
Roberts seconds this sentiment, despite her conviction that the criminal justice system “uses law enforcement to maintain racial inequality”—that unfairness is not a bug in the system, but a feature.
“That doesn’t mean that we can’t collaborate,” she says, “because abolitionists don’t believe that abolition is going to occur tomorrow. We recognize that it’s an incremental process that is going to involve reform. The question is which reforms will move us toward abolition and which reforms will only reinforce the existing system.”
Since “the harms that are done by the criminal justice system that the Quattrone Center wants to eliminate are also harms that abolitionists want to eliminate,” Roberts says, working together can “reduce the suffering of people victimized by the system,” as well as help “move toward a complete transformation.” The ultimate goal of abolitionists remains a more socioeconomically just, less violent society, she says, “where prisons are inconceivable because we’ve dealt with the kinds of harms that people commit today in a radically different way.”
But can such ideological disagreements really be put aside? “You can’t have any kind of legal system without making moral or normative judgments,” Coglianese says, “and there will be disagreement about what those judgments should be, and what form they should take. You can’t get around that. All the more reason that you have to be sure that whatever system you do have is perceived as legitimate.”
The current outcry for racial justice only underlines the value of the Quattrone Center’s careful, incremental, social-science-based approach, Coglianese says. Major changes occur at “the confluence of policy ideas and political pressures,” he says. “So you can have all the protests you want, all the high ideas and the slogans, but then you need to have those ideas that you know work that you can put in place. That’s why the Quattrone Center is so vital.”
Julia M. Klein, a cultural reporter and critic in Philadelphia, writes frequently for the Gazette. Her stories on Eli Rosenbaum W’76 WG’77 and Eva Moskowitz C’86 won the American Society of Journalists and Authors’ award for Outstanding Profile in 2018 and 2019. Follow her on Twitter @JuliaMKlein.