For decades, the Philadelphia Bar Association has enlisted members to spend 40-plus hours screening each judicial candidate in Philadelphia and decide on recommendations. Most everyone agrees the process is thorough and the group is the best at separating the qualified from the unqualified candidates.

But for years the group wondered if its rankings actually mattered. Did anybody see them? And if they did were they ever enough to counter the random question of ballot position, or the organization of the Democratic City Committee and ward leaders?

After an experiment during the May primary, the group discovered its ratings can matter. It teamed up with Philly data analysis firm Econsult Solutions, and had volunteers distribute a listing of candidates it recommended (and candidates it didn’t) at 41 polling places.

So how’d they do?

The group believes its efforts made a difference. The Bar Association’s presence increased the gap between recommended and not recommended Common Pleas candidates by about 0.4 percent at the 41 polling locations, a significant impact in an election where winners needed just over 4 percent of the vote. Had the Bar Association placed volunteers all over the city, only one not-recommended judge would have won, as opposed to the three that actually did.

“It really had an effect,” said Jonathan Tannen, a director at Econsult Solutions.

Given the results, the Bar Association is now launching a committee to strategize how to impact the next judicial election and to approach lawmakers in Harrisburg with the data as part of a plan to overhaul the current, random ballot position selection process for Philadelphia.

“What we know is when the electorate gets informed, the outcome is different than the randomness of the ballot,” said Eric Weitz, founder of The Weitz Firm and chair of the Bar’s judicial commission. “So what will we prefer: An outcome based on randomness or based on information? And I think that’s something the legislators are going to have to wrestle with.”

The process of handing out lists of recommended candidates was spearheaded by Matt Olesh, senior counsel at Chamberlain Hrdlicka, chair of the Bar’s Young Lawyers Division and a Billy Penn Who’s Next: Law honoree. Olesh and the Bar had been inspired by an Econsult study showing ballot position had a vastly bigger effect on the election of judges than the Democratic City Committee or the Bar’s recommendations. So the Bar and Econsult Solutions got about 80 volunteers, and they were placed at randomly selected polling positions for half-day shifts.

Nine Common Pleas judges were elected, six of them recommended by the Bar. Three were not recommended, Deborah Cianfrani, Shanese Johnson and Mark B. Cohen. Cianfrani and Johnson were endorsed by the City Committee. All three were positioned in the first column on ballots.

According to Econsult, which plans to explain the findings in greater depth on its website Friday, recommended candidates Daniel R. Sulman and Jennifer Schultz won in place of Cohen and Johnson at the 41 polling places staffed by the Bar’s volunteers. Neither Sulman and Schultz were in the first column on the ballot.

“It’s pretty validating for the work the judicial commission does and the fact that when you get that work in the voters’ hands and educate them about what it means and what goes into it, it makes a difference,” Olesh said. “On the other hand, to get us to ultimately impact the election, we would’ve required a lot more volunteers.”

Per Econsult, the Bar would have needed a representative at about 500 polling locations for Sulman and Schultz to have won in place of Cohen and Johnson. That’s 1,000 volunteers.

“The good news is this seems to have demonstrated the homework they do is worth the effort,” said David Thornburgh, president and CEO of the local government watchdog group Committee of Seventy. “The second question is can you scale it up to really have a system-wide impact, not just a spot impact?”

Olesh would love to get enough volunteers to staff 500 polling locations but admits it would be ambitious. There are other ways the Bar may seek to better utilize its information. For one thing, Econsult’s data show the volunteers were statistically insignificant at higher-income polling spots because they were mostly already voting for recommended candidates. Tannen said the volunteers impacted moderate income neighborhoods to a much greater extent, particularly those with majority white or Hispanic populations.

The Bar is also hoping the study can be part of a larger strategy where it can convince lawmakers proper information can influence voters. The group doesn’t have any concrete positions yet on how to produce change, but it wants to push for getting rid of that infamous coffee can and random selection of ballot position.

“Don’t we now have statistical proof,” Weitz said, “that maybe the ballot itself needs to change?”

Mark Dent is a reporter/curator at BillyPenn. He previously worked for the Pittsburgh Post-Gazette, where he covered the Jerry Sandusky scandal, Penn State football and the Penn State administration. His...