SEPTA Key glitch kept riders from buying passes and fares

Update, 8/12/24, 3 p.m.: SEPTA says the problem has been fixed after a software update by Conduent. The agency is continuing to evaluate the cost in lost revenue.

SEPTA’s often-troubled Key payment system hit another glitch Monday morning that kept riders from adding money to their Key cards and stymied Regional Rail conductors trying to collect fares.

Starting around 5 a.m., people trying to use any part of the SEPTA Key system for purchasing passes and single trip fares — including the Key website and app, kiosks on the El, and handheld devices used by rail conductors — couldn’t log in or received error messages.

“Keycard kiosks, site and windows are down. How are we supposed to pay if we don’t have cash?” one rider asked SEPTA’s Twitter account later Monday morning.

The potential loss of several hours of revenue from new fare purchases comes as budget woes force the cash-strapped transit agency to freeze spending and consider fare hikes. State lawmakers declined to approve a bailout for SEPTA in June, providing only a one-time, $51 million bump rather than the $161 million Gov. Josh Shapiro had proposed.

“We’re not in a position where we can really lose any revenue that we have coming in,” SEPTA spokesperson Andrew Busch said. “So when something like this comes up, we move as fast as possible to try to get our hands around it and get it resolved.”

The glitch also occurs as SEPTA prepares to move beyond the problem-prone, overbudget SEPTA Key system and potentially end its relationship with Conduent, the company that built and maintains the fare technology. 

Within the next two months the agency’s board expects to select a contractor to build its next-generation Key 2.0 fare system, Busch said.

Waiving fares, tracking losses

People who already had money loaded on their cards on Monday morning could still use them to pay train, bus, and subway fares, and the contactless readers that let riders swipe credit or debit cards on the bus and subway were working fine, Busch said.

Conductors also waived fares for affected passengers and were trying to tally their numbers, he said. Conduent could be on the hook for SEPTA’s lost revenues.

“We’ve instructed our personnel in the field that when people have that problem, to keep track of that,” he said. “Certainly no one’s being denied service, and we’re working with the vendor, Conduent, on a fix. They’re actively looking at it and determining what the cause is and how to fix it.”

SEPTA has lost out on fares many times in the past when the card-swipe validators on buses freeze up, an experience familar to frequent bus riders.

For example, a spate of freezes plagued the card readers for months starting in August 2018, early in the system’s history. The issue was attributed to a software memory leak that caused validators to get stuck on a “processing” message, the Inquirer reported at the time.

This week’s glitch is different in that the card validators are unaffected, Busch said. Rather, there seems to be a problem with Conduit’s underlying software system that handles purchases, knocking out that capability across different devices.

“Anything that would normally process a new sale is down,” he said. 

The number of people affected and the financial impact weren’t yet clear as of late Monday morning. Because the problem didn’t affect many SEPTA pass holders and other riders, and might only end up lasting a few hours, it could end up only being a “relatively minor” issue financially, Busch said.

A long and troubled history

The hiccup is the latest in a long string of problems the five-county transit authority has encountered since it first began the process of creating an electronic smart card fare system 16 years ago.

Proposed in 2008 at a cost of $100 million, the system wasn’t bid out until 2011 due to state budget issues and other hurdles. A division of Xerox Corp. — which was later spun off as Conduent Inc. — ended up getting the contract valued at $129.5 million, with the goal of getting it up and running by 2013.

Because of the supposed complexity of the software design, numerous change orders, and other challenges, SEPTA Key didn’t launch until 2016. 

It still wasn’t available on the Regional Rail, and a confusing kiosk interface led thousands of users to accidentally double-order Key cards and necessitated a software redesign. By March 2018, Conduent’s contract had risen to nearly $160 million and the project’s total cost was nearing $300 million.

Regional Rail finally got Key in 2019. By late 2021, Conduent’s contract had soared to $238 million, even though riders still couldn’t pay by swiping their credit cards at turnstiles and validators, as envisioned when the system was first proposed, or with smartphones. 

Tap-to-pay or “contactless” payment finally began rolling out on buses, trolleys, and the subway last fall, and is expected to reach Regional Rail later this year, Busch said.

SEPTA is now preparing to move on to Key 2.0, with the requirement that the next system have an “open architecture” that allows the agency to hire any company to address problems, make upgrades, or connect to new payment methods, rather than being stuck with just one firm as it has been for the last 13 years.

More than 20 firms reportedly expressed interest in building the new system. It’s unclear if Conduent is among them. Meanwhile, SEPTA’s payments to the company have reportedly exceeded $285 million and could continue to grow, as it could be 2027 — or later — before Key 2.0 is phased in.