A decadelong passion for local journalism and a push for government transparency in California have culminated in the latest version of Digital Democracy, a groundbreaking project led by a dedicated professor.
Digital Democracy offers journalists and citizens unparalleled access to a comprehensive, searchable database of state-level legislative information, fostering greater civic engagement and healthier public interest decisions in California – the world’s fifth-largest economy with an annual budget of about $300 billion.
“We are processing 200 hours of hearings a week, with up to 10 happening simultaneously during the Legislature’s busy periods,” said Foaad Khosmood, computer science professor and research director at the Institute for Advanced Technology & Public Policy (IATPP). “It would take an army of 200 reporters to cover that many hearings.”
Since his days as a technology manager for the Mustang Daily (now Mustang News) at Cal Poly, Khosmood has been a staunch supporter of local journalism. Disturbed by the ongoing cuts to reporters and resources in newsrooms, he has witnessed the rise of “news deserts” – communities with limited access to credible and comprehensive information – across California and the country.
Digital Democracy has provided Khosmood with an outlet for action and activism.
“We wanted to empower journalists to write stories that help Californians understand their state government and hold their politicians accountable,” Khosmood said. “With local journalism devastated, Digital Democracy is our way of making a difference.”
Project Evolution
Digital Democracy was initially launched in 2015 by former state Sen. Sam Blakeslee and then-Lt. Gov. Gavin Newsom through the IATPP with key contributors including Khosmood and Cal Poly alumna Christine Robertson, the original designer and manager of the platform.
In March 2024, a revamped Digital Democracy was launched by CalMatters, a nonprofit news organization, broadening its reach and impact.
The database consolidates hard-to-access public information, capturing every word from public hearings and floor sessions, the full text of bills with amendments, votes, and lists of supporters and opponents. It also includes financial data such as campaign donations, expenditures, gifts and travel, and district data like voter registration, election results and demographics.
“We use all kinds of AI on this,” said Khosmood, noting that the database and artificial intelligence were designed and built by faculty and students at Cal Poly.
Additionally, it supports journalism by providing reporters with quick and easy access to extensive legislative information, enabling them to uncover relationships, patterns and anomalies within the Legislature and the policymaking process.
“We are leveraging the uniqueness of this database, which compiles records that aren’t available in a single location anywhere else,” Khosmood said. “Some of these records were never available before.”
For example, through Digital Democracy, CalMatters reporters uncovered that legislators often kill bills by simply not voting. This insight was made possible by the database, which provided access to 1 million votes cast by current legislators over the past five years.
Digital Democracy also assists journalists in identifying stories about the legislative process with a custom AI tool that scans the database and suggests story ideas. These ideas and the resulting stories are then shared with news outlets across the state.
Innovative Data Solutions
Cal Poly computer science graduate Thomas Gerrity, who was one of at least 12 graduate students to write their master’s thesis on Digital Democracy, now oversees the technical program for CalMatters.
“It’s interesting to see legislators take notice, realizing they are being monitored,” he said.
He highlighted a feature that tracks the number of words each legislator speaks, creating a “Top 10 Most Talkative” list. Legislators have taken note, with some even mentioning their ranking during addresses about bills or issues.
Gerrity regularly collaborates with Cal Poly students and at least one from outside Cal Poly who is working on the project this summer. Sasha Prostota, a data science student from UCLA studying applied math, is helping tackle one of the major challenges in mining data from legislative hearings: names.
There is no standardized identification when a person, organization or company testifies in a public hearing. Variations like “ACLU” vs. “American Civil Liberties Union” or subgroups such as “Association for Commuter Transportation” vs. “Association for Commuter Transportation, Southern California Chapter” can complicate data analysis, as these entities are the same.
“This was a serious problem that came up early, and there was no existing solution,” Khosmood said. “Many organizations have different names, but it’s crucial we accurately characterize these groups to determine their political influence.”
Working out of IATPP headquarters on campus, Prostota and Gerrity are developing a methodology to identify relationships between groups and their names, with the goal of creating an automated system.
“Applying what I’ve learned in my classes to something impactful and meaningful is very rewarding,” said Prostota, who advocates for greater transparency between citizens and their government in California.
For more details and to see Digital Democracy in action, explore the project here.
By Emily Slater
