https://www.propublica.org/article/using-ai-responsibly-for-reporting PROPUBLICA The news is chaos. The truth is not. Help ProPublica dig up the facts. DONATE Skip to content Menu Donate Investigative Journalism in the Public Interest Menu Search Donate --------------------------------------------------------------------- * Nonprofit Explorer * Local Initiatives * Newsletters * About Us Email Address: [ ] Sign Up Menu Donate Close Donate Search ProPublica: [ ] Search Topics * Racial Justice * Health Care * Politics * Criminal Justice * more... Browse by Place * Midwest * Northwest * South * Southwest * Texas Type * Graphics & Data * Newsletters * Series * Videos * Local Reporting Network * Electionland * Data Store Info * About Us * Impact * Jobs & Fellowships * Contact Us Follow ProPublica * Bluesky * Instagram * Facebook * (Twitter) * YouTube * Threads * LinkedIn * RSS [GettyImages-1097249104_maxWidth_3000_maxHeight_3000_ppi_72_quality_95_embedColorProfile_true] Credit: Getty Images Technology How ProPublica Uses AI Responsibly in Its Investigations When our reporters prompted a large language model to help identify "woke" themes in a database of grants, AI helped them tell a vital accountability story about science funding and Ted Cruz. --------------------------------------------------------------------- by Charles Ornstein March 13, 2025, 8 a.m. EDT Share Change Appearance Change Appearance [Auto ] Republish Series: A Closer Look: Examining the News More in this series ProPublica is a nonprofit newsroom that investigates abuses of power. This story was originally published in our Dispatches newsletter; sign up to receive notes from our journalists. In February, my colleague Ken Schwencke saw a post on the social media network Bluesky about a database released by Sen. Ted Cruz purporting to show more than 3,400 "woke" grants awarded by the National Science Foundation that "promoted Diversity, Equity, and Inclusion (DEI) or advanced neo-Marxist class warfare propaganda." Given that Schwencke is our senior editor for data and news apps, he downloaded the data, poked around and saw some grants that seemed far afield from what Cruz, a Texas Republican, called "the radical left's woke nonsense." The grants included what Schwencke thought was a "very cool sounding project" on the development of advanced mirror coatings for gravitational wave detectors at the University of Florida, his alma mater. The grant description did, however, mention that the project "promotes education and diversity, providing research opportunities for students at different education levels and advancing the participation of women and underrepresented minorities." Schwencke thought it would be interesting to run the data through an AI large language model -- one of those powering ChatGPT -- to understand the kinds of grants that made Cruz's list, as well as why they might have been flagged. He realized there was an accountability story to tell. In that article, Agnel Philip and Lisa Song found that "Cruz's dragnet had swept up numerous examples of scientific projects funded by the National Science Foundation that simply acknowledged social inequalities or were completely unrelated to the social or economic themes cited by his committee." What We're Watching During Donald Trump's second presidency, ProPublica will focus on the areas most in need of scrutiny. Here are some of the issues our reporters will be watching -- and how to get in touch with them securely. [ProPublica-Reporters-Trump_maxWidth_3000_maxHeight_3000_] Learn more about our reporting team. We will continue to share our areas of interest as the news develops. Portrait of Andy Kroll Andy Kroll I cover justice and the rule of law, with a focus on the Justice Department, the U.S. Attorney's Office for the District of Columbia and the federal courts. * Signal * Phone * Email Portrait of Robert Faturechi Robert Faturechi I have been reporting on Trump Media, the parent company of Truth Social. I'm also reporting on the Trump administration's trade policies, including tariffs. * Signal * WhatsApp * Email Photo of Maryam Jameel Maryam Jameel I'm an engagement reporter interested in immigration, labor and the federal workforce. * Phone * Signal * Email Portrait of Mark Olalde Mark Olalde I'm interested in Trump's and his allies' promises to dismantle the federal bureaucracy and laws that protect the environment. * Email * Signal [20191029-federal-govt-callout-1x1] If you don't have a specific tip or story in mind, we could still use your help. Sign up to be a member of our federal worker source network to stay in touch. We're trying something new. Was it helpful? Among them: a $470,000 grant to study the evolution of mint plants and how they spread across continents. As best Philip and Song could tell, the project was flagged because of two specific words used in its application to the NSF: "diversify," referring to the biodiversity of plants, and "female," where the application noted how the project would support a young female scientist on the research team. Another involved developing a device that could treat severe bleeding. It included the words "victims" -- as in gunshot victims -- and "trauma." Neither Cruz's office nor a spokesperson for Republicans on the Senate Committee on Commerce, Science and Transportation responded to our requests for comment for the article. The story was a great example of how artificial intelligence can help reporters analyze large volumes of data and try to identify patterns. First, we told the AI model to mimic an investigative journalist reading through each of these grants to identify whether they contained themes that someone looking for "wokeness" may have spotted. And crucially, we made sure to tell the model not to guess if it wasn't sure. (AI models are known to hallucinate, and we wanted to guard against that.) For newsrooms new to AI and readers who are curious how this worked in practice, here's an excerpt of the actual prompt we used: Background: We will be showing you grants from the national science foundation that have been targeted for cancellation because they contain themes as identified by Republican Senator Ted Cruz's office as involving woke ideology; diversity, equity, and inclusion; or pro-Marxist ideology. We are looking to analyze themes of the award descriptions in this list to determine what may have terms or themes that would be considered "woke" or related to Diversity, Equity, and Inclusion (DEI). It is your task to determine whether or not the text contains these themes and tell me about what you've found. Only extract information from the NSF grant if it contains the information requested. -- As an investigative journalist, I am looking for the following information -- woke_description: A short description (at maximum a paragraph) on why this grant is being singled out for promoting "woke" ideology, Diversity, Equity, and Inclusion (DEI) or advanced neo-Marxist class warfare propaganda. Leave this blank if it's unclear. why_flagged: Look at the "STATUS", "SOCIAL JUSTICE CATEGORY", "RACE CATEGORY", "GENDER CATEGORY" and "ENVIRONMENTAL JUSTICE CATEGORY" fields. If it's filled out, it means that the author of this document believed the grant was promoting DEI ideology in that way. Analyze the "AWARD DESCRIPTIONS" field and see if you can figure out why the author may have flagged it in this way. Write it in a way that is thorough and easy to understand with only one description per type and award. citation_for_flag: Extract a very concise text quoting the passage of "AWARDS DESCRIPTIONS" that backs up the "why_flagged" data. Of course, members of our staff reviewed and confirmed every detail before we published our story, and we called all the named people and agencies seeking comment, which remains a must-do even in the world of AI. Philip, one of the journalists who wrote the query above and the story, is excited about the potential new technologies hold but also is proceeding with caution, as our entire newsroom is. "The tech holds a ton of promise in lead generation and pointing us in the right direction," he told me. "But in my experience, it still needs a lot of human supervision and vetting. If used correctly, it can both really speed up the process of understanding large sets of information, and if you're creative with your prompts and critically read the output, it can help uncover things that you may not have thought of." This was just the latest effort by ProPublica to experiment with using AI to help do our jobs better and faster, while also using it responsibly, in ways that aid our human journalists. In 2023, in partnership with The Salt Lake Tribune, a Local Reporting Network partner, we used AI to help uncover patterns of sexual misconduct among mental health professionals disciplined by Utah's licensing agency. The investigation relied on a large collection of disciplinary reports, covering a wide range of potential violations. To narrow in on the types of cases we were interested in, we prompted AI to review the documents and identify ones that were related to sexual misconduct. To help the bot do its work, we gave it examples of confirmed cases of sexual misconduct that we were already familiar with and specific keywords to look for. Each result was then reviewed by two reporters, who used licensing records to confirm it was categorized correctly. In addition, during our reporting on the 2022 school shooting in Uvalde, Texas, ProPublica and The Texas Tribune obtained a trove of unreleased raw materials collected during the state's investigation. This included hundreds of hours of audio and video recordings, which were difficult to sift through. The footage wasn't organized or clearly labeled, and some of it was incredibly graphic and disturbing for journalists to watch. Read More [svg] Washington Blues: A Government Town Faces a Gloomy Future We used self-hosted open-source AI software to securely transcribe and help classify the material, which enabled reporters to match up related files and to reconstruct the day's events, showing in painstaking detail how law enforcement's lack of preparation contributed to delays in confronting the shooter. We know full well that AI does not replicate the very time-intensive work we do. Our journalists write our stories, our newsletters, our headlines and the takeaways at the top of longer stories. We also know that there's a lot about AI that needs to be investigated, including the companies that market their products, how they train them and the risks they pose. But to us, there's also potential to use AI as one of many reporting tools that enables us to examine data creatively and pursue the stories that help you understand the forces shaping our world. Agnel Philip, Ken Schwencke, Hannah Fresques and Tyson Evans contributed reporting. Filed under -- * Technology Protect Journalism That Demands Accountability The story you just read was made possible by our readers. We hope it inspires you to support ProPublica, so we can continue producing investigations that shine a light on power, uncover the truth, and drive real change. ProPublica is a nonprofit newsroom dedicated to nonpartisan, evidence-based journalism that holds power accountable. Founded in 2008 to address the decline in investigative reporting, we've spent over 15 years uncovering injustice, corruption, and abuse of power -- work that is slow, expensive and more critical than ever to our democracy. With seven Pulitzer Prizes and reforms sparked in state and local governments, businesses, institutions and more, our reporting ensures that the public interest comes first. Today, the stakes are higher than ever. From ethics in our government offices, to reproductive health care, to the climate crisis and beyond, ProPublica remains on the front lines of the stories that matter most. Your gift helps us keep the powerful accountable and the truth accessible. Join over 60,000 supporters nationwide in standing up for investigative journalism that informs, inspires, and creates lasting impact. Thank you for making this work possible. Donate to ProPublica Portrait of Charles Ornstein Charles Ornstein * * Charles Ornstein is managing editor, local, overseeing ProPublica's local initiatives. These include offices in the Midwest, South, Southwest and Northwest, a joint initiative with the Texas Tribune, and the Local Reporting Network, which works with local news organizations to produce accountability journalism on issues of importance to their communities. * More Stories * Contact Me More in Technology [svg] How a Global Online Network of White Supremacists Groomed a Teen to Kill The murders of two people outside an LGBTQ+ bar at first looked like the act of a lone shooter. A ProPublica and FRONTLINE investigation shows they were, in fact, the culmination of a coordinated, international recruiting effort by online extremists. by A.C. Thompson, ProPublica and FRONTLINE, James Bandler, ProPublica, and Lukas Diko, Investigative Center of Jan Kuciak, March 8, 2025, 5 a.m. EST [svg] As Facebook Abandons Fact-Checking, It's Also Offering Bonuses for Viral Content Meta decided to stop working with U.S. fact-checkers at the same time as it's revamping a program to pay bonuses to creators with high engagement numbers, potentially pouring accelerant on the kind of false posts the company once policed. by Craig Silverman, Feb. 24, 2025, 6 a.m. EST [svg] Zero Trust Following a Series of Government Hacks, Biden Closes Out His Administration With New Cybersecurity Order The executive order follows a ProPublica investigation that found Microsoft prioritized profit over security, leaving the federal government vulnerable to the largest hack in U.S. history. Vendors must now demonstrate that their products are secure. by Renee Dudley, Jan. 17, 2025, 4:25 p.m. EST [svg] Zero Trust Microsoft's "Free" Plan to Upgrade Government Cybersecurity Was Designed to Box Out Competitors and Drive Profits, Insiders Say When the White House welcomed Microsoft's offer of $150 million in tech services, it helped the world's largest software provider tighten its grip on federal business and freeze out competitors. by Renee Dudley, with research by Doris Burke, Nov. 15, 2024, 6 a.m. EST [svg] Exploiting Meta's Weaknesses, Deceptive Political Ads Thrived on Facebook and Instagram in Run-Up to Election Despite Meta's stated commitment to crack down on harmful content, it failed to catch tens of thousands of ads that used false claims and deepfakes of political figures to collect users' sensitive personal data or bait them into monthly charges. by Craig Silverman, ProPublica, and Priyanjana Bengani, Tow Center for Digital Journalism, Oct. 31, 2024, 5 a.m. EDT [svg] Neo-Nazi Telegram Users Panic Amid Crackdown and Arrest of Alleged Leaders of Online Extremist Group An analysis by ProPublica and FRONTLINE shows a surge in activity on Telegram channels aligned with the Terrorgram Collective, as allies tried to rally support for their comrades in custody and sought to oust users they believed to be federal agents. by A.C. Thompson, James Bandler and Brandon Roberts, Sept. 25, 2024, 12 p.m. EDT Most Read [svg] "The President Wanted It and I Did It": Recording Reveals Head of Social Security's Thoughts on DOGE and Trump In a recording obtained by ProPublica, acting Social Security Commissioner Leland Dudek portrayed his agency as facing peril, while also encouraging patience with "the DOGE kids." by Eli Hager, March 12, 2025, 10:25 a.m. EDT [svg] National Cancer Institute Employees Can't Publish Information on These Topics Without Special Approval Vaccines. Fluoride. Autism. Communications involving these and 20 other "controversial, high profile, or sensitive" topics will get extra scrutiny under Health Secretary Robert F. Kennedy Jr. by Annie Waldman and Lisa Song, March 10, 2025, 1:45 p.m. EDT [svg] Ethical Concerns Surround Sen. Joni Ernst's Relationships With Top Military Officials Who Lobbied Her Committee Ernst, an Iowa Republican, is one of the most influential voices in Congress on military topics. Ethics and military experts say her relationships with top Air Force and Navy officials created potential conflicts of interest. by Robert Faturechi, March 4, 2025, 7 a.m. EST [svg] Texas Lawmakers Want a Charter School Network to Stop Paying Its Superintendent Nearly $900K. The School Board Says No. The rebuke from lawmakers and charter school leaders came after an investigation from ProPublica and The Texas Tribune revealed that Salvador Cavazos, who oversees fewer than 1,000 students, is among the most well-paid superintendents in the country. by Lexi Churchill, ProPublica and The Texas Tribune, and Ellis Simani, ProPublica, March 14, 2025, 6 a.m. CDT [svg] Two Transgender Girls, Six Federal Agencies. How Trump Is Trying to Pressure Maine Into Obedience. Maine said it wouldn't break state law to follow President Donald Trump's order barring transgender girls from playing girls' sports. Then came a barrage of investigations and threats targeting the state's federal funding. by Callie Ferguson and Erin Rhoda, Bangor Daily News, and Jennifer Smith Richards and Jodi S. Cohen, ProPublica, March 11, 2025, 2:45 p.m. EDT Stay informed with the Daily Digest. Email Address: [ ] Sign Up Investigative Journalism in the Public Interest (c) Copyright 2025 Pro Publica Inc. Sites * ProPublica * Local Reporting Network * Texas Tribune Partnership * The Data Store * Electionland Sections * Topics * Series * Videos * News Apps * Get Involved * The Nerd Blog * @ProPublica * Events Info * About * Board and Advisors * Officers and Staff * Diversity * Jobs and Fellowships * Media Center * Reports * Impact * Awards * Corrections Policies * Code of Ethics * Advertising Policy * Privacy Policy Follow * Newsletters * iOS and Android * RSS Feed More * Send Us Tips * Steal Our Stories * Browse via Tor * Contact Us * Donate * More Ways to Give Close this dialogue Republish This Story for Free Creative Commons License (CC BY-NC-ND 3.0) --------------------------------------------------------------------- Thank you for your interest in republishing this story. You are free to republish it so long as you do the following: * You have to credit ProPublica and any co-reporting partners. In the byline, we prefer "Author Name, Publication(s)." At the top of the text of your story, include a line that reads: "This story was originally published by ProPublica." You must link the word "ProPublica" to the original URL of the story. * If you're republishing online, you must link to the URL of this story on propublica.org, include all of the links from our story, including our newsletter sign up language and link, and use our PixelPing tag. * If you use canonical metadata, please use the ProPublica URL. For more information about canonical metadata, refer to this Google SEO link. * You can't edit our material, except to reflect relative changes in time, location and editorial style. (For example, "yesterday" can be changed to "last week," and "Portland, Ore." to "Portland" or "here.") * You cannot republish our photographs or illustrations without specific permission. Please contact [email protected]. * It's okay to put our stories on pages with ads, but not ads specifically sold against our stories. You can't state or imply that donations to your organization support ProPublica's work. * You can't sell our material separately or syndicate it. This includes publishing or syndicating our work on platforms or apps such as Apple News, Google News, etc. * You can't republish our material wholesale, or automatically; you need to select stories to be republished individually. (To inquire about syndication or licensing opportunities, contact [email protected].) * You can't use our work to populate a website designed to improve rankings on search engines or solely to gain revenue from network-based advertisements. * We do not generally permit translation of our stories into another language. * Any website our stories appear on must include a prominent and effective way to contact you. [ ] Copy HTML undefined undefined