https://techxplore.com/news/2021-06-dna-databases-method-indexing-weeks.html logotype Topics * Week's top * Latest news * Unread news * Subscribe [ ] Science X Account [ ] [ ] [*] Remember me Sign In Click here to sign in with or Forget Password? Not a member? Sign up Learn more * Automotive * Business * Computer Sciences * Consumer & Gadgets * Electronics & Semiconductors * Energy & Green Tech * Engineering * Hardware * Hi Tech & Innovation * Internet * Machine learning & AI * Other * Robotics * Security * Software * Telecom * * share this! * 253 * 9 * Share * Email 1. Home 2. Computer Sciences 1. Home 2. Machine learning & AI * * * --------------------------------------------------------------------- June 28, 2021 DNA databases: New method cuts indexing from weeks to hours, searches to minutes by Rice University genome Credit: CC0 Public Domain Rice University computer scientists are sending RAMBO to rescue genomic researchers who sometimes wait days or weeks for search results from enormous DNA databases. DNA sequencing is so popular, genomic datasets are doubling in size every two years, and the tools to search the data haven't kept pace. Researchers who compare DNA across genomes or study the evolution of organisms like the virus that causes COVID-19 often wait weeks for software to index large, "metagenomic" databases, which get bigger every month and are now measured in petabytes. RAMBO, which is short for "repeated and merged bloom filter," is a new method that can cut indexing times for such databases from weeks to hours and search times from hours to seconds. Rice University computer scientists presented RAMBO last week at the Association for Computing Machinery data science conference SIGMOD 2021. "Querying millions of DNA sequences against a large database with traditional approaches can take several hours on a large compute cluster and can take several weeks on a single server," said RAMBO co-creator Todd Treangen, a Rice computer scientist whose lab specializes in metagenomics. "Reducing database indexing times, in addition to query times, is crucially important as the size of genomic databases are continuing to grow at an incredible pace." To solve the problem, Treangen teamed with Rice computer scientist Anshumali Shrivastava, who specializes in creating algorithms that make big data and machine learning faster and more scalable, and graduate students Gaurav Gupta and Minghao Yan, co-lead authors of the peer-reviewed conference paper on RAMBO. RAMBO uses a data structure that has a significantly faster query time than state-of-the-art genome indexing methods as well as other advantages like ease of parallelization, a zero false-negative rate and a low false-positive rate. "The search time of RAMBO is up to 35 times faster than existing methods," said Gupta, a doctoral student in electrical and computer engineering. In experiments using a 170-terabyte dataset of microbial genomes, Gupta said RAMBO reduced indexing times from "six weeks on a sophisticated, dedicated cluster to nine hours on a shared commodity cluster." Yan, a Ph.D student in computer science, said, "On this huge archive, RAMBO can search for a gene sequence in a couple of milliseconds, even sub-milliseconds using a standard server of 100 machines." RAMBO improves on the performance of Bloom filters, a half-century-old search technique that has been applied to genomic sequence search in a number of previous studies. RAMBO improves on earlier Bloom filter methods for genomic search by employing a probabilistic data structure known as a count-min sketch that "leads to a better query time and memory trade-off" than earlier methods, and "beats the current baselines by achieving a very robust, low-memory and ultrafast indexing data structure," the authors wrote in the study. Gupta and Yan said RAMBO has the potential to democratize genomic search by making it possible for almost any lab to quickly and inexpensively search huge genomic archives with off-the-shelf computers. "RAMBO could decrease the wait time for tons of investigations in bioinformatics, such as searching for the presence of SARS-CoV-2 in wastewater metagenomes across the globe," Yan said. "RAMBO could become instrumental in the study of cancer genomics and bacterial genome evolution, for example." --------------------------------------------------------------------- Explore further 'Rambo' protein may not be so violent after all --------------------------------------------------------------------- More information: Gaurav Gupta et al, Fast Processing and Querying of 170TB of Genomics Data via a Repeated And Merged BloOm Filter (RAMBO), Proceedings of the 2021 International Conference on Management of Data (2021). DOI: 10.1145/3448016.3457333 Provided by Rice University Citation: DNA databases: New method cuts indexing from weeks to hours, searches to minutes (2021, June 28) retrieved 29 June 2021 from https://techxplore.com/news/ 2021-06-dna-databases-method-indexing-weeks.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only. 262 shares * Facebook * Twitter * Email Feedback to editors * Featured * Last Comments * Popular A model to predict how much humans and robots can be trusted with completing specific tasks 9 hours ago 0 Plans in place to build carbon capture facility in Scotland that can capture a million tonnes a year Jun 28, 2021 6 LyricJam: A system that can generate lyrics for live instrumental music Jun 28, 2021 0 Vulnerabilities found in Dell BIOSConnect features within Dell SupportAssist Jun 25, 2021 0 An autonomous drone for search and rescue in forests using optical sectioning algorithm Jun 25, 2021 0 --------------------------------------------------------------------- [gif] Longer-lived lithium-metal battery marks step forward for electric vehicles 3 hours ago [gif] Conductive seams, when strategically placed in clothing, can accurately track body motion: study 6 hours ago [gif] Faster, greener technique to improve recycling process for electric vehicle batteries 7 hours ago [gif] Steering wind turbines creates greater energy potential 8 hours ago [gif] A model to predict how much humans and robots can be trusted with completing specific tasks 9 hours ago [gif] Kiriform structures harness buckling for stable, deployable structures 15 hours ago [gif] How to build a better wind farm 15 hours ago * Related Stories [gif] 'Rambo' protein may not be so violent after all Jul 06, 2015 [gif] Apple may sweeten macOS 10.15 with iPad external display support Apr 18, 2019 [gif] Bad news for fake news: New research helps combat social media misinformation Dec 10, 2020 [gif] Scientists show benefits of bioinformatics with PlasmidHawk tool Feb 26, 2021 [gif] New search engine for single cell atlases Mar 03, 2021 [gif] Study finds growth of genomic databases affects species accuracy Oct 30, 2018 * Recommended for you [gif] A model to predict how much humans and robots can be trusted with completing specific tasks 9 hours ago [gif] LyricJam: A system that can generate lyrics for live instrumental music Jun 28, 2021 [gif] AI learns to predict human behavior from videos Jun 28, 2021 [gif] Using photos of toasters and fridges to train algorithms in detecting COVID-19 Jun 28, 2021 [gif] How AI can transform layering technique for semiconductors Jun 25, 2021 [gif] Researchers create an artificial tactile skin that mimics human tactile recognition processes Jun 24, 2021 User comments Let us know if there is a problem with our content Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form. For general feedback, use the public comments section below (please adhere to guidelines). Please select the most appropriate category to facilitate processing of your request [-- please select one -- ] [ ] [ ] [ ] [ ] [ ] Your message to the editors [ ] Your email (only if you want to be contacted back) [ ] Send Feedback Thank you for taking time to provide your feedback to the editors. Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages. E-mail the story DNA databases: New method cuts indexing from weeks to hours, searches to minutes Your friend's email [ ] Your email [ ] [ ] I would like to subscribe to Science X Newsletter. Learn more Your name [ ] Note Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Tech Xplore in any form. [ ] [ ] [ ] [ ] [ ] [ ] [ ] Your message [ ] Send Phys.org logotype Phys.org Daily science news on research developments and the latest scientific innovations MedicalXpress logotype Medical Xpress Medical research advances and health news ScienceX logotype Science X The most comprehensive sci-tech news coverage on the web Newsletters [ ] Subscribe Science X Daily and the Weekly Email Newsletter are free features that allow you to receive your favorite sci-tech news updates in your email inbox Follow us * * * * * Top * Home * Search * Mobile version * Help * FAQ * About * Contact * Science X Account * Sponsored Account * Newsletter * Archive * Android app * iOS app * RSS feeds * Push notification (c) Tech Xplore 2014 - 2021 powered by Science X Network Privacy policy Terms of use Your Privacy This site uses cookies to assist with navigation, analyse your use of our services, and provide content from third parties. By using our site, you acknowledge that you have read and understand our Privacy Policy and Terms of Use. Ok More Information E-mail newsletter [ ] Subscribe Follow us * * * * It appears that you are currently using Ad Blocking software. What are the consequences? x Quantcast