https://github.com/dato-ai/dato.rss Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Changelog * Solutions + By Size + Enterprise + Teams + Compare all + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} dato-ai / dato.rss Public * * Notifications * Fork 14 * Star 483 The best RSS Search experience you can find datorss.com License MIT license 483 stars 14 forks Star Notifications * Code * Issues 1 * Pull requests 3 * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights dato-ai/dato.rss This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 8 branches 1 tag Code * Clone HTTPS GitHub CLI [https://github.com/d] Use Git or checkout with SVN using the web URL. [gh repo clone dato-a] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @davidesantangelo davidesantangelo Update README.md ... 356470f Sep 15, 2022 Update README.md 356470f Git stats * 162 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github Initial commit Feb 2, 2021 app Increase the number of imported feeds Sep 8, 2022 bin Initial commit Feb 2, 2021 config Transfer onwership to dato-ai organization Aug 11, 2022 db Add published_at index to make faster queries Aug 10, 2022 lib Add X-Robots-Tag: none to avoid indexing May 3, 2021 log Initial commit Feb 2, 2021 public Update robots.txt May 2, 2021 spec Update gems and JS libraries Nov 10, 2021 storage Initial commit Feb 2, 2021 test Initial commit Feb 2, 2021 tmp Initial commit Feb 2, 2021 vendor Initial commit Feb 2, 2021 .DS_Store Add og:image logo Feb 5, 2021 .browserslistrc Initial commit Feb 2, 2021 .gitattributes Initial commit Feb 2, 2021 .gitignore Initial commit Feb 2, 2021 .rubocop.yml Add RuboCop performance extension Apr 22, 2021 .rubocop_todo.yml Add RuboCop performance extension Apr 22, 2021 .ruby-version Initial commit Feb 2, 2021 Capfile Initial commit Feb 2, 2021 Gemfile Add RuboCop performance extension Apr 22, 2021 Gemfile.lock Increase the number of imported feeds Sep 8, 2022 LICENSE Initial commit Feb 2, 2021 README.md Update README.md Sep 15, 2022 Rakefile Initial commit Feb 2, 2021 SECURITY.md Create SECURITY.md Feb 15, 2021 babel.config.js Initial commit Feb 2, 2021 config.ru Initial commit Feb 2, 2021 package.json Initial commit Feb 2, 2021 postcss.config.js Initial commit Feb 2, 2021 tailwind.config.js Code CleanUp Feb 3, 2021 yarn.lock Improper Verification of Cryptographic Signature in node-forge Aug 16, 2022 View code [ ] DATO.RSS SUPPORT BETA Live Example Search Feed Rank Machine Learning Add Feed Wiki Built With Sponsor me Contributing License README.md DATO.RSS A seamless RSS Search Engine experience with a hint of Machine Learning. SUPPORT github sponsor BETA Dato.RSS is in beta, and will likely see many changes in the near future. If you have comments or suggestions, please send them to us using the Issues TAB. Thanks for trying the beta! Alt Text Search Engine: Quickly search through the millions of available RSS feeds. RESTful API: Turns feed data into an awesome API. The API simplifies how you handle RSS, Atom, or JSON feeds. You can add and keep track of your favourite feed data with a simple, fast and clean REST API. All entries are enriched by Machine Learning and Semantic engines. Live https://datorss.com Example curl 'https://datorss.com/api/searches?q=news' | json_pp { "data": [ { "id": "86b0f829-e300-4eef-82e1-82f34d03aff6", "type": "entry", "attributes": { "title": "\"Pandemic, Infodemic\": 2 Cartoon Characters Battling Fake News In Assam", "url": "https://www.ndtv.com/india-news/coronavirus-pandemic-infodemic-2-cartoon-characters-battling-fake-news-in-assam-2222333", "published_at": 1588448805, "body": "An English daily in Assam's Guwahati has been publishing a cartoon strip to tackle the fake news related to the coronavirus pandemic. The two central characters- \"Pandemic and Infodemic\"- are being...\"\"/", "text": "An English daily in Assam's Guwahati has been publishing a cartoon strip to tackle the fake news related to the coronavirus pandemic. The two central characters- \"Pandemic and Infodemic\"- are being...", "categories": [ "all india" ], "sentiment": null, "parent": { "id": "c97bdae6-b5d1-4966-b9f3-615e29d4d47d", "title": "NDTV News - Special", "url": "feed:http://feeds.feedburner.com/NDTV-LatestNews", "rank": 99 }, "tags": [] }, "relationships": { "feed": { "data": { "id": "c97bdae6-b5d1-4966-b9f3-615e29d4d47d", "type": "feed" } } } }, ] } Search Search is just implemented with Full Text Search Postgres feature. I used the pg_search Gem, which can be used in two ways: Multi Search: Search across multiple models and return a single array of results. Imagine having three models: Product, Brand, and Review. Using Multi Search we could search across all of them at the same time, seeing a single set of search results. This would be perfect for adding federated search functionality to your app. Search Scope: Search within a single model, but with greater flexibility. execute <<-SQL ALTER TABLE entries ADD COLUMN searchable tsvector GENERATED ALWAYS AS ( setweight(to_tsvector('simple', coalesce(title, '')), 'A') || setweight(to_tsvector('simple', coalesce(body,'')), 'B') || setweight(to_tsvector('simple', coalesce(url,'')), 'C') ) STORED; SQL Feed Rank Feed Ranking is provided by openrank a free root domain authority metric based on the common search pagerank dataset. The value is normilized by ((Math.log10(domain_rank) / Math.log10(100)) * 100).round Machine Learning Machine Learning is provided by dandelion API Semantic Text Analytics as a service, from text to actionable data. Extract meaning from unstructured text and put it in context with a simple API. Add Feed At the moment is not possible to add source Feed if you have feed proposals open an issue with the URLs to add https://github.com/dato-ai/dato.rss/issues Wiki All API documentation is in the Wiki section. Feel free to make it better, of course. https://github.com/dato-ai/dato.rss/wiki To use some features such as adding a new feed you need a token with write permission. Currently only I can enable it. In case contact me Built With * Ruby on Rails -- Our back end API is a Rails app. It responds to requests RESTfully in JSON. * PostgreSQL -- Our main data store is in Postgres. * Redis -- We use Redis as a cache and for transient data. * Feedjira -- Feedjira is a Ruby library designed to parse feeds. * Dandelion -- Semantic Text Analytics as a service. * Sidekiq -- Simple, efficient background processing for Ruby. * JSON:API Serialization -- A fast JSON:API serializer for Ruby Objects.. * PgSearch -- PgSearch builds named scopes that take advantage of PostgreSQL's full text search. * TailwindCSS -- A utility-first CSS framework for rapidly building custom user interfaces. Plus lots of Ruby Gems, a complete list of which is at /main/Gemfile. Sponsor me If you want to support me in server costs to keep dato.ess free and up, consider sponsorize! Thanks! GitHub sponsor Contributing Bug reports and pull requests are welcome on GitHub at https:// github.com/davidesantangelo/dato.rss. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct. License The gem is available as open source under the terms of the MIT License. About The best RSS Search experience you can find datorss.com Topics ruby rails rss search-engine machine-learning rest-api postgresql rss-feed searching tailwindcss Resources Readme License MIT license Stars 483 stars Watchers 11 watching Forks 14 forks Releases 1 0.1 (woody) Latest Feb 19, 2021 Sponsor this project * * https://www.buymeacoffee.com/582rhJH Learn more about GitHub Sponsors Packages 0 No packages published Contributors 3 * @davidesantangelo davidesantangelo Davide Santangelo * @dependabot[bot] dependabot[bot] Languages * Ruby 72.1% * HTML 23.1% * JavaScript 3.4% * Other 1.4% Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.