https://github.com/adithya-s-k/omniparse Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions By size + Enterprise + Teams + Startups By industry + Healthcare + Financial services + Manufacturing By use case + CI/CD & Automation + DevOps + DevSecOps * Resources Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} adithya-s-k / omniparse Public * Notifications You must be signed in to change notification settings * Fork 52 * Star 1.1k Ingest, parse, and optimize any data format [?] from documents to multimedia [?] for enhanced compatibility with GenAI frameworks docs.cognitivelab.in License GPL-3.0 license 1.1k stars 52 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 1 * Pull requests 0 * Discussions * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights adithya-s-k/omniparse This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit message Last commit date Latest commit History 106 Commits .github .github docs docs examples examples omniparse omniparse python-sdk python-sdk .gitignore .gitignore Dockerfile Dockerfile LICENSE LICENSE README.md README.md pyproject.toml pyproject.toml server.py server.py View all files Repository files navigation * README * Code of conduct * GPL-3.0 license OmniParse OmniParse GitHub Stars GitHub Forks GitHub Issues GitHub Pull Requests License Important OmniParse is a platform that ingests and parses any unstructured data into structured, actionable data optimized for GenAI (LLM) applications. Whether working with documents, tables, images, videos, audio files, or web pages, OmniParse prepares your data to be clean, structured, and ready for AI applications, such as RAG, fine-tuning, and more. Try it out Open In Colab Features Completely local, no external APIs Fits in a T4 GPU Supports ~20 file types Convert documents, multimedia, and web pages to high-quality structured markdown Table extraction, image extraction/captioning, audio/video transcription, web page crawling Easily deployable using Docker and Skypilot Colab friendly Interative UI powered by Gradio \ Problem Statement It's challenging to process data as it comes in different shapes and sizes. OmniParse aims to be an ingestion/parsing platform where you can ingest any type of data, such as documents, images, audio, video, and web content, and get the most structured and actionable output that is GenAI (LLM) friendly. Installation Note: The server only works on Linux-based systems. This is due to certain dependencies and system-specific configurations that are not compatible with Windows or macOS. To install OmniParse, you can use pip: git clone https://github.com/adithya-s-k/omniparse cd omniparse Create a Virtual Environment: conda create --name omniparse-venv python=3.10 conda activate omniparse-venv Install Dependencies: poetry install # or pip install -e . [?] Docker To use OmniParse with Docker, execute the following commands: 1. Pull the OmniParse API Docker image from Docker Hub: 2. Run the Docker container, exposing port 8000: Docker Image docker pull savatar101/omniparse:0.1 # if you are running on a gpu docker run --gpus all -p 8000:8000 savatar101/omniparse:0.1 # else docker run -p 8000:8000 savatar101/omniparse:0.1 Alternatively, if you prefer to build the Docker image locally: Then, run the Docker container as follows: docker build -t omniparse . # if you are running on a gpu docker run --gpus all -p 8000:8000 omniparse # else docker run -p 8000:8000 omniparse Usage Run the Server: python server.py --host 0.0.0.0 --port 8000 --documents --media --web * --documents: Load in all the models that help you parse and ingest documents (Surya OCR series of models and Florence-2). * --media: Load in Whisper model to transcribe audio and video files. * --web: Set up selenium crawler. Supported Data Types Type Supported Extensions Documents .doc, .docx, .pdf, .ppt, .pptx Images .png, .jpg, .jpeg, .tiff, .bmp, .heic Video .mp4, .mkv, .avi, .mov Audio .mp3, .wav, .aac Web dynamic webpages, http://.com API Endpoints Client library compatible with Langchain, llamaindex, and haystack integrations coming soon. * API Endpoints + Document Parsing o Parse Any Document o Parse PDF o Parse PowerPoint o Parse Word Document + Media Parsing o Parse Any Media o Parse Image o Process Image o Parse Video o Parse Audio + Website Parsing o Parse Website Document Parsing Parse Any Document Endpoint: /parse_document Method: POST Parses PDF, PowerPoint, or Word documents. Curl command: curl -X POST -F "file=@/path/to/document" http://localhost:8000/parse_document Parse PDF Endpoint: /parse_document/pdf Method: POST Parses PDF documents. Curl command: curl -X POST -F "file=@/path/to/document.pdf" http://localhost:8000/parse_document/pdf Parse PowerPoint Endpoint: /parse_document/ppt Method: POST Parses PowerPoint presentations. Curl command: curl -X POST -F "file=@/path/to/presentation.ppt" http://localhost:8000/parse_document/ppt Parse Word Document Endpoint: /parse_document/docs Method: POST Parses Word documents. Curl command: curl -X POST -F "file=@/path/to/document.docx" http://localhost:8000/parse_document/docs Media Parsing Parse Image Endpoint: /parse_image/image Method: POST Parses image files (PNG, JPEG, JPG, TIFF, WEBP). Curl command: curl -X POST -F "file=@/path/to/image.jpg" http://localhost:8000/parse_media/image Process Image Endpoint: /parse_image/process_image Method: POST Processes an image with a specific task. Possible task inputs: OCR | OCR with Region | Caption | Detailed Caption | More Detailed Caption | Object Detection | Dense Region Caption | Region Proposal Curl command: curl -X POST -F "image=@/path/to/image.jpg" -F "task=Caption" -F "prompt=Optional prompt" http://localhost:8000/parse_media/process_image Arguments: * image: The image file * task: The processing task (e.g., Caption, Object Detection) * prompt: Optional prompt for certain tasks Parse Video Endpoint: /parse_media/video Method: POST Parses video files (MP4, AVI, MOV, MKV). Curl command: curl -X POST -F "file=@/path/to/video.mp4" http://localhost:8000/parse_media/video Parse Audio Endpoint: /parse_media/audio Method: POST Parses audio files (MP3, WAV, FLAC). Curl command: curl -X POST -F "file=@/path/to/audio.mp3" http://localhost:8000/parse_media/audio Website Parsing Parse Website Endpoint: /parse_website Method: POST Parses a website given its URL. Curl command: curl -X POST -H "Content-Type: application/json" -d '{"url": "https://example.com"}' http://localhost:8000/parse_website Arguments: * url: The URL of the website to parse Coming Soon/ RoadMap LlamaIndex | Langchain | Haystack integrations coming soon Batch processing data Dynamic chunking and structured data extraction based on specified Schema [?] One magic API: just feed in your file prompt what you want, and we will take care of the rest Dynamic model selection and support for external APIs Batch processing for handling multiple files at once New open-source model to replace Surya OCR and Marker Final goal: replace all the different models currently being used with a single MultiModel Model to parse any type of data and get the data you need. License OmniParse is licensed under the GPL-3.0 license. See LICENSE for more information. Acknowledgements This project builds upon the remarkable Marker project created by Vik Paruchuri. We express our gratitude for the inspiration and foundation provided by this project. Special thanks to Surya-OCR and Texify for the OCR models extensively used in this project, and to Crawl4AI for their contributions. Models being used: * Surya OCR, Detect, Layout, Order, and Texify * Florence-2 base * Whisper Small Thank you to the authors for their contributions to these models. --------------------------------------------------------------------- Contact Star History Chart For any inquiries, please contact us at adithyaskolavi@gmail.com About Ingest, parse, and optimize any data format [?] from documents to multimedia [?] for enhanced compatibility with GenAI frameworks docs.cognitivelab.in Topics ocr parser-library web-crawler parse-server whisper-api ingestion-api vision-transformer omniparser Resources Readme License GPL-3.0 license Code of conduct Code of conduct Activity Stars 1.1k stars Watchers 11 watching Forks 52 forks Report repository Releases No releases published Packages 0 No packages published Contributors 3 * * * Languages * Python 98.4% * Dockerfile 1.6% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.