https://github.com/vectordotdev/vector Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} vectordotdev / vector Public * Notifications * Fork 1.4k * Star 16.1k * A high-performance observability data pipeline. vector.dev License MPL-2.0, Unknown licenses found Licenses found MPL-2.0 LICENSE Unknown license-tool.toml 16.1k stars 1.4k forks Branches Tags Activity Star Notifications * Code * Issues 1.7k * Pull requests 100 * Discussions * Actions * Wiki * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Wiki * Security * Insights vectordotdev/vector This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 11,225 Commits .cargo .cargo .config .config .github .github benches benches changelog.d changelog.d config config cue.mod cue.mod distribution distribution docs docs lib lib licenses licenses proto proto regression regression rfcs rfcs rust-doc rust-doc scripts scripts src src testing testing tests tests tilt tilt vdev vdev website website workload-checks workload-checks .dockerignore .dockerignore .editorconfig .editorconfig .gitattributes .gitattributes .gitignore .gitignore .graphqlconfig .graphqlconfig .ruby-version .ruby-version .rustfmt.toml .rustfmt.toml CHANGELOG.md CHANGELOG.md CODE_OF_CONDUCT.md CODE_OF_CONDUCT.md CONTRIBUTING.md CONTRIBUTING.md Cargo.lock Cargo.lock Cargo.toml Cargo.toml Cross.toml Cross.toml LICENSE LICENSE LICENSE-3rdparty.csv LICENSE-3rdparty.csv Makefile Makefile NOTICE NOTICE PRIVACY.md PRIVACY.md README.md README.md RELEASES.md RELEASES.md SECURITY.md SECURITY.md STYLE.md STYLE.md Tiltfile Tiltfile VERSIONING.md VERSIONING.md buf.work.yaml buf.work.yaml build.rs build.rs clippy.toml clippy.toml deny.toml deny.toml license-tool.toml license-tool.toml rust-toolchain.toml rust-toolchain.toml View all files Repository files navigation * README * Code of conduct * MPL-2.0 license * License * Security Quickstart * Docs * Guides * Integrations * Chat * Download * Rust Crate Docs Vector What is Vector? Vector is a high-performance, end-to-end (agent & aggregator) observability data pipeline that puts you in control of your observability data. Collect, transform, and route all your logs and metrics to any vendors you want today and any other vendors you may want tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where it is most convenient for your vendors. Additionally, it is open source and up to 10x faster than every alternative in the space. To get started, follow our quickstart guide or install Vector. Principles * Reliable - Built in Rust, Vector's primary design goal is reliability. * End-to-end - Deploys as an agent or aggregator. Vector is a complete platform. * Unified - Logs, metrics (beta), and traces (coming soon). One tool for all of your data. Use cases * Reduce total observability costs. * Transition vendors without disrupting workflows. * Enhance data quality and improve insights. * Consolidate agents and eliminate agent fatigue. * Improve overall observability performance and reliability. Community * Vector is relied on by startups and enterprises like Atlassian, T-Mobile, Comcast, Zendesk, Discord, Fastly, CVS, Trivago, Tuple, Douban, Visa, Mambu, Blockfi, Claranet, Instacart, Forcepoint, and many more. * Vector is downloaded over 100,000 times per day. * Vector's largest user processes over 30TB daily. * Vector has over 100 contributors and growing. Documentation About * Concepts * Under the hood + Architecture - data model (log, metric), pipeline model, concurrency model, runtime model + Networking - ARC + Guarantees Setup * Quickstart * Installation - operating systems, package managers, platforms ( Kubernetes), manual * Deployment - roles, topologies Reference * Configuration + Sources - docker_logs, file, http, journald, kafka, socket, and many more... + Transforms - dedupe, filter, [geoip][docs.transforms.geoip], log_to_metric, lua, remap, and many more... + Sinks - aws_cloudwatch_logs, aws_s3, clickhouse, elasticsearch, gcp_cloud_storage, and many more... + Unit tests * Remap Language * API * CLI Administration * Management * Monitoring & observing * Upgrading * Validating Resources * Community - chat, calendar, @vectordotdev * Releases * Policies - Code of Conduct, Privacy, Releases, Security, Versioning Comparisons Performance The following performance tests demonstrate baseline performance between common protocols with the exception of the Regex Parsing test. Test Vector Filebeat FluentBit FluentD Logstash SplunkUF SplunkHF TCP to 86mib/s n/a 64.4mib/s 27.7mib 40.6mib/ n/a n/a Blackhole /s s File to 76.7mib 7.8mib/s 35mib/s 26.1mib 3.1mib/s 40.1mib/ 39mib/s TCP /s /s s Regex 13.2mib n/a 20.5mib/s 2.6mib/ 4.6mib/s n/a 7.8mib/s Parsing /s s TCP to 26.7mib n/a 19.6mib/s <1mib/s 2.7mib/s n/a n/a HTTP /s TCP to 69.9mib 5mib/s 67.1mib/s 3.9mib/ 10mib/s 70.4mib/ 7.6mib/s TCP /s s s To learn more about our performance tests, please see the Vector test harness. Correctness The following correctness tests are not exhaustive, but they demonstrate fundamental differences in quality and attention to detail: Test Vector Filebeat FluentBit FluentD Logstash Splunk Splunk UF HF Disk Buffer [?] Persistence File Rotate (create) File Rotate (copytruncate) File Truncation Process [?] (SIGHUP) JSON (wrapped) To learn more about our correctness tests, please see the Vector test harness. Features Vector is an end-to-end, unified, open data platform. Vector Beats Fluentbit Fluentd Logstash Splunk Splunk Telegraf UF HF End-to-end Agent Aggregator Unified Logs Metrics [?] [?] [?] [?] [?] [?] Traces Open Open-source Vendor-neutral Reliability Memory-safe Delivery guarantees Multi-core [?] = Not interoperable, metrics are represented as structured logs --------------------------------------------------------------------- Developed with [?] by Datadog - Security Policy - Privacy Policy About A high-performance observability data pipeline. vector.dev Topics rust parser events router pipeline metrics vector logs stream-processing forwarder observability Resources Readme License MPL-2.0, Unknown licenses found Licenses found MPL-2.0 LICENSE Unknown license-tool.toml Code of conduct Code of conduct Security policy Security policy Activity Custom properties Stars 16.1k stars Watchers 146 watching Forks 1.4k forks Report repository Releases 75 v0.36.1 Latest Mar 11, 2024 + 74 releases Contributors 404 * @dependabot[bot] * @jszwedko * @binarylogic * @bruceg * @spencergilbert * @lukesteensen * @StephenWakely * @neuronull * @LucioFranco * @ktff * @blt * @JeanMertz * @fanatid * @fuchsnj + 390 contributors Languages * Rust 60.4% * CUE 36.6% * HTML 1.3% * TypeScript 0.8% * JavaScript 0.3% * Shell 0.1% * Other 0.5% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.