https://github.com/transform-data/metricflow Skip to content Sign up * Product + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Learning Lab + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} transform-data / metricflow Public * Notifications * Fork 12 * Star 98 MetricFlow allows you to define, build, and maintain metrics in code. transform.co/metricflow AGPL-3.0 License 98 stars 12 forks Star Notifications * Code * Issues 1 * Pull requests 1 * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 2 branches 1 tag Code Latest commit @marcodamore marcodamore Merge pull request #6 from transform-data/ marco_fix_pypi_workflow ... 31958c8 Apr 6, 2022 Merge pull request #6 from transform-data/marco_fix_pypi_workflow fix pypi workflow by giving it access to the Pypi Publish environment secrets 31958c8 Git stats * 14 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github fix pypi workflow by giving it access to the Pypi Publish environment... Apr 6, 2022 assets Initial commit. Apr 6, 2022 metricflow Add a few more examples to the tutorial (#1) Apr 6, 2022 .gitignore Initial commit. Apr 6, 2022 .pre-commit-config.yaml Initial commit. Apr 6, 2022 CHANGELOG.md Initial commit. Apr 6, 2022 CONTRIBUTING.md Improve contributor guide (#3) Apr 6, 2022 LICENSE Initial commit. Apr 6, 2022 Makefile Initial commit. Apr 6, 2022 README.md fixes typo with --skip-dw Apr 6, 2022 TransformCLA.md Initial commit. Apr 6, 2022 attribution.md fixing commit sha Apr 6, 2022 poetry.lock Initial commit. Apr 6, 2022 pyproject.toml Add a few more examples to the tutorial (#1) Apr 6, 2022 tox.ini Initial commit. Apr 6, 2022 View code [ ] Welcome to MetricFlow Core Tenets Features Resources Documentation Getting Started Additional Resources Install MetricFlow Contributing and Code of Conduct License README.md [MetricFlow] Welcome to MetricFlow MetricFlow is a computational framework for building and maintaining consistent metric logic. The name comes from the approach taken to generate metrics. Using the user-defined semantic model, a query is first compiled into a metric dataflow plan. The plan is then converted to an abstract SQL object model, optimized, and rendered to engine-specific SQL. MetricFlow provides a set of abstractions that help you construct complicated logic and dynamically generate queries to handle: * Complex metric types such as ratio, expression, and cumulative * Multi-hop joins between fact and dimension sources * Metric aggregation to different time granularities * And so much more As a developer, you can also use MetricFlow's interfaces to construct APIs for integrations to bring metrics into downstream tools in your data stack. MetricFlow itself acts as a semantic layer, compiling the semantic information described in the MetricFlow spec to SQL that can be executed against the data warehouse and served to downstream applications. It acts as a proxy, translating metric requests in the form of "metrics by dimensions" into SQL queries that traverse the data warehouse and the underlying semantic structure to resolve every possible combination of metric and dimension. Core Tenets The framework relies on a set of core tenets: * DRY (Don't Repeat Yourself): This principle is the core objective of the underlying MetricFlow spec. Duplication of logic leads to incorrectly constructed metrics and should be avoided through thoughtfully-designed abstractions. * SQL-centric compilation: Metric logic should never be constructed in a black-box. This SQL-centric approach to metric construction means that metric logic remains broadly accessible and introspectable. * Maximal Flexibility: Construct any metric on any data model aggregated to any dimension. There are escape hatches, but we continually work to make them unnecessary. Features Key features of MetricFlow include: * Metrics as Code: MetricFlow's metric spec allows you to define a wide-range of metrics through a clean set of abstractions that encourage DRY expression of logic in YAML and SQL. * SQL Compilation: Generate SQL to build metrics, without the need to repeatedly express the same joins, aggregations, filters and expressions from your data warehouse in order to construct datasets for consumption. * DW Connectors: Support for data warehouse (DW) connectors give the open-source community the power to contribute to DW-specific optimizations and support. DW connectors allow users to construct metric logic to various data warehouses. * Command Line Interface (CLI): Pull data into a local context for testing and development workflows. * Python Library: Pull metrics into local Python environments such as Jupyter or other analytical interfaces. * Materializations: Define a set of metrics and a set of dimensions that you want to materialize to the data warehouse. This enables rapid construction of denormalized datasets back to the warehouse. * Accessible interfaces: Construct APIs and SDKs so you can pull metric datasets into downstream applications. + React Components to build embedded analytics + Airflow Operators to schedule API requests and pre-construction of metrics + GraphQL interface for end-users to build their own interfaces for metrics Resources Documentation MetricFlow documentation can be found on the Transform Documentation site. Getting Started If you're new to MetricFlow, we suggest that you review our Getting Started section and tutorial. To get started you can run the following commands: Install MetricFlow: If you do not have postgres on your machine, first install Postgres: * Postgres provides pre-built packages for download and installation * Mac users might prefer to use Homebrew: brew install postgresql Then, proceed with regular install as follows: pip install metricflow mf setup # Now you can connect your datawarehouse by modifying the parameters in {$HOME}/.metricflow/config.yml mf health-checks # confirm your datawarehouse connection is working Run the tutorial: mf tutorial # optionally add `--skip-dw` if you have already confirmed your datawarehouse connection works For reference, the tutorial steps are below: Please run the following steps, 1. In '{$HOME}/.metricflow/config.yml', `model_path` should be '{$HOME}/.metricflow/sample_models'. 2. Try validating your data model: `mf validate-configs` 3. Check out your metrics: `mf list-metrics` 4. Query your first metric: `mf query --metrics transactions --dimensions ds --order ds` 5. Show the SQL MetricFlow generates: `mf query --metrics transactions --dimensions ds --order ds --explain` 6. Add another dimension: `mf query --metrics transactions --dimensions ds,customer__country --order ds` 7. Add a higher date granularity: `mf query --metrics transactions --dimensions ds__week --order ds__week` 8. Try a more complicated query: `mf query --metrics transactions,transaction_usd_na,transaction_usd_na_l7d --dimensions ds,is_large --order ds --where "ds between '2022-03-20' and '2022-04-01'"` 9. For more ways to interact with the sample models, go to 'https://docs.transform.co/docs/metricflow/metricflow-tutorial'. 10. Once you're done, run `mf tutorial --skip-dw --drop-tables` to drop the sample tables. Additional Resources * Website * Documentation * Slack Community * MetricFlow Git Repository * CHANGELOG.md Install MetricFlow MetricFlow can be installed from PyPi for use as a Python library with the following command: pip install metricflow Once installed, MetricFlow can be setup and connected to a data warehouse by following the instructions after issuing the command: mf setup To see what MetricFlow can do without custom configurations, start the tutorial by running: mf tutorial To get up and running with your own metrics, you should rely on MetricFlow's documentation available at MetricFlow docs. Contributing and Code of Conduct This project will be a place where people can easily contribute high-quality updates in a supportive environment. You might wish to read our code of conduct and engineering practices before diving in. To get started on direct contributions, head on over to our contributor guide License MetricFlow is open source software. The project relies on several licenses including AGPL-3.0-or-later and Apache (specified at folder level). MetricFlow is built by Transform, the company behind the first metrics store. About MetricFlow allows you to define, build, and maintain metrics in code. transform.co/metricflow Topics data analytics metrics pypi business-intelligence data-modeling semantic-layer Resources Readme License AGPL-3.0 License Stars 98 stars Watchers 9 watching Forks 12 forks Releases 1 tags Packages 0 No packages published Contributors 14 * @allegraholland * @tlento * @belindabennett * @marcodamore * @nhandel * @WilliamDee * @andykram * @alliehowe * @plypaul * @robertabbott * @QMalcolm + 3 contributors Languages * Python 100.0% * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.