https://github.com/qaware/protocurl Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + For + Enterprise + Teams + Startups + Education + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} qaware / protocurl Public * Notifications * Fork 5 * Star 154 protoCURL is cURL for Protobuf: The command-line tool for interacting with Protobuf over HTTP REST endpoints using human-readable text formats License View license 154 stars 5 forks Star Notifications * Code * Issues 18 * Pull requests 1 * Actions * Security * Insights More * Code * Issues * Pull requests * Actions * Security * Insights qaware/protocurl This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 4 branches 34 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/q] Use Git or checkout with SVN using the web URL. [gh repo clone qaware] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @github-actions @GollyTicker github-actions[bot] and GollyTicker [AutoUpdate] Goreleaser v1.15.2 ( #103) ... 03e2481 Feb 14, 2023 [AutoUpdate] Goreleaser v1.15.2 (#103) Co-authored-by: GollyTicker 03e2481 Git stats * 299 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github dev doc release src test .dockerignore .gitattributes .gitignore EXAMPLES.md LICENSE.md README.md RELEASE.md TESTS.md template.goreleaser.yaml View code [ ] protoCURL Why? Install Native CLI Docker Quick Start Usage and Example Protobuf JSON Format Protobuf Text Format Maintainer Development Setup Updating Docs after changes Enhancements and Bugs FAQ README.md protoCURL Tests Status GitHub Release (latest SemVer) DockerHub Version (latest semver) Pull Requests Pull Requests (closed) protoCURL is cURL for Protobuf: The command-line tool for interacting with Protobuf over HTTP REST endpoints using human-readable text formats. Why? Every data interchange format, which is used between service boundaries, benefits from a command-line-tool enabling one to directly talk to a service. We do not want to use a full-blown programming language and all its libraries just for quick and simple requests. Such a tool is very useful during development and debugging. Text-based formats, like JSON and XML, can simply be used with tools like curl as the requests can be written by hand. However, since Protobuf is a binary-encoded format, writing these requests by hand is hard and cumbersome. Hence, protoCURL. protoCURL enables us to write requests in a human-readable textual-format while talking to a binary-encoded Protobuf over HTTP REST endpoint. Without protoCURL, one would either need to duplicate the HTTP REST endpoints returning equivalent JSON/XML responses - or one needs to write custom code and run it from the IDE just to send simple requests. The first option is technical debt at its finest - and the second one is less ergonomic than using protoCURL. For an introduction with an example: Read the intro Blogpost Install protocurl includes and uses a bundled protoc by default. It is recommended to install curl into PATH for configurable http requests. Otherwise protocurl will use a simple non-configurable fallback http implementation. Native CLI Archive 1. Download the latest release archive for your platform from https: //github.com/qaware/protocurl/releases 2. Extract the archive into a folder, e.g. /opt/protocurl. 3. Add symlink to the binary in the folder. e.g. ln -s /opt/ protocurl/bin/protocurl /usr/bin/protocurl Or add the binary folder /opt/protocurl/bin to your system-wide path. 4. Test it via protocurl -h Debian .deb package 1. Download the latest release .deb for your architecture from https://github.com/qaware/protocurl/releases 2. Install dependency curl: sudo apt install curl 3. Install sudo dpkg -i .deb 4. Test it via protocurl -h Alpine .apk package 1. Download the latest release .apk for your architecture from https://github.com/qaware/protocurl/releases 2. Install dependencies curl and gcompat: sudo apk add curl gcompat 3. Install sudo apk add --allow-untrusted .apk 4. Test it via protocurl -h Docker Simply run docker run -v "/path/to/proto/files:/proto" qaware/ protocurl . See Quick Start for how to use. Quick Start After installing protocurl a request is as simple as: protocurl -I test/proto -i ..HappyDayRequest -o ..HappyDayResponse \ -u http://localhost:8080/happy-day/verify -d "includeReason: true" where * -I test/proto points to the directory of protobuf files of your service + with docker one needs to instead mount the directory to / proto via -v $PWD/test/proto:/proto * -i ..HappyDayRequest and -o ..HappyDayResponse are Protobuf message types. The .. makes protocurl infer their full package paths. * -u http://localhost:8080/happy-day/verify is the url to the HTTP REST endpoint accepting and returning binary protobuf payloads + with docker one may additionally need --network host * -d "includeReason: true" is the protobuf payload in Protobuf Text or JSON Format Then protocurl will * encode the textual Protobuf message to a binary request payload * send the binary request to the HTTP REST endpoint (via curl, if possible) and receive the binary response payload * decode the binary response payload back to text and display it and produce the following output: =========================== Request Text =========================== >>> includeReason: true =========================== Response Text =========================== <<< isHappyDay: true reason: "Thursday is a Happy Day! " formattedDate: "Thu, 01 Jan 1970 00:00:00 GMT" See below for usage notes and EXAMPLES.md for more information. Usage and Example See usage notes, EXAMPLES.md as well as the intro Blogpost. Protobuf JSON Format protoCURL supports the Protobuf JSON Format. Note, that the JSON format is not a straightforward 1:1 mapping as it is in the case of the Protobuf Text Format (described below). For instance, the JSON mapping for timestamp.proto uses a human-readable string representation, whereas the payload itself and the text format use a representation with seconds and nanoseconds. Protobuf Text Format Aside from JSON, Protobuf primarily and natively supports a text format which represents the fields 1:1 like in the wire-format. For instance, repeated fields are condensed into an array in the JSON format - whereas they are simply ' repeated' without an array type in the text format. For instance, for the following .proto file syntax = "proto3"; import "google/protobuf/timestamp.proto"; message HappyDayRequest { google.protobuf.Timestamp date = 1; bool includeReason = 2; double myDouble = 3; int64 myInt64 = 5; repeated string myString = 6; repeated NestedMessage messages = 9; } message NestedMessage { Foo fooEnum = 1; repeated int32 i = 4; } enum Foo { BAR = 0; BAZ = 1; } a HappyDayRequest message in text format might look like this: includeReason: true, myInt64: 123123123123, myString: "hello world" myString: 'single quotes are also possible' myDouble: 123.456 messages: { fooEnum: BAR, i: 0, i: 1, i: 1337 }, messages: { i: 15, fooEnum: BAZ, i: -1337 }, messages: { }, date: { seconds: 123, nanos: 321 } In summary: * No encapsulating { ... } are used for the top level message (in contrast to JSON). * fields are comma separated and described via : . + Strictly speaking, the commas are optional and whitespace is sufficient * repeated fields are simply repeated multiple times (instead of using an array) and they do not need to appear consecutively. * nested messages are described with { ... } opening a new context and describing their fields recursively * scalar values are describes similar to JSON. Single and double quotes are both possible for strings. * enum values are referenced by their name * built-in messages (such as google.protobuf.Timestamp are described just like user-defined custom messages via { ... } and their message fields The text format is defined in the Protobuf: Text Format Language Specification. Maintainer The project was created and is currently maintained by GollyTicker. qaware provided initial sponsorship for the project. Development For development it is recommended to use the a bash-like Terminal either natively (Linux, Mac) or via MinGW on Windows. About the CI/CD tests: TESTS.md How to make a release: RELEASE.md Setup * As for script utilities, one needs bash, jq, zip, unzip and curl. * One also needs to download the protoc binaries for the local development via release/10-get-protoc-binaries.sh. For development the generated-local.Dockerfile (via generate-local.Dockerfile.sh) is used. To build the image simply run source test/suite/setup.sh and then buildProtocurl Updating Docs after changes Generate the main docs (.md files etc.) in bash/WSL via doc/ generate-docs.sh . Once a pull request is ready, run this to generate updated docs. Enhancements and Bugs See issues. FAQ * How is protocurl different from grpccurl? grpccurl only works with gRPC services with corresponding endpoints. However, classic REST HTTP endpoints with binary Protobuf payloads are only possible with protocurl. * Why is the use of a runtime curl recommended with protocurl? curl is a simple, flexible and mature command line tool to interact with HTTP endpoints. In principle, we could simply use the HTTP implementation provided by the host programming language (Go) - and this is what we do if no curl was found in the PATH. However, as more people use protocurl, they will request for more features - leading to a feature creep in such a 'simple' tool as protocurl. We would like to avoid implementing the plentiful features which are necessary for a proper HTTP CLI tool, because HTTP can be complex. Since is essentially what curl already does, we recommend using curl and all advanced features are only possible with curl. * What are some nice features of protocurl? + The implementation is well tested with end-2-end approval tests (see TESTS.md). All features are tested based on their effect on the behavior/output. Furthermore, there are also a few cross-platform native CI tests running on Windows and MacOS runners. + The build and release process is optimised for minimal maintenance efforts. During release build, the latest versions of many dependencies are taken automatically (by looking up the release tags via the GitHub API). + The documentation and examples are generated via scripts and enable one to update the examples automatically rather than manually. The consistency of the outputs of the code with the checked in documentation is further tested in CI. About protoCURL is cURL for Protobuf: The command-line tool for interacting with Protobuf over HTTP REST endpoints using human-readable text formats Topics cli rest protobuf curl protocol-buffers rest-client Resources Readme License View license Stars 154 stars Watchers 5 watching Forks 5 forks Releases 38 v1.5.5 Latest Dec 14, 2022 + 37 releases Contributors 4 * @qa-swaneet-sahoo qa-swaneet-sahoo Swaneet Sahoo (QAware GmbH) * @GollyTicker GollyTicker Swaneet * @dependabot[bot] dependabot[bot] * @github-actions[bot] github-actions[bot] Languages * Go 54.3% * Shell 26.8% * TypeScript 9.8% * Dockerfile 5.0% * PowerShell 4.1% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.