[HN Gopher] Convert curl commands to code in several languages
___________________________________________________________________
Convert curl commands to code in several languages
Author : djhaskin987
Score : 201 points
Date : 2022-03-11 16:50 UTC (6 hours ago)
(HTM) web link (curlconverter.com)
(TXT) w3m dump (curlconverter.com)
| capableweb wrote:
| I agree with the comments saying it should say "Language + Client
| library" instead of just language, as there are many ways of
| doing requests with different libraries.
|
| Also, funny enough, the first command I tried didn't work as I
| expected, but thinking about it, it also seems hard to implement
| correctly. The cURL command I tried was "curl -v
| https://google.com/" but none of the implementations include
| verbose logging.
|
| Weirdly, JSON is listed there, and it is absolutely not clear how
| JSON can be considered a language that can make HTTP requests. I
| wonder what client library that is?
| verhovsky wrote:
| The JSON output just dumps curlconverter's internal
| representation of a parsed curl command. You could use it if
| you want to use curlconverter to parse curl commands and then
| do stuff with that parsing yourself in a language other than
| JavaScript. You would install the command line tool then use
| your language's command-running library. For example, from
| Python: import subprocess import json
| result = subprocess.run(['curlconverter', '--language', 'json',
| 'example.com'], capture_output=True, text=True) command
| = json.loads(result.stdout) print(command)
| awsanswers wrote:
| I just noticed yesterday that Postman has functionality like
| this, with over 20 options for the language or language+framework
| used.
| detaro wrote:
| Seems to be mostly doing text-replacement and not actually
| understand what it's handling and what it can't actually do:
| https://github.com/icing/blog/blob/main/curl_on_a_weekend.md (not
| my article, saw the link on twitter today)
|
| E.g. it turns curl ftp://host.com into response =
| requests.get('http://ftp://host.com')
| chazeon wrote:
| I hope there is a better way to stringify / parse these headers
| and stuff; curl options seems be the best way there is right now.
| Unfortunately Insomnia etc. does not parse these.
| steerablesafe wrote:
| I expected C with libcurl on the list, but in a sense that might
| be the toughest to implement.
| tyree731 wrote:
| Maybe it's hard to implement, but it's implementation is a
| standard part of the curl tool:
|
| $ curl ---libcurl curl.c ...
| jandrese wrote:
| Seems like it should be pretty straightforward.
| curl_easy_init(), ..._setopt(), ..._perform(), ..._cleanup()
| covers the use cases I think.
|
| libcurl only gets weird when you're threading multiple fetches
| or integrating with a GUI toolkit or something. Even then it's
| not hard at all to integrate with GTK unless you need absolute
| peak performance or are handling tens of thousands of
| simultaneous connections.
| verhovsky wrote:
| Re-using curl's C code and compiling to WASM is the dream.
|
| libcurl does stuff like open files for reading, so you would
| have to at least modify the code to just pass the file name,
| you also have to modify the struct.
|
| libcurl parses an array of strings, but curlconverter gets a
| Bash AST (or a list of strings if you use it from the command
| line). When there's a bash variable in the command, we want
| to generate code that gets that environment variable at
| runtime, e.g. `os.environ['MY_VARIABLE']` in Python so the
| struct needs to store pointers to AST nodes instead of
| strings/booleans/ints. Though to be fair curlconverter
| doesn't work this way either (yet), we convert AST nodes to
| strings/booleans/ints after command parsing and store that,
| then when we're generating the code if we see a $ in a string
| we assume that was a variable in the input AST.
| verhovsky wrote:
| Sorry, it's too late to delete my comment but I realize now
| that you were asking about _generating_ C code, not which
| language the project should be implemented in. "Curl
| already has argument parsing code, why can't we just call
| it?" is something I've thought a lot so I assumed someone
| would also suggest it here.
| folli wrote:
| As a Java hobbyist, I'm dying seeing the difference between the
| Java boilerplate and the Python two-liner
| pcdavid wrote:
| This is not representattive of modern Java, which has much
| better APIs. Since Java 11: public
| HttpResponse<String> fetch(String url) { HttpClient
| client = HttpClient.newHttpClient(); HttpRequest
| request = HttpRequest.newBuilder(URI.create(url)).build();
| return client.send(request, BodyHandlers.ofString()); }
| hiram112 wrote:
| But of course, nobody uses the standard library since it came
| late to the game, and instead there are at least half a dozen
| popular HTTP libs commonly used (Apache, Jetty, Spring,
| OkHttp, Google, etc.). It's not too different from the
| logging clusterf*ck in Java, where there are now 9 ways to do
| it, and none of them are compatible with each other.
| dulvui wrote:
| Why should you do this in Java:
|
| httpConn.getResponseCode() / 100 == 2
|
| instead of:
|
| httpConn.getResponseCode() == 200
| 4khilles wrote:
| 204 satisfies the first check but not the second.
| willcipriano wrote:
| If you intend to do anything with the response you'd probably
| handle 204 differently otherwise you'll get the world famous
| null pointer.
| detaro wrote:
| I guess so it accepts all 2xx codes and not just 200?
| dulvui wrote:
| Oh now I understand: its to match all 200 HTTP codes like 201,
| 202, 203 etc.
| [deleted]
| dragonwriter wrote:
| Because all 2xx responses are success; for simple (not Range or
| other special request scenarios) GET/HEAD it probably matters
| less in practice than for PUT/POST/DELETE, but 203 and 205 (and
| 204 for an existing but empty resource) are still
| possibilities.
| dontchooseanick wrote:
| Where is haskell :) ?
| codazoda wrote:
| It's interesting and I like it. It does seem to be rather
| opinionated without saying so. The PHP example requires an http
| library I hadn't heard of (rmccue/requests). Copy/paste of that
| code doesn't work unless you know how to find and install that
| library. The related tool they reference, however, does work
| using the PHP curl library.
|
| I presume they were trying to keep the code to 4 lines or so. The
| curl library in PHP is rather verbose and requires 5 to 8 lines
| (depending on if you try to catch the error).
|
| https://www.php.net/manual/en/book.curl.php
| NickC_dev wrote:
| At the time we were using that particular library heavily at
| WizeHire. I had built the tool for personal convenience.
| rmccue/requests was likely one of the first "generators".
|
| I expect that we'd accept a PR for PHP curl library
| implementation. Having a non-third-party option would be
| valuable.
| lambic wrote:
| I feel like more granular options would be useful here like
| "Python with beautifulsoup" or "PHP with Guzzle".
| blueflow wrote:
| Why use Guzzle at all when you can use the PHP libcurl bindings
| directly? Guzzle is translating to libcurl either way...
| oefrha wrote:
| I use Paw[1] for this, which is obviously a lot more powerful by
| allowing me to easily toggle and/or modify query params, headers,
| edit body as structured JSON / XML / application/x-www-form-
| urlencoded / multipart/form-data, etc. Probably doable in other
| API clients like Insomnia[2], too.
|
| [1] https://paw.cloud/
|
| [2] https://insomnia.rest/
| reese_john wrote:
| Postman enables you to convert your HTTP requests to several
| languages as well.
| [deleted]
| croes wrote:
| JSON?
| verhovsky wrote:
| I work on this tool (but I'm not the original author), feel free
| to ask me questions.
| hanniabu wrote:
| I'm surprised it doesn't support Javascript
| [deleted]
| account-5 wrote:
| I'm struggling to get VBA to accept a .pem client cert as well as
| a username and password. Wish this has VBA.
| cushychicken wrote:
| Love curl converter!
|
| They even have some resources to let you turn into python, go,
| and a few other languages.
| jraph wrote:
| With the feature of browsers to copy requests as Curl and now
| this, it's funny to see curl commands be(com)ing some kind of
| lingua franca for HTTP request representation!
|
| Now you can play a request and boom, you have your code to make
| this request in your favorite programming language.
| spacemanmatt wrote:
| I've gotten a lot out of Chrome debugger's "copy request as cURL"
| (among others) functionality. I gotta like seeing more of this
| kind of thing in the wild.
| flatiron wrote:
| If you have postman you can import curl commands and export
| them with a lot more options.
|
| That being said turn off all tracking in postman. I got a
| creepy sales email once asking about my usage and they had a
| lot of details that made me feel creepy.
| protoman3000 wrote:
| Anybody else having reached the point of language
| indifference/boredom?
|
| Everything I everywhere now see is just the same: Gluecode,
| library calls, unnecessary verbosity, Blackbox inside Blackbox
| etc.
|
| And this project slaps it again into my face, with every example
| it's literally always the same - only that language uses a colon,
| the other a semicolon, the other catches exceptions with ? etc.
| but in a way it's all just the same.
|
| How to overcome this?
|
| I find, only pointfree notation has some beauty to it that makes
| reading code sweet again.
| makeworld wrote:
| Maybe try programming for Uxn.
|
| https://100r.co/site/uxn.html
| SnooSux wrote:
| Instead of exploring different languages learn about different
| fields and domains. The language may be basically the same
| (outside of some more esoteric ones), but Data Science is a
| very different experience from Web Dev which is different from
| Embedded Systems. Get out of your comfort zone a little.
| capableweb wrote:
| > How to overcome this?
|
| You start with branching out to _actually different_
| programming languages. If you already know PHP, don 't go and
| learn Java or C#, you basically already know those from PHP,
| you're only missing knowledge of the public API (what methods
| should be called when, naming conventions) and so on.
|
| Instead, seek languages that when you see them the first time
| you say "wow, I don't understand anything" and sit down and try
| to learn one of those. For me, Clojure was always one of those
| languages that kindled the "what the hell is that" feeling in
| me, but after reading a lot of praise for the language, I
| started learning it anyways.
|
| Now I work on Clojure(Script) code full-time, and programming
| is finally a fun thing to do again.
|
| I won't try to learn Common Lisp or anything like that next, as
| I know that if I spent the time, it'd be easy, it's just
| Clojure but with different functions to call, different
| conventions, but mostly the same. So the next language I'm
| targeting is Rust. A C-like language, but new concepts around
| the borrow-checker and lifetimes, something I've been exposed
| to before, but not this literally.
| [deleted]
| conrs wrote:
| Work on solving problems using declarative languages and
| functional languages.
|
| So, either Prolog, or Lisp.
|
| Structure and Interpretation of Computer Programs is a good
| jumping off point for Lisp while also providing fundamental
| insights.
|
| Prolog.. honestly, just google "basic" array problems and try
| and do them in prolog. It takes a mental shift, which is hard
| at first, and you feel silly not being able to count to 10. But
| learning how to think in this paradigm is useful - e.g. SQL is
| declarative.
| DonHopkins wrote:
| >I find, only pointfree notation has some beauty to it that
| makes reading code sweet again.
|
| I haven't written FORTH code in ages, but I still enjoy reading
| well written FORTH code for its pure literary pleasure!
|
| https://github.com/MitchBradley/openfirmware/blob/master/for...
| brimble wrote:
| I've found programming language differences less and less
| interesting the longer I do this. They all do the same thing.
| Everything's pointers and stacks, usually not that far beneath
| the surface. Some are more or less ergonomic, but nothing (that
| sees any widespread use, anyway) is _really_ very different.
| Just another set of quirks and (probably bad) tools to learn.
| Library availability & quality matters more to me than what
| the language looks like.
| superdisk wrote:
| I prescribe Prolog, Lisp, Forth, x86 assembly, and Smalltalk.
| Each one is remarkably different and not just a coat of paint
| on top of C (which is 80% of popular languages). You'll have so
| much fun.
| account-5 wrote:
| What about APL, or J? Would these be something different
| enough.
| skulk wrote:
| This project is a curiosity. It may help someone trying to
| build a quick one-off script (though, then I have the question
| of why curl wasn't good enough). In any real-world use-case,
| you can't just plop in some isolated code to make a HTTP
| request. I haven't looked yet, but I assume this website
| generates code that blocks the current thread or it spins up
| its own threadpool; neither of these things are acceptable for
| any production application.
|
| In my experience, the main concepts behind application
| architecture remains the same regardless of what language
| you're using. Concepts like asynchronous execution and how it
| affects composability and resource management transcend syntax.
| perth wrote:
| When you master the hammer you can use it to build stuff, it's
| not a competition to know every toy language in existence.
| protoman3000 wrote:
| > It's not a competition to know every toy language in
| existence.
|
| I would argue against this and claim that it appears to be a
| competition since "X written in {Rust,Go,SexyLang,...}"
| became the prevalent bullshit bingo.
| Swizec wrote:
| Juggling chainsaws is more impressive than hammers. But
| it's all just juggling
|
| The question you have to ask yourself is: Are you juggling
| for the sake of juggling or to make a good show for the
| crowd?
|
| If your goal is the outcome, maybe don't go into juggling
| bags of poo even though it's more challenging than
| chainsaws because they keep changing shape.
| djbusby wrote:
| Sure, however if you play a different game you can get good
| wage on $UNCOOL_LANG (is that Perl now or what?)
|
| Do you want to win bingo (chance) or get steady-pay?
|
| Prevalent is just what you're paying attention too - there
| are 359 other degrees to enjoy.
| revicon wrote:
| Why is language indifference bad? You're correct, most language
| differences really come down to one language using a colon, the
| other a semicolon, etc.
|
| Find a problem you feel passionate about solving and then use
| your skills to go solve it for mankind. You've learned how to
| be a mason, now go build something beautiful.
| po1nt wrote:
| I'm a bit dissapointed it doesn't convert into raw curl module
| function calls in PHP.
| Minor49er wrote:
| I came here to say this. It assumes that you have
| rmccue/requests installed, which is an odd choice since that
| library has 3% of the downloads that Guzzle has. Not to mention
| the native cURL module that I also expected to be used
| esnard wrote:
| Yeah, this is a really odd choice. I find
| https://incarnate.github.io/curl-to-php/ (found in the
| footer) way more interesting since it uses PHP's cURL lib
| NickC_dev wrote:
| Original author here. Many smart people have contributed code
| over the years, but one warrants special mention.
|
| About a year ago, verhovsky showed up out of nowhere. He rewrote
| the core of the application and increased the professionalism
| across the board. (dedicated domain, github page hosting, UI
| refresh, privacy improvements, and much more)
|
| The tree-sitter PR is a monster achievement:
| https://github.com/curlconverter/curlconverter/pull/278
|
| Search for parseAnsiCString in there. I don't think that had ever
| been implemented in JavaScript before.
|
| For you, verhovsky, 10x engineer might be an understatement.
| Thank you!
| DyslexicAtheist wrote:
| how about C ... after all curl is just a frontend to libcurl?
| re wrote:
| Check out the `--libcurl` argument to curl.
|
| https://everything.curl.dev/libcurl/libcurl
| akmittal wrote:
| This looks great, was working myself on same. Good that i saw the
| link without wasting anymore time.
|
| Had completed UI part as of now
| https://news.ycombinator.com/item?id=30642254
| eu wrote:
| I like `curlify` to convert from `python requests` to `curl`
| sethammons wrote:
| I haven't touched it in years, but I made a utility to do the
| opposite in Go. Start with Go's request and transform it into a
| curl call. It was helpful for testing at the time. Looking at it,
| wow, that was 7 years ago!
|
| https://github.com/sethgrid/gencurl
| racl101 wrote:
| That is very cool!
| benbristow wrote:
| Something a bit different but I'm a huge fan of NSwag recently.
|
| https://github.com/RicoSuter/NSwag
|
| We use it on a project at work in C# and I've also been using it
| personally on pet projects with TypeScript.
|
| Allows you to automatically generate REST API clients based on
| Swagger/OpenAPI specs and is very customizable.
|
| Really nice if building out microservices and you're wanting to
| communicate between each other or building web apps and want to
| communicate with APIs. Saves a lot of time writing code to make
| requests and handle different status codes etc. Automatically
| generates all the classes/interfaces with sane naming schemes
| etc.
|
| Even has a GUI application to generate the configs also if you
| wish.
| beranabus wrote:
| The go version seems to be missing timeouts. There is none by
| default and the --connect-timeout flag is not setting one either.
| johnx123-up wrote:
| Few other similar open source projects:
|
| 1. https://github.com/mholt/curl-to-go
|
| 2. https://github.com/incarnate/curl-to-php
| mholt wrote:
| Curl-to-Go also understands JSON payloads and will even
| generate the struct definition for you to fill in, so you
| aren't working with opaque JSON strings.
| skywal_l wrote:
| And if you look to their page you will find an even more
| complete list: curl-to-Go curl-to-PHP
| HAR-to-curl http-translator (Python, JS and JSON)
| uncurl (to Python) hrbrmstr/curlconverter (to R)
| curl-to-postman Insomnia Paw-cURLImporter
| [deleted]
| gscho wrote:
| Here's the one I often use for Ruby.
| https://jhawthorn.github.io/curl-to-ruby/
| kevinsundar wrote:
| I always thought this would be a cool feature to have in a http
| library. The library knows the best way to convert a curl command
| to its own internal representation. Something like
| requests.from_curl()
| tkiolp4 wrote:
| The amount of boilerplate Go code for a simple GET is
| astonishing. I love Go, but sometimes I miss the succinctness of
| Python.
| skrtskrt wrote:
| While Go is definitely more verbose, Python benefits from
| decades of development of common libraries (requests, attrs,
| click, SQLAlchemy, Django ORM/REST framework, etc) that have
| become the de facto standard - there is a LOT of code hidden
| behind those simple interfaces and the hyper-dynamic nature of
| "clever" Python libraries make them mind-crushingly hard to
| grok.
|
| I've developed in Python for >6 years, with tons of deep
| exploration of the common libraries, and been working in Go for
| like 3 months - in terms of my ability to understand them, I'll
| take most Go libraries over Python any day.
| skrtskrt wrote:
| Also note that the Python example naturally has no exception
| handling, while Go "has to" handle the errors (enforced by
| convention and linters, you could always just throw them
| away)
|
| also it's only one more line, but they didn't even bother to
| print the response
|
| once you look at the fact that Go has to have `func main()`
| declared, the response printing, and the error handling, Go
| only has a couple more lines than Python
| debdut wrote:
| Awesome work! I'm amazed at how many low hanging easy but really
| really useful projects are left to be done! It seems infinite
___________________________________________________________________
(page generated 2022-03-11 23:00 UTC)