[HN Gopher] Show HN: API Parrot - Automatically Reverse Engineer...
___________________________________________________________________
Show HN: API Parrot - Automatically Reverse Engineer HTTP APIs
When automating business processes at work, I found it difficult
and time-consuming to reverse engineer business systems' APIs. I
often had to manually reverse engineer APIs using developer tools
or settle for less optimal technologies such as Robotic Process
Automation (RPA). Often, the issue is that it can be hard to
resolve all the cookies, access tokens, and other elements required
to successfully execute the requests. Manually trying to resolve
these dependencies using developer tools is especially challenging
with multiple requests where data is stored in JavaScript objects
or HTML elements. To try to solve this issue, I built a tool
called API Parrot that automatically identifies the data
correlations between requests and builds a graphical representation
of the flow to give users a better understanding. To streamline the
process, I also included functionality to record requests, define
your own inputs and outputs, and export the entire flow--or parts
of it--as JavaScript code. The application is Electron-based and
currently compiled for Windows and Linux. Please try it out and
give feedback! Online Tutorial: A simple example of reverse
engineering the USPS API is available at
https://docs.apiparrot.com/docs/category/tutorial---reverse-...
Author : pvarghav
Score : 250 points
Date : 2025-01-01 13:15 UTC (9 hours ago)
(HTM) web link (apiparrot.com)
(TXT) w3m dump (apiparrot.com)
| vhayda wrote:
| Nice! It needs some refinement and a macOS version.
| tommiegannert wrote:
| Could you give some examples of what refinement you think it
| needs?
| davide_v wrote:
| Nice, I was looking for something like this. I tried it on Ubuntu
| but after clicking Capture requests > Launch Chrome, nothing
| happens.
| ashenke wrote:
| Yep same problem
| chompin wrote:
| Same issue, would prefer the option to use any browser also.
| Chrome is not my cup of tea
| 7357 wrote:
| Looks like it wants to run chrome using `start chrome` which is
| AFAIK a Windows-only command.
| pvarghav wrote:
| Thank you for pointing this out. I've addressed the issue, and
| it should now be fixed in version 0.2.1, which is available for
| download on the website. Please update to the latest version,
| and let me know if you encounter any more problems.
| brushfoot wrote:
| Impressive project. I was curious how it discovers data
| relationships and was going to check the repo, but it looks like
| there's no code, only issues and releases. Is that right?
|
| Which leads me to...
|
| - Is this closed source?
|
| - Does it cost money?
|
| - How _does_ it discover data relationships?
| skeptrune wrote:
| It's entertaining that Github has become such a common place to
| find information that even closed source projects put
| _something_ up there
| pvarghav wrote:
| Thanks for your interest!
|
| - Is this closed source?
|
| Currently, the code is not open source, but I might open-source
| parts of it in the future.
|
| - Does it cost money?
|
| The software is free to use. If there is demand, I might create
| a "pro" version for businesses in the future. However, I intend
| to always have a free version available for individuals.
|
| - How does it discover data relationships?
|
| I've discussed how it discovers data relationships in the
| documentation here: https://docs.apiparrot.com/docs/tutorial-
| extras/exchange-mod....
|
| In short, the tool breaks down the data in the requests and
| responses into smaller parts by identifying their formats. For
| example, `["foo", "bar"]` would be recognized as a JSON array
| and broken down into the elements `"foo"` and `"bar"`. By
| applying this method recursively, you build a tree-like
| structure of the data.
|
| If an exact match is found between data in a response from a
| previous request and data in a subsequent request, a
| correlation is detected.
|
| Please feel free to ask if you have any more questions!
| urronglol wrote:
| Sounds like it has little utility in the real world.
| bjt12345 wrote:
| If this can save me time at work, I'd be happy to throw some
| money at it.
|
| My bosses OTOH...let's just say, there's no penalty within
| companies for pointy haired bosses not making decisions to
| purchase something like this and ignoring staff.
|
| It's a false economy but I'm tired of it and just purchase
| what I can afford.
| victor106 wrote:
| Looks great, but no Mac app?
| gtirloni wrote:
| Is there a ToS/License somewhere?
| enricotal wrote:
| Fantastic Tool ... Mac version is paramount
| yellow_lead wrote:
| Hi, it seems youve spelled reverse wrong
|
| > API Parrot is the tool specifically designed to reverese
| engineer the HTTP APIs of any website.
| pvarghav wrote:
| Thanks for pointing this out!
|
| It should now be fixed.
| yawndex wrote:
| Any current plans for a macOS release?
| pvarghav wrote:
| Yes, I plan to release a macOS version of API Parrot.
| Unfortunately, I currently don't own a Mac, and since building
| macOS applications requires one, this has delayed the release.
| I'm actively exploring solutions, such as accessing a Mac
| environment remotely or acquiring the necessary hardware.
| devops000 wrote:
| Feedback: add a newsletter form to get notification when you will
| release the MacOSX version
| toomuchtodo wrote:
| +1, ready to buy
| pvarghav wrote:
| Thank you for your suggestion!
|
| I've added a newsletter sign-up form at the bottom of the
| webpage: https://apiparrot.com/#newsletter
|
| Feel free to subscribe to receive notifications when we release
| the MacOSX version.
| setheron wrote:
| Very sad half the comments are asking for MacOS app. The rise of
| development on MacOS for server development when the final target
| is Linux will cause long term harm to the newer generation of
| engineers
| yoavmmn wrote:
| Nowadays everything runs on docker anyway
| setheron wrote:
| You'd never see a Windows developer work in MacOS or a iOS
| developer work in Linux but Linux developers (server side)
| routinely work in MacOS
|
| Unnecessary abstraction
| quesera wrote:
| Counter-argument: it could be risky to dev on and deploy to
| a single monoculture.
|
| But empirically, I've been developing on macOS (etc) and
| Linux (often simultaneously), and deploying to Linux
| (Debian, RHEL/AL), Solaris (etc), and FreeBSD ... for more
| than 20 years.
|
| Aside from package management tooling differences, package
| naming, and package content splits (e.g. pkg vs pkg-dev) --
| all of which are equally inconsistent between Linux distros
| -- I cannot recall a single issue caused by this
| heterogeneity.
| prophesi wrote:
| > iOS developer work in Linux
|
| I dream of the day Apple releases official docker images.
| Building for iOS is the only reason I have to touch a Mac.
| cellwebb wrote:
| All SNES games should have been developed in Mario Paint or
| it was an unnecessary abstraction
| cdaringe wrote:
| Not sad at all! Mac has excellent hardware, excellent
| reliability, excellent day to day performance. Im not a fanboy,
| but it won for (IMHO) clear and obvious reasons. Of course
| folks want a mac app. No comment on the "harm" bit.
| bearjaws wrote:
| It is always amazing to me people who will chastise people
| for using Macs.
|
| It is by far the most robust hardware and _15 years_ later
| Windows laptops may finally be catching up.
|
| My first programming job was LAMP so I had a Linux desktop
| and loved it. Later I got a new job that gave us laptops, but
| they were quite beefy.
|
| I had a Dell laptop with an Nvidia GPU and an Intel iGPU...
| After updating my OS my gpu was the only way to use my
| laptop, which made the battery die in under an hour and of
| course it was much hotter.
|
| I tried numerous driver installs, proprietary, open source,
| reinstall OS, different OS... Nothing got it working again on
| a newer version of the Linux kernel.
|
| Went to the Apple Store bought a MBP and have never had an
| issue since. Not one dead laptop, in 10 years, I plug in my
| USB C dock and go.
|
| 2 years later, what happened to one of my coworkers? Same
| exact thing. He spent 3 days trying to fix it and basically
| had a workaround that crashed occasionally.
|
| I get paid to produce working software not configure my OS,
| and people wonder why Macs are so popular?
| rorroe53 wrote:
| Macbooks have been nice since M1 era, but the Intel
| Macbooks between years 2013-2020 were hardly robust. My
| partner's 2014 MBP Retina's screen plastic film started
| peeling off, which was a known design flaw of those models.
| Later the ones with butterfly keyboard were notoriously
| unreliable, with keys getting stuck.
|
| Personally I haven't had much trouble with Linux on modern
| Thinkpads. Very little to configure manually, as long as
| you pick the right distro. Even a Dell laptop at work with
| Linux isn't causing me much OS-related issues, although
| battery life sucks.
| Klonoar wrote:
| Well, no. The 2015 MBP is a well known workhorse that
| stretched many people professionally up to the M1. I
| would absolutely agree that the 2016-2020 Intel MacBooks
| were rough though.
| rafram wrote:
| Most people scraping sites aren't writing anything low-level
| enough to care about the particular flavor of Unix-like OS it
| runs on.
| F7F7F7 wrote:
| I'd argue that there's no correlation at all between the two.
| victorbjorklund wrote:
| Why? I mostly code on Mac and deploy on Linux (or FreeBSD).
| Never really encountered a situation where programming a web
| app on Mac has caused issues when deploying to the server.
| UltraSane wrote:
| What about issues with CPU architecture?
| lionkor wrote:
| When you write web code you should never have to worry
| about that. Actually, if you write any user space code,
| except drivers, you shouldn't have to worry about that. If
| you have to worry about it, reconsider your tooling very
| seriously
| Sardtok wrote:
| Yes, when you write web apps in x86 assembly, it gets
| tricky.
| rob wrote:
| I'm still on OS/2 Warp.
| SparkyMcUnicorn wrote:
| AWS Graviton is ARM.
|
| My experience is that having a team with mixed platforms
| has helped reduce deployment woes, with the rare platform-
| specific bugs getting worked out beforehand.
| dangoodmanUT wrote:
| conjecture?
| criddell wrote:
| Or maybe some of the newer generation will take time to update
| Linux to be more competitive with macOS for developers. Could
| be a long term win for Linux fans.
| Sardtok wrote:
| Linux is good for development, but Apple hardware is pretty
| damned nice.
|
| Now if Framework laptops were available in Norway, I'd
| probably rather have that, even if they're not as powerful.
|
| Also, depending on where you work, there might be
| restrictions in the choice of platform. Usually limited to
| Mac or Windows.
| forty wrote:
| I agree, people don't realize the value of not depending on a
| single company to do their work. We can see this problem even
| more with LLM code generators.
| chuckadams wrote:
| How about a real-world example of the harm you're clutching
| your pearls over?
|
| Besides, most devs doing web development on Macs are also using
| Docker, which is always Linux.
| Merad wrote:
| Really? In the modern .Net world (originally .Net Core) it's
| very common for devs to use Windows machines to write code
| whose CI pipelines and deployed environments are all Linux.
| I've seen a handful of issues with things like path separators
| and file system case sensitivity, but we're talking about 3 or
| 4 minor problems in 6-7 years that I've been using it.
| colesantiago wrote:
| This is the easy part.
|
| One of the issues with these tools is that more and more websites
| now employ multiple aggressive CAPTCHAs, fingerprints, device
| check, etc, rendering tools like API Parrot almost useless.
| m00dy wrote:
| can it reverse websocket-protocols ? If so, how does it do binary
| decoding etc ?
| pvarghav wrote:
| Currently only HTTP requests are supported. I might add support
| for websockets later, however that is a harder problem to solve
| due to the binary encoding etc.
| moon82 wrote:
| looks amazing! thanks for sharing, will give it a shot in a short
| while. Btw, how do you keep yourself motivated on working on free
| projects? Obviosly it takes a lot of effort and no one is paying
| for that.
| pvarghav wrote:
| Thank you!
|
| Working on this side project has been both fun and rewarding.
| I've learned a lot throughout the process, which keeps me
| motivated even without immediate financial gain. I have plenty
| of ideas on how to improve the software in various ways. Some
| of these enhancements could become part of a "pro" version
| tailored for businesses. My long-term ambition is to turn this
| into a full-fledged product, which would enable me to dedicate
| more time to its development.
| pkkkzip wrote:
| interesting but not sure what the value add here is, it gives you
| a graph flow of all the API requests being made? and then the
| goal is to replay them?
|
| aren't there github libraries that do this already?
| rynn wrote:
| How does it compare to mitmmitmproxy2swagger?
|
| https://github.com/alufers/mitmproxy2swagger
| 1a527dd5 wrote:
| The first and immediate difference for me is the ability to
| recall the name. I can recall Postman/Insomina fine, and now
| for API Parrot. I'm never going to be able to recall
| mitmproxy2swagger.
|
| Unfortunately, names matter.
| itsafarqueue wrote:
| Thanks 1a527dd5.
| 1a527dd5 wrote:
| Ha! Nicely played. That was out of purely laziness. I don't
| like using one handle across sites, so I take the first 8
| chars of (New-Guid).ToString() and then dump it in my
| password manager.
| SparkyMcUnicorn wrote:
| I often forget the name of things, sometimes even the big
| ones. GitHub search is one of the primary ways I rediscover
| them. "reverse-engineer API" returns mitmproxy2swagger as the
| third result, and this is how I found it last time I needed
| it.
|
| It is a bit frustrating when a project on GitHub doesn't have
| good tags or searchable keywords, making it harder to find.
| yoavm wrote:
| As someone who uses mitmproxy and swagger quite often, I
| actually think the name isn't so bad. I haven't even looked
| at the readme but I already know what it does, how to run it
| and what output to expect.
| faizshah wrote:
| This might be more useful than the OP. This thing lets you
| translate HAR to Swagger...
|
| My usual process is Dev tools -> Copy as CURL -> delete
| unnecessary headers -> translates to requests in python (these
| days I just use ChatGPT) -> wrap in python sdk for managing
| auth etc.
|
| The OP's correlation features are really nice though.
| sumanyusharma wrote:
| How is this different from Integuru? They posted a few weeks back
| here: https://news.ycombinator.com/item?id=41983409
| teichman wrote:
| Integru has been really great for us. Curious how you think about
| differentiation?
| sidgarimella wrote:
| Love this. I've worked on a few projects in RPA prior and I'm
| losing faith in selectors. I think either direct data access like
| this or AI based CV are the automation arms of the future.
| remoquete wrote:
| Looks very interesting. Does it produce an OpenAPI file? That'd
| help immensely in documenting APIs that lack specifications.
| 1a527dd5 wrote:
| This is pretty cool, I ran it against one of a largest customer
| sites and it was very interesting to see how the page all
| interconnects. I'm pretty sure it can be used to spot
| architecture/performance problems.
| ozim wrote:
| If only there would be something with schema like XML that people
| would use for the APIs ;) You could generate diagrams from WSDL
| and even generate client code from that.
|
| There is also bunch of JSON schema stuff nowadays.
|
| But yeah for a lot of people schema of API contracts feels like
| too much work and too much hassle.
|
| JSON serialization doesn't throw errors for new properties
| quickly added on sending side and receiving side can ignore stuff
| - well as long as API semantics allow but that's generally going
| to be a hassle always even with LLMs somehow autofixing your
| ,,schema".
| tveyben wrote:
| I'm not able to read what the product actually does - I keep
| getting distracted by the 'snake' animation surrounding the
| content .. not sure what the purpose is ;-)
| sebmellen wrote:
| This is incredible. We've spent ages and ages figuring out the
| weird internals of certain legacy systems that we've ended up
| having to use bots or RPA to integrate with. If you can polish
| this into a true product, we would pay for it!
|
| Any chance of a Mac version?
___________________________________________________________________
(page generated 2025-01-01 23:00 UTC)