[HN Gopher] Mitmproxy 8
___________________________________________________________________
Mitmproxy 8
Author : soheilpro
Score : 159 points
Date : 2022-03-20 14:06 UTC (8 hours ago)
(HTM) web link (mitmproxy.org)
(TXT) w3m dump (mitmproxy.org)
| jorgesborges wrote:
| This is cool. I'm learning a lot reading through the docs. Why is
| subversion so instructive? I've read about SSL certificates a
| gazillion times and it's simple enough in theory but for some
| reason this is making everything click.
|
| What's the use most common use case for something like this, by
| the way?
| bspammer wrote:
| I recently used it to modify the response from the replay
| server for an autobattler game, to let me watch arbitrary
| replays for theory-crafting the ideal team. The game is called
| Super Auto Pets, here's the code with the mitmproxy extension:
| https://github.com/bspammer/super-auto-pets
| cellularmitosis wrote:
| Believe it or not, the most common use-case is simply
| discovering what your code is actually doing. Modern frameworks
| are often so far removed from the nuts and bolts of HTTP and
| TCP that it can be difficult to know what's actually happening
| on the wire without using a tool like this.
|
| A similar frustration is the tendency for SDK / library vendors
| to document their product using code snippets in various
| languages, but without simply showing you what the raw HTTP
| requests look like. Typically the closest you can get is a set
| of curl commands.
|
| When you show me a python/ruby/swift/js snippet, I have a guess
| as to what's happening. When you show me HTTP, I know _exactly_
| what 's going on. And if you won't show me HTTP, time to fire
| up mitmproxy.
| gizmo686 wrote:
| Debugging things that happen over TLS.
| memco wrote:
| Not sure about the most common, but I used it to inspect some
| requests that an app was making so I could find the needed
| parameters to get the data from the App's backend servers in a
| tool I was developing. I was able to proxy the app's connection
| and inspect the traffic as I used the app and was easily able
| to see what it was doing and replicated the parts I needed.
| jeroenhd wrote:
| You can use this to debug stuff happening over TLS connections,
| but there are also other practical ways to use it.
|
| For example, install the CA certificate on your phone (with
| root if you have Android) and set up a WiFi network that
| transparently proxies everything through mitmproxy. Let it
| collect data for a while and you'll be surprised how many
| random servers even well known apps try to contact. You'll also
| be able to see what data they're trying to send! With modern
| TLS pinning that's a little hard, but there are Frida scripts
| out there for rooted/jailbroken devices that will let you
| bypass that. Just make sure to remove the cert afterwards and
| to reboot the device to make sure none of your mods remain in
| memory where they might affect your device's security.
|
| You don't always need TLS, either. It's also useful as a
| general proxy for protocols like HTTP, for example to
| intercept, alter, and repeat requests. This can be very useful
| if you're running into a bug accessing a backend endpoint from
| code that doesn't easily allow you to iterate through the
| request code, like some mobile dev frameworks.
| gizmo686 wrote:
| I haven't tried it for replacing TLS certificates, but you do
| not need a rooted phone to mess with 3rd party apps on
| android. You can unpack an APK and re-sign it using your own
| key, allowing you to make whatever modifications to the APK
| you want. In addition to making whatever changes you want to
| directly (such as just replacing the pinned certificates),
| you can update the manifest to set the android:debuggable
| flag to true, which allows you to access and modify the apps
| private files through ADB.
|
| The main downside of this approach is you cannot do an in-
| place replacement of already installed app, as your version
| is signed by a different key. You need to uninstall the
| previous version first, causing you to lose existing app
| data. At one point, I was able to work around this using
| Android's backup feature, but I never bothered to retain data
| since then since it was too much of a hassle and turned out
| not to matter that much for any of the apps I was playing
| with.
| jeroenhd wrote:
| You're right, but this does have side effects. APIs like in
| app purchases and DRM freak out about the rogue signature.
|
| It's also a huge hassle if you just want to see what
| metadata your phone is leaking.
|
| If you're only going after a few specific apps then sure,
| you can just alter the APK. It's not hard to disassemble
| and patch the custom validation code most apps use either.
| The downside is just that it's a huge bother.
| almog wrote:
| Since Android Nougat I think you cannot install user
| certificates unless you: 1. Root your device. 2. Add the
| certificate to the APK manifest.
|
| I'm not well versed in Android changes, this is just the
| conclusion I came to when Nougat came out and haven't tried
| it again since. If there's an easier way to use MITM with
| Android without modifying the APK / rooting your device, I'd
| love to know!
| jeroenhd wrote:
| You can install certificates just fine, but apps need to
| opt in to use user certificates. You can MitM apps that
| have this flag enabled, but very few mobile developers seem
| to even know what TLS is, let alone how and why you should
| change trust store settings.
|
| With root access, you can put your certificate in the root
| store, which most apps use for validation. I wrote a blog
| about it around the time Android 7 came out. Some apps
| (either not enough or too many, depending on your
| perspective) leverage Android's certificate pinning feature
| to connect a domain to a specific TLS certificate. You
| can't MitM those apps without either modifying the APK or
| hijacking them via Frida.
|
| Lastly, if you want to MitM Firefox, you need to enable
| their secret debug menu (available on some builds) to
| enable the flag that enables user certificates. Flagging
| the relevant setting in about:config does not work and
| about:config isn't even accessible on stable builds of
| Firefox for Android.
|
| Sadly, there's no good way to MitM an Android device
| without full system access. Android has always been quite
| annoying to use custom CA certs for, but since Android 7
| it's pretty much impossible without root access. I'm not
| entirely sure why they went this route instead of sticking
| to their existing warnings (i.e. a notification every time
| a custom CA gets used, like in Android 4, and a constant
| "your network may be monitored" label in the notification
| tray like in Android 5+).
| almog wrote:
| You are right, I didn't phrase it correctly, what I is
| meant that since Nougat I'm not aware of way to force
| system-wide use of a user certificate without gaining
| root access.
|
| Thanks for the Firefox tip as well, it's been a while
| since I used Mitmproxy and I see things have not become
| easier since.
| tprynn wrote:
| This is true, by default Android apps do not trust user-
| installed certificate authorities. IMO the easiest solution
| if you're doing security testing on a dedicated device is
| MagiskTrustUserCerts[1]. If you're not testing on a
| dedicated device or you don't want to root the device, I'd
| recommend using the objection[2] tool which has a guided
| mode for patching an apk, and you can modify the manifest
| to add your CA or to trust all user-installed CAs.
|
| [1]: https://github.com/NVISOsecurity/MagiskTrustUserCerts
|
| [2]: https://github.com/sensepost/objection/wiki/Patching-
| Android...
| almog wrote:
| Neat! Both Magisk (which I might use on an older phone
| which I don't mind rooting) and Objection are new to me,
| thank you!
| _ZeD_ wrote:
| why should I use this instead of fiddler[0]?
|
| [0] https://www.telerik.com/fiddler/fiddler-classic
| marcodiego wrote:
| When I was a teenager I wrote generic proxy in Borland C++
| Builder. I say generic because it was not protocol specific, at
| the time I called it a "bridge". I had lots of fun convincing
| 'friends' to try my "own IRC server" and watch they interact with
| other people.
|
| These days I think it would be a simple matter of using tee and
| netcat, but it was fun at the time.
|
| It is a shame getting a "real ip" is such a hassle nowadays.
| cjbprime wrote:
| Neat! Anyone happen to know if there's progress on interactive
| interception of TCP TLS streams (non-http)?
| BrianHenryIE wrote:
| I was reverse engineering my microwave a few years ago and
| couldn't MITM on XMPP because of the TLS so I used a jailbroken
| iPhone and was able to "swizzle" the Objective-C methods to log
| the data at a method right before encryption started. Fun.
|
| There was another XMPP MITM I could have used, but I think I
| was already invested in figuring out the first idea before I
| learned of the simpler, existing way:
|
| https://github.com/BrianHenryIE/XMPPFrameworkLogger
|
| https://github.com/iamultra/xmppmitm
| mhils wrote:
| Not yet. What works reasonably well is to iteratively work with
| a custom addon (addons are livereloaded on every change). But
| I'd love to see progress on that front, it'd make a great GSoC
| project! :)
| Tretiotrr wrote:
| Is there any benchmark against squid proxy in regards of cpu and
| memory usage?
|
| Anyone has expiernce with it or can recommend something else?
| orev wrote:
| Squid is meant to be used as a general proxy for things like
| caching to help your local network save bandwidth, etc.
| Mitmproxy is more of a pen testing/security tool. They're very
| different things and not something you would be comparing
| benchmarks to try to choose one.
| tills13 wrote:
| FWIW from experience MITMproxy eventually reaches a point on my
| 2019 MacBook Pro where the whole computer becomes unusable. I
| have to restart MITMproxy.
|
| I don't think they are doing all the necessary GC when
| disposing of flows from the UI.
| sanqui wrote:
| I've been looking to use mitmproxy to capture traffic in a
| project, but the serialization format is something called
| TNetStrings [1], which _postfixes_ types and is thus not
| streamable. There seem to be some efforts to switch flow captures
| to use SQLite [2] or Protobuf [3], although PCAP would also be
| nice, I think.
|
| [1] https://tnetstrings.info/
|
| [2] https://github.com/mitmproxy/mitmproxy/issues/3075
|
| [3] https://github.com/mitmproxy/mitmproxy/pull/3232
| ireneu wrote:
| There's an addon that supports PCAP files [0]. It works quite
| well in my experience.
|
| [0] https://github.com/muzuiget/mitmpcap
| mhils wrote:
| Could you explain what your use case is? What format would you
| prefer?
| markuta wrote:
| I've used mitmproxy with the transparent proxy mode for
| decrypting VPN traffic. Works really well. The mitmweb feature is
| quite usually too.
| mhils wrote:
| mitmproxy dev here! Happy to answer any questions. Answers may
| take a while as I'm on a remote mountain hut, but please shoot.
| :)
| igetspam wrote:
| No question, just a thanks. First used mitmproxy to build a
| demo CDN to show off for a customer that hadn't signed yet, so
| they wouldn't give me certs (reasonable). It was so simple to
| use, it scares people. I had to explain how CA trusts worked
| before the company would consider using it to show off.
| GordonS wrote:
| I've used several other apps like Mitmproxy, and some of them
| have nicer UIs... but Mitmproxy is the only one I've found that
| supports MITM'ng connections using client certificates.
| daeken wrote:
| FWIW, Burp also does this. But I'm not sure I'd argue that it
| actually has a nicer UI ;)
| mhils wrote:
| Shameless plug: We (mitmproxy) have a somewhat reasonable web
| UI (React/Typescript), but I think there's a lot more
| potential here. We're also looking for Google Summer of Code
| students at the moment, so if someone wants to spend a few
| months hacking on cool FOSS stuff while getting a nice
| stipend for it, please check out
| https://github.com/mitmproxy/mitmproxy/issues/5048 and reach
| out to me. I'd love to work with good front end folks on
| this! :)
| djbusby wrote:
| I'm still loving the TUI. Thanks for the awesome work.
| Groxx wrote:
| er. don't they all? at least all the sophisticated ones.
| otherwise you can't see the contents of https/etc requests.
| Charles does this as well for example, and it's reasonably
| popular: https://www.charlesproxy.com/
|
| AFAICT it's roughly as widely used as mitmproxy, and e.g.
| google trends favors Charles by a moderate amount:
| https://trends.google.com/trends/explore?geo=US&q=%22charles...
|
| or are you referring to some other cert-related feature?
| hug wrote:
| I assume he means client certificates for HTTPS mutual auth.
|
| https://techcommunity.microsoft.com/t5/iis-support-
| blog/clie...
| GordonS wrote:
| I think you're confusing server certificates and client
| certificates - I'm specifically referring to client
| certificates here, aka TLS mutual authentication.
|
| Also, isn't Charles OSX only?
| Groxx wrote:
| OSX-only: I think so. But google trends doesn't care about
| that, so the scale-of-support side still stands.
|
| And yeah, I wasn't thinking of client certs for auth
| purposes. I assume that'll hide the encryption keys, so you
| need a proxy that can do its own exchange? I have no idea
| what support for that looks like, I've never tried.
| armitron wrote:
| Nice idea but terrible implementation. Mitmproxy solves a lot of
| annoying problems, but, Python holds it back to such an extent
| that it's hard to make the case for deploying it in production
| due to massive inefficiencies.
|
| Can't wait for someone to do it in Go.
| develatio wrote:
| I thought mitmproxy was designed to make it easier to debug
| stuff in a local environment. Why would you want to deploy it
| to production? How would you use it?
| throwaway1777 wrote:
| Maybe they write malware?
| retrobox wrote:
| I'm also curious about the use case. For debugging things
| locally, it is ideal for my needs.
| armitron wrote:
| Many different applications: Red/blue teaming, metric
| collection, policy-based decision making.
| marginalia_nu wrote:
| Seems like a bit of a foot-gun to do these things by
| effectively undermining your own security.
| creeble wrote:
| Not sure I understand "MITM" and "in production".
| armitron wrote:
| Every single major corporation does "MITM in production".
| [deleted]
| oarsinsync wrote:
| This isn't a replacement for a Bluecoat, this is a
| developer/debugger application.
| Beltalowda wrote:
| mitmproxy is explicitly intended as an _interactive_ MITM
| for debugging: "your swiss-army knife for debugging,
| testing, privacy measurements, and penetration testing".
|
| What you're talking about here was never a goal. And being
| in Python means it's much easier to extend, which is also
| greatly useful.
| creeble wrote:
| Well, okay, got me. But you don't need to do what mitmproxy
| does in that case.
| stgn wrote:
| One of the things I like about mitmproxy is how easy it is to
| develop and use addons. You just pass a script file to it. How
| would you achieve such painless extensibility if it were
| written in Go?
| Beltalowda wrote:
| Exactly this.
|
| It's very hard to do with Go; also see Hugo, where Go is in
| many ways "holding it back" compared to something like
| Jekyll, which you can customize and hack with great ease.
|
| I love Go, but this kind of things is not where it shines.
| fork-while-fork wrote:
| Check out bettercap
___________________________________________________________________
(page generated 2022-03-20 23:01 UTC)