[HN Gopher] Mirrord - mirror production traffic to your developm...
___________________________________________________________________
Mirrord - mirror production traffic to your development environment
Author : capableweb
Score : 77 points
Date : 2022-06-05 18:16 UTC (4 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jonahbenton wrote:
| I used goreplay in an earlier life, is this similar?
| aviramha wrote:
| Somewhat similar. mirrord works live only (currently) but
| doesn't just replay the traffic, it wraps your process and
| duplicates connection to your local service as if it's running
| in the remote environment. We'll soon add filesystem access,
| environment variables and outgoing traffic enabling developers
| to work seamless on a remote environment.
| geiser wrote:
| How does this compare to Telepresence?
| https://www.telepresence.io/
| eyalbukchin wrote:
| First of all, mirrord runs on the process level. Telepresence
| either runs on your local machine, changing its network
| behavior or in a container making development less smooth than
| running a local process. Second, mirrord's default is to
| duplicate traffic and not intercept it, so useful when you
| don't want to disrupt the cloud environment, want to work
| concurrently with other developers on the same service, etc.
| onion2k wrote:
| Please don't copy production data to computers you connect to
| coffee shop WiFi.
| [deleted]
| blumomo wrote:
| Does mirrord not allow to connect via an encrypted protocol
| such as HTTPS?
| onion2k wrote:
| If you copy prod data to a computer that's a public
| connection then _any compromised app_ can exfiltrate the
| data. Keep data secure all the time, including when it 's on
| a local machine. Even if the machine isn't compromised,
| someone could just mug you for it and then you have to start
| reporting the data breach. It would be... messy.
|
| To be honest, while I imagine there are times when access to
| prod traffic might be useful, I suspect even having a tool
| like this would be an instant ISO27001 or ISO27002 audit
| fail.
| blumomo wrote:
| How does the attack vector that you describe differ from
| being connected to your office or home network? Once this
| kind of malicious app that you describe is running on your
| machine, it doesn't matter any longer what kind of network
| you're connected to.
| simonw wrote:
| A trick I've used to do something similar in the past is to setup
| detailed API logs that are designed to be replayed elsewhere.
|
| This involves more than just standard web access logs, since you
| also need to capture any POST bodies too in order to be able to
| replay them later against a test environment.
|
| I wrote more about this here:
| https://simonwillison.net/2021/Apr/12/porting-vaccinateca-to...
| rocmcd wrote:
| What would the benefits of mirrord be over something like the
| NGINX Ingress Controller with mirroring enabled?
| aviramha wrote:
| Right now the focus of mirrord is for back end engineers, so
| the use case is backend engineers wanting to work on real
| traffic and real environment being able to do so without help
| of or change of infrastructure
| beanclap wrote:
| This looks really cool! Any plans to support rerouting traffic
| entirely, rather than just mirroring?
| eyalbukchin wrote:
| Yes! This is actually in development at the moment so should be
| out really soon. Along that same line, we're also planning to
| support routing of outgoing traffic back to the cloud
| environment. We're obviously thinking of both those features as
| more suitable for use with a staging environment rather than
| production.
| beanclap wrote:
| oh wow, super curious to check it out once its released!
| awild wrote:
| This looks really neat. Haven't taken much time to look in depth,
| it it took me a while to find info on whether this was k8s or
| podman related. Maybe the docs could make it a little more
| obvious. (for those wondering it's k8s)
|
| Would mirrord actually work with podman? It would be pretty neat
| to setup something akin to a canary deployment
| m3adow wrote:
| It's literally in the first sentence of the README:
|
| > mirrord lets you easily mirror traffic from your Kubernetes
| cluster to your development environment.
|
| That said, cool application, although I can see a lot of
| compliance and/or data protection problems when production data
| is leaving the (hopefully protected) production environment.
| aviramha wrote:
| Thanks for the feedback! One of the creators here :) Our
| initial thought is to create a solution similar to what Lyft
| has done (https://eng.lyft.com/scaling-productivity-on-
| microservices-a...) to enable using staging as a dev
| environment and sharing it with many developers at once. We
| do want people to use at us production later on, but right
| now we know there might be compliance/security issues.
| reilly3000 wrote:
| This is a real concern for PCI, PII, and SOX services. If you
| have good service inventory you could gate this to only work
| for approved apps, but some live rewriting of form data could
| also be helpful. Faker on demand :)
| aviramha wrote:
| Hey, one of the creators or mirrord here. The technology behind
| mirrord is actually agnostic - what it does is wrap a local
| process and "proxy" it's API calls to a remote environment and
| remote traffic is also mirrored from there. We tailored the
| solution for K8S to provide seamless experience, but if there's
| a need for another orchestration solution, we can consider
| adding support :)
| idealmedtech wrote:
| Forgive my ignorance, web et al is not my specialty, but how
| does this work with HTTPS traffic? Do you install a root cert
| in the dev environment like Wireshark to be able to mirror
| agnostically? Or does there need to be application config
| (DNS change, internal routing changes) to make this work? If
| the latter, what's the value add here?
| eyalbukchin wrote:
| Our agent sniffs traffic in the same network namespace as
| the pod whose incoming traffic we mirror, so in most cases
| that would be after the HTTPS traffic is decrypted (by e.g.
| a load balancer, or a service mesh)
| reilly3000 wrote:
| How does this work IRL for side-effectful services.for example,
| if somebody posts an order and it mirrors to my dev service, does
| that also hit my dev DB? Then if there is a live request to
| access a record detail page, does that expect my dev DB to have
| it? I'm just trying to wrap my brain about teams use tools like
| this in practice. Is it mostly for load testing or a better form
| of E2E?
| opportune wrote:
| I would generally use this for e2e testing since it's too
| expensive to run a 100% mirror of a large production
| environment. Well, depends on your scale and whether your
| production costs are a significant factor in your total
| expenses - I guess if your production costs are tiny compared
| to revenue/dev time spent debugging, you could justify a full
| mirror. For something like an eshop where revenue >> production
| costs you could probably justify cloning out production
| multiple times.
|
| Sampling is another useful tool but I don't see any sampling in
| the API for this. If you do sample though you could end up with
| inconsistent or divergent state if you just sampled a flat % of
| traffic, so in some contexts it makes sense for sampling to be
| determined per-user/tenant/etc. rather than per-request.
|
| This is pod based so you could easily slice by pod. If you have
| a pod with a lot less traffic, more impactful traffic, or that
| is harder to debug (example: pod that periodically aggregates
| and exports metrics but you believe it's dropping data
| sometimes) you could just enroll that and have some kind of dev
| shard of a portion of the stack.
|
| I could also see this being very useful if you believe it to be
| too big a liability to record actual user requests (or
| implement something special to scrub out sensitive stuff) or
| let devs peek too close. You have a way of investigating some
| classes of issues without necessarily looking at sensitive data
| or requiring extensive back and forth in a support case.
| eyalbukchin wrote:
| Up to you: you're going to be able to configure what works
| locally vs what works with the cloud environment. So for
| example, you could configure all read functionality (incoming
| traffic, DB reads) to work with the cloud, and all write
| functionality to work locally. Further down the line, we're
| planning to support Copy On Write for databases - so you'd read
| from the cloud, write locally, then read the records you wrote
| locally (and records you haven't written, from the cloud).
| Disclaimer: we're still pretty early in the roadmap, so
| currently only support mirroring incoming traffic. But we're
| moving fast :)
___________________________________________________________________
(page generated 2022-06-05 23:00 UTC)