[HN Gopher] Maestro - Next generation mobile UI automation
___________________________________________________________________
Maestro - Next generation mobile UI automation
Author : jztan
Score : 101 points
Date : 2025-02-25 17:01 UTC (4 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jztan wrote:
| First of all, I didn't make this, let me be clear, and I don't
| work for the company Mobile.dev.
|
| I've been looking for a replacement for Appium because the
| documentation for that site is absolutely garbage. Maestro boils
| everything down to YAML and runs its own test server so that you
| don't have to worry about connecting to the device drivers. It's
| missing an API but who needs an API when the CLI is so beautiful.
|
| Does anyone know of anything on par with this that I should try?
| So far this has knocked my socks off.
| leland-takamine wrote:
| Co-author of Maestro here - really appreciate that support
| jztan! If you get a chance you should also try out web support
| which we recently released! And always open to feedback, so
| please let me know if there's anything you think can be
| improved!
| hugs wrote:
| What was the most garbage thing in your opinion re: Appium
| documentation?
| cschiller wrote:
| Hi Jztan, glad you're exploring this space! I'm the co-founder
| of MobileBoost, and I'd love to introduce our product, GPT
| Driver (https://www.mobileboost.io/).
|
| We started two years ago with an AI-native approach, which is
| particularly useful for handling dynamic flows, hard-to-locate
| UI elements, and testing across multiple platforms and
| languages. Our main objective is to reduce test maintenance
| effort.
|
| Duolingo recently shared their experience adopting our tooling:
| https://blog.duolingo.com/reduced-regression-testing/
|
| We offer: a Web Studio - A no-setup-required platform with all
| tooling preconfigured. SDKs - Directly integrate with existing
| test suites (Appium, XCUI, Espresso).
|
| Happy to answer any questions!
| bartekpacia wrote:
| Can I run this locally?
| cschiller wrote:
| Yes, you can use our SDKs to run it locally on Simulators,
| Emulators, and real devices. We also support popular third-
| party device farms via the WebDriver protocol.
| herval wrote:
| worth mentioning it also supports web testing (in beta), so it's
| not exclusively for mobile testing!
|
| https://docs.maestro.dev/platform-support/web-desktop-browse...
|
| (disclosure: I currently work for mobile.dev, so if you have any
| feedback or questions, feel free to drop a reply and I'll try to
| answer)
| Hawzen wrote:
| Maestro is great. However it lacks so many important features you
| might need
|
| For example, Maestro does not let you to coordinate multiple flow
| tests together. One test case I had is one phone initiating a
| call, and another answers it. Instead, Maestro prefers that every
| flow is self contained and will not run both in parallel reliably
|
| I found many such limitations in its design only after writing a
| whole lot of their custom flow syntax
| tibbe wrote:
| We used to use Maestro but then they unfortunately decided to go
| all in on AI and hiked the price to match, making it no longer
| worthwhile for us.
| leland-takamine wrote:
| Hey tibbe - co-founder of Mobile.dev here. First off, totally
| get where you're coming from. We do offer a startup discount,
| but would love to dig in more to see if there's something we
| can work out. My co-founder and I would love to chat if you're
| open to it! Just shoot me a note if interested!
| leland@mobile.dev
| wiradikusuma wrote:
| Is the open-source version not usable/allowed for CI/CD
| environment?
| leland-takamine wrote:
| Yep free and open-source to use. Plenty of folks run Maestro
| on directly on GitHub Actions, Bitrise, etc. Teams often run
| on our hosted cloud infra for parallelism and reliability
| when scaling up their testing, but that's totally up to you!
| smalugu wrote:
| At SoFi we are using it with gitlab and AWS Linux/Mac
| runners. We can do parallel runs, data driven, get test logs,
| screen recordings etc
| serial_dev wrote:
| We loved Maestro but we didn't like the pricing and the team
| wanted something more predictable. We are using Moropo and it's
| been great. Very affordable, good DX, and it's all basically
| just Maestro with extras. The pros of using open source!
| leland-takamine wrote:
| Hi everyone, co-founder of Mobile.dev and co-author of Maestro
| here . Thanks jstan for sharing Maestro and for the kind words--
| really glad to hear it's working well for you!
|
| We built Maestro because E2E testing felt unnecessarily
| complicated, and we wanted something simple and powerful that
| anyone could use -- whether you're a seasoned developer or just
| getting started with automation. It's been amazing to see it
| adopted at companies like Meta, Block, DoorDash, Stripe, and
| Disney, but honestly, what excites us most is seeing teams who've
| never done test automation before finally get a solid strategy in
| place because Maestro is so easy to use and get started with.
|
| Oh, and if you're wondering -- yes, it works for web testing too!
|
| We're constantly iterating and adding features, so if you've got
| ideas, run into issues, or just want to chat, let me know. Always
| happy to hear how we can make it better.
|
| Thanks again for checking it out, and happy testing!
| tcoff91 wrote:
| Please add the ability to drive an actual iOS device instead of
| simulators.
| leland-takamine wrote:
| In the works - stay tuned!
| SoKamil wrote:
| How does it compare to writing Appium tests?
| leland-takamine wrote:
| There was a good thread on reddit a while back on this topic.
| Post was simply a request for opinions on Appium, but virtually
| everyone ended up recommending Maestro instead:
| https://www.reddit.com/r/QualityAssurance/comments/1771ca7/o...
|
| Honestly, I'd just give it a try yourself - you can get started
| in minutes
| hugs wrote:
| Appium and Selenium project creator here. Just saying hi.
| metanonsense wrote:
| Cheers, huge Huggins fan here :) Looking forward to Valet
| hugs wrote:
| so am i! (it's been a crazy month, but that's not very
| obvious from the outside.)
| mrbombastic wrote:
| Trying this out at work and so far it has been leagues better
| than other mobile automation tools. I have just gotten started
| but it has been encouraging.
| leland-takamine wrote:
| Awesome to hear! Feel free to tag me (@Leland) in our Slack
| community if you run into any issues! Slack invite:
| https://docsend.com/view/3r2sf8fvvcjxvbtk
| billylo wrote:
| Installed and test-driven it...
|
| Really like the fact that it's easy to start doing something
| useful. I may end up using it for some screen scraping too.
| Puppeteer is powerful, but the scripts tends to be brittle.
|
| Keep up the good work!
| leland-takamine wrote:
| Awesome to hear! There's still tons we want to do on the Web
| side, so please let me know if there's anything you think
| should be added or improved there! Feel free to tag/DM me
| (@Leland) in our Slack community or email me leland@mobile.dev
| with questions/suggestions!
| 999900000999 wrote:
| Why Yaml ?
|
| I've worked in this niche for a very long time, I seriously need
| to be able to use a normal programming language. A lot of test
| tools need to be a part of a larger workflow, if this was good
| ole NodeJS I could use some other tricks, for example
| intercepting network request,custom logic in JavaScript, etc.
| leland-takamine wrote:
| We've found that YAML encourages maintainable testing
| practices. But when you need to fire off a network requests,
| add custom logic, Maestro does have javascript support:
| https://docs.maestro.dev/advanced/javascript/run-javascript
|
| Very sophisticated companies like DoorDash and Kraken have
| written and maintain hundreds of Maestro tests using this
| approach!
| 999900000999 wrote:
| Awesome, thank you for replying to me!
|
| However, is it possible to use your testing framework as a
| library inside of a larger project.
| Onavo wrote:
| The newer UI testing tools like mobileboost and QA buddy all
| support using vision language models and natural language to make
| testing easier. Do you plan to add support for that?
|
| https://www.uber.com/en-US/blog/generative-ai-for-high-quali...
| leland-takamine wrote:
| 100% - here are some of Maestro's AI-powered commands:
|
| * https://docs.maestro.dev/api-reference/commands/assertwithai
|
| * https://docs.maestro.dev/api-
| reference/commands/extracttextw...
|
| Main difference is that Maestro takes a reliable-by-default
| approach. We hear plenty of stories of folks exploring tools
| like the ones you mentioned, then ultimately coming back to
| Maestro due to reliability / reproducibility issues, which are
| non-negotiable when it comes to end to end testing
| Henchman21 wrote:
| I gather this is unrelated to the old CA/Unison project called
| Maestro or maybe Tivoli Workload Scheduler?
| nomadev wrote:
| What is stopping adoption of such projects? The quality of most
| mobile apps sucks so not sure why mobile testing is not
| mainstream?
| divan wrote:
| Installed and tried it for a sample Flutter app. So far looks too
| good to be true :) Super easy to start and tinker with. And
| surprisingly fast. Learning how to write real world tests with
| Flutter apps probably will have some learning curve, but that's
| expected.
|
| Would be amazing to use it with Flutter desktop (macos at least)
| to avoid running iOS simulators.
| serial_dev wrote:
| I've been using Maestro for two very large Flutter apps and
| it's been so ahead of every other option is not even funny.
|
| No long compilation times, no half baked testing dev
| experience, supports iOS and Android, no pumpAndSettle BS, No
| Flutter hacks, multiple cloud providers (cloud.mobile.dev,
| moropo), you can interact with native elements, so you can work
| with push notifications, system dialogs, system settings, email
| clients, web views, browsers, and a very simple test definition
| files that every capable QA engineer can maintain with very
| little supervision from developers (no need for dart expertise
| for writing tests).
|
| I can only recommend it.
| bartekpacia wrote:
| I'm curious what other solutions you tried to test your
| Flutter app.
| serial_dev wrote:
| Vanilla Flutter tests, Honey, Patrol, these are the ones I
| remember.
|
| And yes, amongst those Patrol was the best but at the time
| we decided, the Maestro experience was significantly
| better.
___________________________________________________________________
(page generated 2025-03-01 23:00 UTC)