[HN Gopher] Show HN: OtterTune - Automated Database Tuning Servi...
___________________________________________________________________
Show HN: OtterTune - Automated Database Tuning Service for RDS
MySQL/Postgres
Yo. OtterTune is a database optimization service. It uses machine
learning to automatically tune your MySQL and Postgres
configuration (i.e., RDS parameter groups) to improve performance
and reduce costs. It does this by only looking at your database's
runtime metrics (e.g., INNODB_METRICS, pg_stat_database,
CloudWatch). We don't need to examine sensitive queries or user
tables. We spun this project out of my research group at Carnegie
Mellon University in 2020. This week we've announced that
OtterTune is now available to the public. We are offering everyone
a starter account to try it out on their Postgres RDS or MySQL RDS
databases (all versions, AWS US AZs only). We have seen OtterTune
achieve 2-4x performance improvements and 50% cost reductions for
these databases compared to using Amazon's default RDS
configuration. I am happy to answer any questions that you may
have about how OtterTune works here. -- Andy ================
More Info: * 5min Demo Video:
https://ottertune.com/blog/ottertune-explained-in-five-minutes *
Free Account Sign-up: https://ottertune.com/try
Author : apavlo
Score : 74 points
Date : 2021-10-14 18:54 UTC (4 hours ago)
| Aaronstotle wrote:
| Heads up, was reading the docs about how it works.
|
| >The OtterTune Agent connects to the database to retrieve the
| necessary information. OtterTune only requires access to retrieve
| the database's runtime metrics and configuration information
| (knobs). It does need to access user tables or queries.
|
| Presumably, the last line should say OtterTune does NOT need to
| access tables/queries?
| apavlo wrote:
| Good catch. Fixed!
| mdasen wrote:
| Hi! This looks really cool and I like that you've
| published/presented at VLDB.
|
| I'm a little curious why AWS is free/cheap, but anything other
| than AWS ends up in the "Enterprise, contact us for pricing"
| bucket. It might simply be based on your costs. I don't know if
| the on-prem stuff is expensive since your software needs to be in
| the same datacenter or if it's just a pricing differentiator.
|
| I'm also wondering if you've thought of going the DBaaS route. It
| looks like AWS is charging an ~80% premium for RDS over regular
| on-demand instances. If you're managing a fleet of them and
| getting lower pricing (through committed-use discounts and
| negotiation) you could potentially get good margin there. For
| example, AWS charges $280/mo (on-demand) for an M5.2xLarge and
| $500/mo (on-demand) for the same instance for MySQL RDS (single
| AZ). So Amazon is already asking for an additional $220 and then
| you'd like $90/database ($450 / 5 databases on the standard
| plan). It seems like there might be more opportunity in the DBaaS
| markup than in trying to convince someone to use your tuning
| engine.
|
| From your video, it doesn't sound like OtterTune does this, but
| it might be worth thinking about: maybe not just tuning the
| database, but also offering suggestions on potential anti-
| patterns happening. For example, if someone has code that should
| be using connection pools and isn't, it could be helpful to
| present it as something potentially important. Likewise, let's
| say that there are unindexed queries. Detecting that and
| recommending solutions is helpful. That seems like it could offer
| ongoing benefit.
|
| Speaking of ongoing benefit, would there be a lot of benefit to
| paying for more than one month? In the 5 minute video, it showed
| the configuration stabilizing pretty quickly once it reached a
| good place. Assuming that my access patterns remain relatively
| the same, it seems like I could just continue with that
| configuration. Maybe I could sign up for one month every year to
| get a check-up. This isn't meant to be a criticism. It's more
| that I've seen so many customers exploit holes in SaaS pricing.
| If you're counting on recurring revenue, what happens if a
| customer notices that after a day they don't need the knobs tuned
| anymore? Can I get that $450 5-DB plan and run it on 5 databases
| on Monday, remove it from those 5 databases on Tuesday and apply
| it to 5 new databases on Tuesday (effectively getting 150
| databases per month for the price of 5)? Or maybe there is a need
| to be tuning the configuration hourly/daily (though that seems
| less likely).
|
| I do like the idea of auto-tuned knobs and it seems like a great
| service. I guess it's just hard to try if, like me, you're not on
| AWS. I'm sure that most people are so I'm not suggesting it's an
| inappropriate restriction for the free tier. Really cool and
| congrats on VLDB!
| onmysofa wrote:
| Wondering why they decided not to give free trials to EU Zones.
| GDPR?
| apavlo wrote:
| We have done two major deployments of OtterTune in Europe. One
| of them we published a VLDB paper about
| (https://ottertune.com/blog/vldb-autonomous-database-
| tuning-i...). Their infosec people looked at the data OtterTune
| collects and determined that there are no GDPR issues.
|
| We are incrementally expanding the AZs we support in the free
| tier in order to carefully scale our service. We plan to
| support EU zones by the end of the year.
| ddlutz wrote:
| Looks like only AWS is supported now. Will there be GCP Cloud SQL
| support? Seems very useful!
| apavlo wrote:
| We have had very few requests to support GCP. After AWS and on-
| prem, the next most requested platform is Azure.
|
| If you really want GCP, let us know here:
| https://ottertune.com/contact
| mritchie712 wrote:
| What are the top three Postgres configurations Otter changes?
| e.g. parameter x from y to z
| apavlo wrote:
| It depends on the workload (OLTP vs. OLAP, read-heavy vs.
| write-heavy). In our original research paper
| (https://db.cs.cmu.edu/papers/2017/p1009-van-aken.pdf), we came
| up with an automated way of determining which knobs have the
| most impact on performance. We've tested up to Postgres v13 in
| the commercial version of OtterTune.
|
| For most workloads, shared_buffers and work_mem have the most
| impact. For write-heavy OLTP workloads, tuning the WAL
| (max_wal_size) and autovacuum knobs (e.g.,
| autovacuum_vacuum_scale_factor) have the most benefit. For
| read-mostly OLAP workloads, Postgres' parallel knobs
| (max_parallel_workers_per_gather) provide the most improvement.
|
| But you need to also tune all the other knobs to get the last
| 15-40% of potential performance improvement. This is what
| OtterTune can do.
|
| More info about how we select knobs:
| https://ottertune.com/blog/prevent-machine-learning-from-wre...
| m_ke wrote:
| Forgot the band link: https://ottertune.bandcamp.com/music
| m_ke wrote:
| Also Andy Pavlo is my #1 celebrity crush, for people who are
| not aware his DB lectures are great:
| https://www.youtube.com/c/CMUDatabaseGroup/featured
| steverob wrote:
| If we leave just monitoring turned on, would we still see
| recommendations done by the tuning engine or do I need to enable
| auto time to start seeing those inputs from Ottertune?
| apavlo wrote:
| Our expected usage for the current version of OtterTune is that
| you will enable tuning and then let the ML algorithms figure
| out your workload patterns. The models will then converge with
| an optimal config and then you switch it back into monitoring
| mode. How long this takes depends on your workload patterns.
|
| We are working on the ability for OtterTune to alert you during
| monitoring mode when it thinks your workload/database have
| changed enough that it merits turning the tuning mode back on.
| We could also have it show the recommendations when this occurs
| as you suggest.
| devops000 wrote:
| Do you plan to support self-hosted Postgres ?
| apavlo wrote:
| Yes. It's on our roadmap for next year.
| miohtama wrote:
| Why is it called OtterTune?
| apavlo wrote:
| It's play on the word "autotune".
|
| Otters are also vicious animals, which is an important trait to
| have if you are working with databases:
| https://youtu.be/J7f6s2g8C0I
| claytonjy wrote:
| I know this Show HN is about RDS specifically, but the site
| suggests this works elsewhere, too; any issues with pointing
| OtterTune at DBs making heavy use of extensions (e.g. Timescale,
| Citus) or nonstandard deployment approaches (k8s, patroni,
| vitess)?
| apavlo wrote:
| We have deployed OtterTune for on-prem databases (baremetal,
| containers). But we are not offering that to everyone right now
| because organizations manage their configurations for these
| deployments in a bunch of different ways (Github actions,
| Chef/Puppet, Terraform). The nice thing about RDS is that they
| have a single API for updating configurations (https://docs.aws
| .amazon.com/AmazonRDS/latest/UserGuide/USER_...).
|
| As for the Postgres-derivates that you mentioned (Timescale,
| Citus), we have not tested OtterTune for them yet. But there is
| no reason they shouldn't also work because they expose the same
| metrics API (pg_stat_database) and config knobs. There are just
| way more people using Postgres (RDS, Aurora), so we are
| focusing on them right now.
| claytonjy wrote:
| Makes sense! thanks for the reply Andy.
| slig wrote:
| Once a database has been tunned, why would people keep paying for
| the service?
| apavlo wrote:
| It's true that for static workloads/databases, you don't need
| continuous tuning.
|
| But as you add new app features, grow database size, upgrade
| DBMS versions, and make other changes, OtterTune tracks your
| system to make sure you are always using the best config. This
| is problem is even harder with 1000s of individual database
| instances. Customers have told us that they want something can
| tune database continuously in a way that is not possible with
| DBAs and existing monitoring tools.
|
| Knob tuning is also the first step in the kind of automation
| that we are working on using OtterTune's ML-based approach.
| apavlo wrote:
| Details about our new self-service offering using Fargate is
| available here:
| https://ottertune.com/blog/ottertune-2021-10-product-update
___________________________________________________________________
(page generated 2021-10-14 23:00 UTC)