[HN Gopher] Show HN: Polytomic Connect - API for two-way ETL and...
___________________________________________________________________
Show HN: Polytomic Connect - API for two-way ETL and data syncs
with customers
Hello HN, we're Ghalib and Nathan from Polytomic. Today we're
launching our second product, Polytomic Connect:
https://www.polytomic.com/connect (documentation here:
https://apidocs.polytomic.com). Connect is an API you can use in
your own products to either pull your customers' data into your own
systems, or push data from your own systems to your customers', or
both. We have first-class support for data warehouses too. And
there's a self-hosted deployment option. You can set up automatic
data syncs in any direction between data warehouses, databases
(including CDC streaming), cloud applications (e.g. Salesforce,
Zendesk, etc), spreadsheets, cloud storage (S3 etc), and arbitrary
APIs and webhooks. We take care of authentication, automatically
pushing updates, type conversions, rate limits, scaling to handle
large volumes, monitoring, and alerting. You can also sync updates
from custom queries powered by data models you define as well join
data across disparate systems (e.g. HubSpot + Stripe + Airtable) to
sync fields from them in one payload to any destination. You can
see demo videos and code examples for a small subset of use cases
here: https://apidocs.polytomic.com/guides/code-examples/overview.
We'd love to take your comments and feedback! Happy to answer
questions too.
Author : gms
Score : 39 points
Date : 2024-04-30 17:30 UTC (5 hours ago)
| garfieldsam wrote:
| I work at Retool and we're a Polytomic customer with users across
| multiple teams - it's a major part of our data infrastructure.
| The product can handle a lot.
| gms wrote:
| (Polytomic co-founder)
|
| Thanks for the kind words! Pleasure serving you.
| diab0lic wrote:
| Hey Ghalib and Nathan,
|
| Amazing work! Two questions:
|
| 1. What is the best avenue for potential customers to express
| interest in the addition of a new data source or sink?
|
| 2. I assume this runs on Polytomic owned compute? Is there a way
| to bring your own compute, or is it on the roadmap?
|
| Thanks!
| gms wrote:
| Hello! Polytomic co-founder here.
|
| 1. All integrations are built on request (see instruction at
| top of this page to submit one:
| https://www.polytomic.com/integrations). We're able to turn
| them around very quickly (our record is two hours from request
| submission time).
|
| 2. Yes, though you can bring your own compute too. We have a
| self-hosted distribution that you can deploy to your own VPC
| using Docker and our Terraform module (see details here
| https://docs.polytomic.com/docs/on-premise-setup). Customers
| that are public/large enterprises often go for this.
|
| If you want to chat over email shoot me a message at
| ghalib@polytomic.com.
| CapnCrunchie wrote:
| We started using Polytomic for only reverse ETL, but are now
| working to move our ETL processes to it as well.
|
| They've been really great to work with and they've been able to
| handle our data as we've scaled at Vercel.
|
| Congrats on the launch!
| gms wrote:
| Thank you! We're happy to serve.
| wintercarver wrote:
| Data team manager here, qq: how do you handle type conversions
| between different systems? Nice work overall!
| gms wrote:
| (Polytomic co-founder here)
|
| We automatically do the right thing depending on the situation.
| The type engine is highly-precise when we generate our own
| schema. For example, JSON objects and arrays get converted to
| native struct and array types in data warehouses (e.g.
| Databricks, BigQuery) rather than strings.
|
| But we're forgiving in other situations where we don't control
| schemas. For example, we'll automatically convert a source
| string type to a date when mapping to a Salesforce or Zendesk
| (or other cloud app) date field.
| canadiantim wrote:
| What's the ELI5 here?
|
| If I'm pushing and pulling customer's data into my own system
| (e.g. postgres), what sort of customer data are we talking about?
| Can it sync with data on their own filesystem? That would be
| neat. Or it just for e.g. syncing a customer's google drive with
| my postgres type of thing? Looks interesting and like something I
| may want to use but not sure I entirely grok it yet.
| gms wrote:
| We're talking about system-to-system pulls and pushes.
|
| For example, maybe your product needs to process data from your
| customers' sales and customer support systems. In that case you
| can have automatic updates show up in Postgres from your
| customers' Salesforce and Zendesk instances. Does that sense?
|
| Alternatively, many products want to the ability to push data
| to their customers' systems. For example, suppose you're a
| billing software vendor showing data in your own portal that
| customers log in to. Your customers may also want you to
| automatically push this data to their accounting systems or
| data warehouses so that they can do their own analysis.
|
| Does that make sense?
| smashbox96 wrote:
| I work at a venture firm. Polytomic is pretty impressive in its
| functionality. We use it to integrate data providers (Harmonic,
| pitchbook, etc) with our CRM (Affinity).
| gms wrote:
| (Polytomic co-founder)
|
| Thank you, whoever you are! :D Hopefully we can continue
| serving you.
| AniKochiashvili wrote:
| We used Polytomic in my last job to sync data between postgres,
| google sheets and quickbooks. Ghalib and Nathan were helpful in
| figuring out how to make sense of the messy QB data and put a
| meaningful structure around it. They worked with us on building
| out special reports that pulled specific data our team required.
| The support was always outstanding and the product never failed,
| always worked smoothly. Excited to learn about the new launch and
| looking forward to exploring it more!
| gms wrote:
| (Polytomic co-founder)
|
| Thank you for the kind words Ani!
| cryptonector wrote:
| I'm having a hard time finding docs on the ETL itself. I want to
| see how flexible the language is.
| gms wrote:
| What are you looking for exactly? Happy to answer questions or
| point you to the right place.
| cryptonector wrote:
| The language. I want to see it. I want to get an idea of how
| capable it is.
| gms wrote:
| Hopefully the links below are helpful!
|
| Our clients come in multiple languages. You can see the
| list here: https://apidocs.polytomic.com/guides/native-
| clients.
|
| You can see here for sample code:
| https://apidocs.polytomic.com/guides/code-
| examples/overview.
|
| The full API reference is here:
| https://apidocs.polytomic.com/api-reference.
| noashavit wrote:
| I used Polytomic at the last place as a reverse ETL. The team is
| amazing and the product lets you move your data around without
| bugging your developers.
|
| Congrats on the launch!
| gms wrote:
| Wow thank you Noa! Hope you are well!
| noashavit wrote:
| Just giving credit where it is due :-)
___________________________________________________________________
(page generated 2024-04-30 23:00 UTC)