[HN Gopher] Yandex open sourced it's BI tool DataLens
___________________________________________________________________
Yandex open sourced it's BI tool DataLens
Author : SergeAx
Score : 185 points
Date : 2023-09-26 11:36 UTC (11 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| cvalka wrote:
| This is awesome! Let's not forget about their second generation
| SQL database YDB which is an open source alternative to TiDB and
| YugabyteDB.
| wutwutwat wrote:
| Yandex open sourced all of its code a few months back when it was
| all stolen and leaked.
| 6c696e7578 wrote:
| Didn't the CEO flee due to the war?
|
| https://www.intellinews.com/russian-tech-titan-yandex-ceo-vo...
| wutwutwat wrote:
| I would have no idea. Why are you asking me and sharing a
| link to something which tells you. That's weird, and I'm not
| interested in clicking the link anyway.
| RedShift1 wrote:
| Any screenshots of how it looks?
| capableweb wrote:
| The website has screenshots: https://datalens.tech/
| RedShift1 wrote:
| Ah thanks, I was browsing through the github repos.
| kasperni wrote:
| https://cloud.yandex.com/en/services/datalens
| londons_explore wrote:
| Yandex makes some pretty cool tech- they clearly have a lot of
| smart engineers.
|
| It's a shame that geopolitics means most of it will have to be
| reinvented by someone else before it'll see any use.
| [deleted]
| efxhoy wrote:
| Yeah. I'd love to use clickhouse but yandex ties to the russian
| government makes me not want to.
| kmac_ wrote:
| Not only you. The company that I work for delivers software
| to government agencies. Dependencies like libraries or stack
| parts that are Russian or Chinese are simply banned. Even
| Harbor was rejected (and probably should be, as they provide
| a bunch of their own built Docker images, so you never know
| what your stack runs). I know "but it's open source" etc.,
| and somebody may feel offended, but that's reality.
| mahoro wrote:
| ClickHouse is a separate company registered in US and some
| part of it is owned by US-based venture funds.
| cpursley wrote:
| Isn't it all open-source?
| Dah00n wrote:
| Some projects have refused code from developers because
| they are Russian or Chinese. Just because it is open source
| doesn't mean it is unbiased, free from politics, and
| without racism.
| [deleted]
| slt2021 wrote:
| some projects racially discriminate developers to avoid
| possible bias, politics, and racism? lolz
| londons_explore wrote:
| They would argue they are discriminating based on
| nationality not race.
|
| I can't see that argument working in a court of law...
| neoromantique wrote:
| >I can't see that argument working in a court of law...
|
| Don't US(or any other country for that matter) laws
| discriminate based on nationality all the time?
| zX41ZdbW wrote:
| ClickHouse was open-sourced in 2016 and moved to an
| independent company in 2021, with no ties remaining with
| Russia or Yandex. Read more: https://clickhouse.com/blog/we-
| stand-with-ukraine/
| Silasdev wrote:
| Yes, that is really a shame, although it's been fully open
| sourced.
|
| Yandex is aware of how the geopolitical situation is hurting
| them and are therefore building a new company called
| Double.Cloud, based in Europe, to work around the negative
| public opinion on Yandex, and thereby continue being able to
| sell Clickhouse cloud services.
| Eumenes wrote:
| Its open source, who care what longitude and latitude the
| steward of the project is physically based in ... copy the
| code and make your own thing up
| mobileexpert wrote:
| Cool. Rolling your own BI seems fraught with peril at most orgs
| where I imagine the buy vs build decision is always buy. How many
| PowerBI or Tableau seats do you need before rolling your own
| internal BI platform starts to make sense?
| htrp wrote:
| I don't think it ever makes sense because the large players
| will always be able to make new innovations (mobile apps,
| natural language querying, SSO integrations, etc) that, short
| of large corps hiring BI teams to invest in the open source
| ecosystem like superset, your open source solution won't have.
| mritchie712 wrote:
| Yeah, unless your at unicorn scale, I don't see how self-
| hosting BI would ever make sense relative to other
| investments you could make.
| totalhack wrote:
| I must like to live dangerously. In all seriousness though
| there are low cost alternatives to those mega BI tools that
| suit many use cases. If I wasn't rolling my own I'd probably
| start with Metabase or Superset. What I use:
| https://github.com/totalhack/zillion
| totalhack wrote:
| Looks pretty neat. Having not fully investigated it yet, I will
| say the one thing I usually run into with this and other BI tools
| is a lack of flexibility in the UI for forming queries. It's
| sometimes limited to one table or view at a time. I wonder why
| more of them don't use more flexible querying techniques, perhaps
| just due to the risk of a bad query being formed?
|
| My preferred approach is implemented in Zillion, which I use for
| BI at my company: https://github.com/totalhack/zillion
| dwheeler wrote:
| Nit on title:
|
| it's => its
| pklausler wrote:
| Just be glad it wasn't " its' " (sic), which has been showing
| up more and more in my input streams.
| riffic wrote:
| glaring misuse of _it 's_ in the title
| Logans_Run wrote:
| Well played ;-)
|
| ps - I spotted the pedant that is technically correct, and I
| claim my 5 McFun bucks!
| [deleted]
| xnx wrote:
| Such a huge landscape of BI and data visualization tools. Are
| there any clear open source leaders? Apache Superset?
| RedShift1 wrote:
| Right now I'm heavily into Grafana but when it comes to BI it
| kinda falls flat, I regularly have to fall back to using the
| Plotly plugin to create the charts (but it's getting better, at
| least you can do a normal scatter or bar chart out of the box
| since version 8. Labeling the axes is still a problem though).
| Navigation is also problematic, like jumping to a source table
| takes a lot of effort to make (basically you have to create a
| new dashboard and do some hyperlinking instead of there being a
| ready to go "view source data" button). I feel like there's a
| lot of friction to get Grafana to do BI, but I've also invested
| so much time in it I'm afraid to jump ship...
| noughtme wrote:
| Also Metabase, which I found easier to deploy and use.
| dacort wrote:
| Metabase 47 also has new serialization features in paid
| editions that allow for git-based workflows.
| https://www.metabase.com/learn/administration/git-based-
| work...
| amcaskill wrote:
| Superset and metabase are definitely the OSS leaders.
|
| I work on an open source code-based BI tool called Evidence,
| which might be of interest to you.
|
| It's effectively a static site generator aimed at building
| automated reports and analysis.
|
| https://github.com/evidence-dev/evidence
|
| Previous discussions on HN:
|
| https://news.ycombinator.com/item?id=28304781 - 91 comments
|
| https://news.ycombinator.com/item?id=35645464 - 97 comments
| cvalka wrote:
| There's redash as well.
| dingdong33 wrote:
| It is a dead project
| antman wrote:
| Seems to be getting updates, last commit 5 days ago
| amcaskill wrote:
| Databricks acquired it if I recall correctly.. so maybe.
| FridgeSeal wrote:
| I'd advise metabase over superset.
|
| Superset looked good, but operating superset quickly runs into
| the same Python issues all Python software suffers from.
|
| Sometimes it would just _break_ for no apparent reason.
| Configuring it was a nightmare of magic Python code and unclear
| settings. Trying to use plugins was equally painful: due to the
| poor boundary separating the applications dependencies from the
| plugins dependencies, adding a db connector could just bork the
| whole application.
| PetahNZ wrote:
| Or like not being able to delete a user without running some
| SQL:
|
| https://github.com/apache/superset/issues/13345
|
| Almost instantly run into this issue setting up a test
| instance of Superset. And the issue has been around for
| years.
| xnx wrote:
| Very sound advice. I started the setup process for Superset,
| and it's even worse on Windows. Contrast that to Superset
| which is a single .jar file and worked instantly.
| doctorpangloss wrote:
| What do people use for an "analytics.js" for reporting events
| with common items like campaign data, user device and user
| profile measurements, and related from browsers and devices?
| vgt wrote:
| I'm Ukrainian
| felixhummel wrote:
| Apache 2.0 licensed (from a cursory glance at the first few
| repos).
| lasermike026 wrote:
| I'm giving up Power BI and I'm moving to Domo. I look at rolling
| my own occasionally.
| RedShift1 wrote:
| If you want to roll your own, maybe have a look at Plotly's
| Dash?
| chatmasta wrote:
| Observable Plot [0] is also nice. AFAIU it's the same library
| powering the visualizations within Observable itself.
|
| [0] https://observablehq.com/plot/
| mritchie712 wrote:
| (I'm the founder of a competitor to Domo)
|
| I really like the concept of Domo. They have ETL, modeling, a
| warehouse and BI in one app ("data-stack-in-a-box"). I've
| interviewed 20 of their customers and the general sentiment was
| pretty bad. There's a long sales process, a longer process to
| get it set it up, and they've built all the modeling and
| connectors themselves (vendor lock in, none are best-in-class).
|
| Definite (https://www.definite.app/) is a data-stack-in-a-box.
| We have a built-in modeling layer for core metrics and an AI
| assistant to answer any one-off questions.
|
| A few ways we're different:
|
| Built on open source - We run the data stack for you and give
| you a single app to manage and analyze your data, but it's all
| built on open source standards. So if you decide at any point
| you want to run it all yourself, the code is yours to lift and
| shift to your own infrastructure.
|
| Battle tested connectors - We're using Meltano / Singer (open
| source library from Stitch) for our connectors, so they've been
| used heavily in production for years.
|
| Self-serve that actually works - A lot of tools promise self-
| serve, but AI is making this real. We've invested heavily in
| making it possible to ask questions and get accurate answers.
| The AI queries a modeled view of your data that can answer
| questions that depend on well defined metrics (e.g. ARR, DAU,
| etc.).
| danr4 wrote:
| all that text and no pricing page. shame.
| mritchie712 wrote:
| Fair point! We have a free plan (capped at 3 users) and
| paid plans (which add ETL and a data warehouse) start at
| $500 a month.
| _boffin_ wrote:
| We have Domo at work and it just seems overly complicated and
| insane. I'm wanting to learn it instead of just polling data
| sources and adding them to a local Opensearch instance, but...
| too verbose for me.
| anonyfox wrote:
| Since the discussion started in the comments already, I have a
| similar question: any recommendations for a solution (don't care
| if OSS or not) that has the best UX for nontechnical people to
| assemble some data and reports anyhow? I have Salesforce, some
| mariadb/postgres and (optionally) hubspot as data sources.
|
| I can buy or manually provision anything, no technical hurdles or
| policies from that side. My absolute focus is the raw UX for
| business people.
|
| Suggestions?
| mritchie712 wrote:
| I'm the founder of Definite (https://www.definite.app/). We do
| ETL, modeling, storage and BI in one app ("data-stack-in-a-
| box").
|
| > has the best UX for nontechnical people to assemble some data
|
| If they can use Excel / pivot tables, they can use Definite.
| They can also just ask in natural language and we generate the
| report for them.
|
| > Salesforce, some mariadb/postgres and (optionally) hubspot as
| data sources
|
| We have pipelines for all of these and can spin up a managed
| data warehouse to store all the data if you don't already have
| one.
|
| Drop me a note at mike@definite.app if you're interested
| arthurwu wrote:
| I'm a co-founder of Dataland.io where we're building a powerful
| dataset viewer + search engine that can work on top of your
| Postgres or data warehouse.
|
| We designed it specifically to provide an excellent UX to
| business users while reducing BI burden on the data team. We
| find that most business users often just need to search,
| filter, and sort instead of looking at charts to make
| operational decisions.
|
| UX-wise, what sets us apart are:
|
| - <1s full-text search (even on billions of rows of data),
| feels like Cmd+F in Google sheets, but faster
|
| - Performance: we stream billions of rows into the web browser,
| seamless scrolling (no paging of 50 records at a tieme)
|
| - Rich cells make tables easier to scan/read (enum strings =>
| colored tags, numbers => color-coded based on value =>
| checkboxes, timestamptz => clear date time pills)
|
| If that fits what you need, happy to give you a demo.
|
| arthur(at)dataland.io
|
| Otherwise, I think the simplest BI (if charts are impt) could
| be something like evidence.dev or Metabase.
|
| But I also think it's going to require some curation on your
| part. Can you reasonably expect business users to navigate the
| entire schema/table tree across these three sources? That's
| where I think the bottleneck often lies -- if your BI tool
| allows engineering to just expose a subset of curated core
| tables.
| tillvz wrote:
| Veezoo (https://www.veezoo.com) is built to make it as easy as
| possible for nontechnical users to get answers to their ad-hoc
| questions.
|
| Follows a conversational "ChatGPT-like" approach since already
| 2016.
|
| Info: I'm one of the founders.
| tomschwiha wrote:
| I dislike about your pricing that it tells me reasonable 29$
| and then in the fineprint it says minimum 5 users. I get the
| reasoning behind your pricing logic, but I really dislike it.
| Now as solo business owner I'm gone.
| tillvz wrote:
| If you have a single data source that you'd like to use you
| can even use it for free up to 5 users.
| robertlagrant wrote:
| The problem is that developing the perfect UI for nontechnical
| people to assemble reports probably requires a bespoke frontend
| for your business, and one that likely lags behind the reality
| of its changes. Most businesses instead opt to just hire
| semitechnical people that can do a bit of data work and give
| answers to the report-writers, as they can accommodate business
| changes over time and understand how to construct new queries
| out of the overall business' data sources.
|
| Maybe that'll change one day with AI, and when it does that
| will be bought by every big company in the world (-:
| [deleted]
| davidarenas wrote:
| Honestly Metabase has given best balance between allowing non-
| technical users to self-service and technical users to dig in
| and use raw sql if that's what they want. Also it OSS core so
| you can self host. It is super feature rich and has most
| everything in the OSS version as long as you don't need
| enterprise features like SAML auth, audit log, ...,etc
| FridgeSeal wrote:
| Came here to say this as well.
|
| Metabase is the only tool I've used where I've managed to get
| non-technical users to actually engage and use to query
| building tools to answer their own questions.
| Aaronstotle wrote:
| If you want a spreasheet interface for business users, Sigma
| Computing (https://www.sigmacomputing.com/)
| esotericimpl wrote:
| [dead]
___________________________________________________________________
(page generated 2023-09-26 23:01 UTC)