[HN Gopher] Show HN: Vince - A self hosted alternative to Google...
___________________________________________________________________
Show HN: Vince - A self hosted alternative to Google Analytics
For the past 3 years I have been working on a Golang port of
plausible analytics dashboard. vince is a single binary, single
user with multiple website system with zero runtime dependency.
Key featues: - Automatic TLS - Outbounds link tracking - File
downloads tracking - 404 pages tracking - Custom event tracking
And so much more, basically everything that you see on plausible
dashboard except funnels and custom properties. You can use vince
as a drop in replacement for plausible for personal websites. The
goal is to make the plausible dashboard easily accessible for
people who like to self host. All features not related to the
dashboard are non goal, hence not implemented. Full dashboard demo
hosted on 6$ vultr instance
https://demo.vinceanalytics.com/share/vinceanalytics.com/v1/...
Author : gernest
Score : 210 points
Date : 2024-11-29 02:54 UTC (20 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| cebert wrote:
| If you haven't checked it out yet, Serverless Website Analytics,
| is a great solution for this too. It's easy to deploy and very
| inexpensive to run. I've been using it and am quite happy with
| it. https://github.com/rehanvdm/serverless-website-analytics
| gernest wrote:
| Interesting, I just checked the readme. Very similar but looks
| like it only works with AWS and has a lot of moving pieces.
|
| How do you deal with location data, do you purchase maxmind db
| license or use their free versions.
|
| Both maxmind and db-ip free versions of city data miss city geo
| id values, rendering city data useless for many cases.
|
| With vince, I had to index embed the whole city data from
| geonames database to work around this.
| reincoder wrote:
| > How do you deal with location data, do you purchase maxmind
| db license or use their free versions.
|
| > Both maxmind and db-ip free versions of city data miss city
| geo id values, rendering city data useless for many cases.
|
| I work for IPinfo.
|
| I think you might find my conversation with Goatcounter's dev
| interesting: https://github.com/arp242/goatcounter/issues/765
|
| I pitched him to use our free country database because of
| MaxMind's EULA issues. MaxMind does not permit distribution
| of the database and requires end users to use their own
| token. Moreover, they actually charge thousands of dollars
| when you distribute the "free" database with a commercial
| intent.
|
| Now, we have a free IP to Country database that we offer
| under a straight CC-BY-SA 4.0 license without an EULA. It is
| free, comes with daily updates, has full accuracy, and you
| can even commercially redistribute the database (via
| providing us an attribution).
|
| I understand we do not have a free city database to offer,
| nor is our database lightweight because we have full
| accuracy. But you can check it out if you are interested. We
| do have a version with ASN (ISP) information as well.
| 8ig8 wrote:
| Matomo is another one...
|
| https://matomo.org/
| rgbrgb wrote:
| > Full dashboard demo hosted on 6$ vultr instance
| https://demo.vinceanalytics.com/share/vinceanalytics.com/v1/...
|
| 404 page not found
| thangngoc89 wrote:
| I found a link from github
| https://demo.vinceanalytics.com/v1/share/vinceanalytics.com?...
| aaronbrethorst wrote:
| Looks interesting. What sort of memory requirements does it have
| and how does it persist data?
| colesantiago wrote:
| Great project keep it up it's good to see competition in this
| space.
|
| Plausible gets crazy expensive on their hosted option and it
| complex to setup (needs elixir + high memory requirements)
|
| If Vince gets 1:1 parity with plausible and has the option to use
| clickhouse, I'll consider moving a few servers and people I know
| over.
|
| Love that Vince is also a single binary as well.
| just-tom wrote:
| The screenshot on your homepage looks very similar to plausible's
| https://plausible.io/ which is also open-source analytics
| software. Is it based on it? What are the differences?
|
| Edit: Just noticed the feature comparison in the readme.
| dewey wrote:
| Also Plausible is almost stock TailwindUI elements + including
| the default color, so many sites look like that.
| brokegrammer wrote:
| This is amazing! I self host Plausible but don't like depending
| on Clickhouse and Postgres because they're annoying to upgrade.
|
| What kind of database is this using though? I don't know enough
| Go to figure it out from the source.
| tricked wrote:
| I checked the go.mod and it seems to be importing a module
| named pebble by cockroachdb i assume that's where everything is
| stored
|
| https://github.com/cockroachdb/pebble
| akshayshah wrote:
| It uses Pebble, the key-value store that backs CockroachDB.
| colesantiago wrote:
| Just saw this notice:
|
| > WARNING: Pebble may silently corrupt data or behave
| incorrectly if used with a RocksDB database that uses a
| feature Pebble doesn't support. Caveat emptor!
|
| Slightly worrying for now running this in prod if there is a
| risk for silent data corruption, but hopefully in a few years
| Vince would have drivers for Postgres / Clickhouse.
| rickette wrote:
| This just warns about using Pebble with an existing RocksDB
| which isn't the case here. Pebble powers CockroachDB which
| is a Serious Database.
| dangoodmanUT wrote:
| Reread the sentence, it says if you mix it with RocksDB
| (another database that has compatible file formats)
| t0mas88 wrote:
| It says GDPR compliant and no cookies on the project page. How
| are unique visitors calculated? And I'm assuming it can't link
| conversions to campaigns without some cookie-alternative?
| withinboredom wrote:
| No idea, but generally, a bloom filter would get you there
| without any identifying information being stored. The counts
| would merely be estimates at that point, not exact values.
| pdyc wrote:
| most likely through one way ip hashing bounded by time
| duration. If you have utm's in your url than it can track
| otherwise probably not.
| beeb wrote:
| At least for Plausible, they state this
| (https://plausible.io/blog/google-analytics-cookies):
|
| > Instead of tagging users with cookies, we count the number of
| unique IP addresses that accessed your website. Counting IP
| addresses is an old-school method that was used before the
| modern age of JavaScript snippets and tracking cookies.
|
| Since IP addresses are considered personal data under GDPR, we
| anonymize them using a one-way cryptographic hash function.
| This generates a random string of letters and numbers that is
| used to calculate unique visitor numbers for the day. Old salts
| are deleted to avoid the possibility of linking visitor
| information from one day to the next. We never store IP
| addresses in our database or logs.
| chrismorgan wrote:
| > _Since IP addresses are considered personal data under
| GDPR, we anonymize them using a one-way cryptographic hash
| function._
|
| Um... hashing IPv4 addresses, even with salt, does _literally
| nothing_ to anonymise (assuming the output space is at least
| ~32 bits, which I think is safe to assume): they'll still be
| PII. IPv6 addresses I'm not so confident about; _maybe_ it
| would be sufficient for some parts, but it's definitely
| inadequate for some concerns.
|
| (For IPv4, enumerating all four billion inputs is so
| completely practical that "one-way" is nonsense.)
|
| I'm almost certain this is legal theatre.
| gizzlon wrote:
| hm.. are you saying they need scrypt or something similar?
| kadoban wrote:
| The problem, in general with hashing IP addresses
| (especially ipv4) is that there's not that many of them.
|
| If I tell you the value is either 1 or 2, but I hashed it
| with sha256 to make it secure, that's bullshit, right?
| You can just hash both and see which it is.
|
| Same concept applies regardless of the hash algo, and
| still applies if you have more than 2 possible values, 4
| billion or so possible ipv4 addresses is _not_ that many
| values to a computer.
|
| Other common places this problem occurs is with any other
| restricted set of values, eg phone numbers and email
| addresses (most are at like 5 domains and are easy to
| guess/know).
| chrismorgan wrote:
| The "PII" label is taint that is probably impossible to
| dispel completely/perfectly, and difficult to dispel
| sufficiently (and deanonymising is an arms race).
|
| Lossless techniques do _nothing_ to dilute that taint.
|
| Lossy techniques are necessary to get _anywhere_ , such
| as disregarding certain bits of the address, or Bloom
| filters.
| kadoban wrote:
| If what they're doing is using a secure salt and then
| throwing the salt away once a day that _might_ be doing
| something.
| chrismorgan wrote:
| What I understand they're doing is storing the salt in
| one place, a set of hashed IP addresses in another place,
| then daily trashing the lot after counting the number of
| elements in the set and storing that.
|
| Information-theory-wise, this is no different to just
| storing the actual IP addresses (and deleting them daily
| after tallying, as before). It _does_ mean that you need
| to obtain _two_ things instead of just one, but if you
| get access to it all, it's straightforward to reverse the
| lot (though computationally a little expensive), and easy
| to check a single value for a match.
|
| The technique may be considered reasonable effort at
| protecting against casual abuse, but it's not technically
| effective of itself, and it doesn't stop the data from
| being PII. The important aspect is that the PII is
| deleted within 24 hours. My personal opinion is that the
| hashing part should probably be considered snake oil and
| whitewash, at least for what they're claiming--I don't
| say it's useless, but it definitely doesn't do what
| they're touting it for.
|
| Unless they're actually keeping the hashed values for
| some reason after one day, and associating them with
| other records? In which case, disregard _part_ of what I
| say, it's obviously better than persisting IP addresses
| long-term! But also it's extremely dubious to call that
| anonymisation as they do, because you can so often tie
| things together, behavioural patterns and such, to
| deanonymise. It's frighteningly effective.
| tingletech wrote:
| If you throw away the daily random salt (but keep the
| obscured IP address), how can you check a single value
| for a match the next day?
| Semaphor wrote:
| One way if you have a salt? Enumerating won't help, you
| need to know the salt, which gets deleted.
|
| That said, the whole IP thing is weird to me. Not only are
| we allowed to log IPs directly for security reasons, we
| even *have* to log IPs in certain cases (newsletter
| subscriptions).
| kadoban wrote:
| > That said, the whole IP thing is weird to me. Not only
| are we allowed to log IPs directly for security reasons,
| we even _have_ to log IPs in certain cases (newsletter
| subscriptions).
|
| The point of designating something as PII isn't that we
| then _never_ store or use it, it's to carefully consider
| if we actually need it or not (and what protections we
| can add for the values we do need to store/use).
|
| We're meant to stop the practice of just collecting and
| storing all data, without consideration for the harms
| that causes.
| jszymborski wrote:
| What matomo does is mask parts of the IP address (you
| choose how much).
| drchaim wrote:
| this is great, congrats!
| lovegrenoble wrote:
| Is is a Plausible clone? https://plausible.io
| __jonas wrote:
| From the Readme:
|
| > vince started as a Go port of plausible with a focus on self
| hosting.
| samdung wrote:
| This is great. I'm def going to use it.
|
| Minor bug: "See Live Demo Dashboard" url is wrongly pointed.
| pdyc wrote:
| Looks exactly like plausible, may be change the ui a bit to avoid
| legal issues.
| carlosjobim wrote:
| I was going to say that it looks exactly like BeamAnalytics,
| and now I'm confused to who's copying who...
| dewey wrote:
| Because everyone is using:
| https://tailwindui.com/components#product-marketing
| huhtenberg wrote:
| It's not just the looks that are the same. The UX /
| mechanics are way too similar too, e.g. how you can apply
| filters (by URL, by referrer, by browser, etc.) to narrow
| down the stats view.
| rkuodys wrote:
| I would say pretty much the idea is as follow: "Let's do
| it so User would know how to use it before we are big",
| and once you're big enough - you can set the trend. But
| at the beginning it's just not worth it and highly risky
| serial_dev wrote:
| I'm wondering when copying becomes just following industry
| best practices...
|
| Twitter, Threads, Mastodon, Blusky all look the same. Project
| management apps all reuse the same UI patterns. The "AI" logo
| looked pretty much the same for all companies for a while.
| Video sharing websites all use YouTube's layout. Forums like
| Reddit and HN share quite a lot in their looks.
|
| If you want to display website analytics, you will want to
| show the most important metrics at a glance, you'll need
| graphs showing visitors over time, top sources and pages...
| There is only so much you can do to display those and have
| users understand what's going on on your website.
| NelsonMinar wrote:
| What legal issues are you imagining?
| notRobot wrote:
| The dashboard demo isn't working :(
| zoidb wrote:
| My go-to self hosted GA alternative is goatcounter
| https://www.goatcounter.com. It would be interesting to know what
| advantages it has over it.
| james-bcn wrote:
| Oh I like that main dashboard. Very simple.
| TravisPeacock wrote:
| If you like that there is https://www.piratepx.com/ which is
| even more minimal (though less data), I also built something
| even MORE minimal (only API calls)
| https://github.com/teamcoltra/ninjapx but I'm certainly not
| recommending it. It is super simplistic (also the readme is
| embarrassing)
| huhtenberg wrote:
| Does it allow filtering visited page list by a specific
| referrer and vice verse?
| cpursley wrote:
| How would y'all go about building analytics into a professional
| marketplace type of app where you can provide the professional
| with their own profile page stats (in a reliable way)?
| Oras wrote:
| If you don't have plans to offer saas, what are you trying to
| achieve from it?
|
| I mean, it is quite nice to have binary installation hosted on a
| single VPS, but will you support it?
| rrr_oh_man wrote:
| FOSS lives!
| rasso wrote:
| Does this work on your average 10,-/month shared hosting server?
| If so, it might really be ,,for everyone". Otherwise, we are
| stuck with matomo.
| diggan wrote:
| > Does this work on your average 10,-/month shared hosting
| server?
|
| Since they usually offer software via cPanel and alike, seems
| unlikely unless you give it lots of time for the project to
| first get popular enough to get on the "admin panels" mind, and
| secondly for them to integrate it.
|
| Besides, do people really pay 10 USD/month for shared hosting?
| Sounds really expensive when you can grab VPSes for half that
| price and run whatever software you want, not just what they've
| packaged for you. I guess ongoing maintainace is included in
| that price, but still sounds kind of expensive for what you
| get.
| rasso wrote:
| I don't know... around here (Germany), that's pretty common.
| No need to manage anything, no usage-based cost, ... my
| favourite is https://all-inkl.com. OG no-bs hosting for
| boring tech.
| manishsharan wrote:
| I think the reason some of us continue using Google Analytics is
| its demographic data. That information is not available elsewhere
| as far as I know , which I admit is not a lot.
| paradite wrote:
| Not sure why I would use this over Plausible CE on docker. Does
| it consume less memory/CPU?
|
| Also I am pretty sure Plausible CE doesn't limit number of sites
| / events, unlike what's listed in "Comparison with Plausible
| Analytics".
| written-beyond wrote:
| Code quality is pristine, really great job! I see that you've
| used protocol buffers, can you expand on why? I am aware of the
| benefits it offers but I think it adds a bit of mental overhead
| initially due to it being an additional type system you have to
| understand.
|
| Also why are you using pebble exactly? I was interested in seeing
| how you're managing your geo databases because that's usually the
| most mind numbing part of handling analytics if your cloud
| provider doesn't add that information into the request header
| already. However, I can't understand why you'd use pebble over
| something like sqlite.
| gernest wrote:
| Thanks,
|
| > Why protocol buffers ?
|
| They are very good for defining API boundaries, in vince we
| only use them for configuration and admin structure. We use
| Roaring Bitmap based storage, so fundamental units persisted
| are Bitmap containers.
|
| > Also why are you using pebble exactly?
|
| Well, vince is write heavy and any LSM based key value store
| would have been nice. It happens pebble is the best option for
| us.
|
| Also, we don't use transactions (We batch writes and use
| snapshots for reads). Combining with the fact we rely on pebble
| batch Merge api.
|
| The merge api allows us to do efficient updates. Since we only
| store bitmap containers, when doing update we just do a
| container union of observed values of a key.
|
| Bitmap unions are pretty fast and efficient.
|
| I hope I covered all your questions.
| written-beyond wrote:
| It answered them alright, but really opened a few hundred
| more. I appreciate your time!
| skeptrune wrote:
| Cool that there are so many of these now. Currently self hosting
| plausible and it does seem quite barebones. Will have to give
| this a shot!
| gonafr wrote:
| How this compares to umami (https://umami.is/)?
| vextea wrote:
| There seems to be some mentions of selling licenses (and pricing)
| in the source. What are the plans around that?
|
| https://github.com/vinceanalytics/vince/blob/f0c2c3cc38cbd8c...
| gernest wrote:
| When I started working on vince, I thought I could bootstrap a
| sustainable business, that was about 3 years ago.
|
| My dream for a business is practically dead now. That snippet
| is a relic of early days of vince and I will remove it.
|
| I am currently looking for work, and will be maintaining vince
| as usual (I do a lot of open source stuff) since I also use it
| with my hobby projects.
|
| I'm struggling finding remote roles now, since remote now means
| Remote US or Remote EU and I'm stuck here in Tanzania.
|
| So, don't worry, I also use vince so I will keep hacking on it.
| vextea wrote:
| Makes sense, wish you the best of luck!
| sira04 wrote:
| Looks great!
|
| I found a small bug, if you click Expand in the Top Pages
| section, the Time on Page column has NaNs.
|
| Dark mode for the dashboard and showing realtime current visitors
| in the <title> would be great.
| cchance wrote:
| "see live dashboard" button on main page just... goes to the top
| of the page lol
___________________________________________________________________
(page generated 2024-11-29 23:00 UTC)