[HN Gopher] Hacking on PostgreSQL is hard
___________________________________________________________________
Hacking on PostgreSQL is hard
Author : fletchr
Score : 105 points
Date : 2024-05-02 00:25 UTC (22 hours ago)
(HTM) web link (rhaas.blogspot.com)
(TXT) w3m dump (rhaas.blogspot.com)
| dathinab wrote:
| through writing extensions luckily isn't
| hinkley wrote:
| So is the path to create an extension and hope that someone
| with a deeper understanding can pull the main features into
| postgres major_version+1?
| dathinab wrote:
| > deeper understanding can pull the main features into
| postgres major_version+1?
|
| for a lot of stuff you don't need it, sometimes even don't
| want it
| paulddraper wrote:
| Many features should only be extensions.
| Deeg9rie9usi wrote:
| Original submission:
| https://news.ycombinator.com/item?id=40234398
| dang wrote:
| OP was original submission--you can tell this because its ID is
| smaller.
|
| Timestamps get relativized when we re-up a post [1]. That's an
| artifact of HN's re-upping system [2].
|
| [1]
| https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
|
| [2] https://news.ycombinator.com/item?id=26998308
|
| Sorry for the confusion--I know it's weird but the alternative
| turns out to be even more confusing and we've never figured out
| how to square that circle!
| NomDePlum wrote:
| The most enlightening part of the article was in the comments
| where it was observed that given the length of time Postgres has
| been going, the number of talented developers who have worked on
| the project there are no easy enhancements or fixes left, just
| hard knotty problems. (paraphrasing).
|
| It's almost a mark of success of the project. There is obviously
| a lot of dedication too.
| MBlume wrote:
| Coming at this from a naive outsider perspective, the central
| problem described in the post (commits to PostgreSQL frequently
| have serious defects which must be addressed in follow-up
| commits) seems like one that would ideally be addressed with
| automated testing and CI tooling. What kind of testing does the
| Postgres project have? Are there tests which must pass _before_ a
| commit can be integrated in the main branch? Are there tests that
| are only run nightly? Is most core functionality covered by
| quick-running unit tests, or are there significant pieces which
| can only be tested by hours-long integration tests? How expensive
| is it, in machine-hours, to run the full test suite, and how
| often is this done? What kinds of requirements are in place for
| including new tests with new code?
| bedman12345 wrote:
| Most database companies run only a small amount of tests before
| committing. After committing, you run tests for thousands of
| hours. It sucks. You probably do this all day every day. You
| just run the tests on whatever you have currently committed.
| you kind of have to be careful about not adding more tests that
| make it take much much longer. See
| https://news.ycombinator.com/item?id=18442941
| kapilvt wrote:
| I would also note that the fix prs started landing the day
| after the initial commit, and other issues noted had fixes
| within three weeks. And of course postgresql has testing, but
| at universal distribution and use cases on things that will
| test both scheduler, network, fs, io drivers (Linux kernel,
| postgresql, etc, among others), some things need wider
| audiences or more extreme testing scenarios (SQLite for a
| strict subset of those considerations), and project health is
| measured by responding to that in a timely fashion. Afaics this
| is all about trunk/main, versus releases as well. So while its
| labeled its hard on the post (from a long time pg contributor),
| and yeah i might agree (cause maintainer on other software, so
| yeah all this resonates heavily), I'd also say its an example
| of things done right.
|
| Seems like a reason to celebrate the open source model, and
| specifically here on how to do things better. Not to detract
| from universal issues for any project on maintainer
| availability. But, imagine a non oss database vendor with that
| degree of transparency or velocity, i can't think of any that
| are doing anything close unless they got popped on a remote
| cve, aka prioritized above features or politics on a corporate
| dev sprint. Aka all software has bugs, it's about how fast
| things are fixed, and in the context of oss imho fostering
| evolution among a diverse set of maintainers and use cases
| seems to be a better way.
|
| As another example of that, 'twas a PostgreSQL hacker at MS,
| that prevented Libxz from going wide because of caring due to
| perf regression and doing the analysis.
| pstuart wrote:
| It seems like having a more robust testing framework might help,
| akin to what SQLite uses?
|
| Obviously easier said than done, but the way he presents things
| it seems kind of like a push and pray environment.
| DowagerDave wrote:
| I don't think that's fair. The PostgreSQL codebase includes a
| lot of stuff that isn't included in SQLite, where it's covered
| by 3rd party projects - I'd bet their quality is nothing like
| SQLite or PostgreSQL. Tom Lane was involved in these commits
| and sounds like some of this got by him. His comments have
| frequently been described as "complete technical manuals", so I
| think that speaks to the complexity.
| swader999 wrote:
| PostgreSQL has different execution paths depending on load,
| table size, hardware etc which also complicates testing and
| test setup.
| paulddraper wrote:
| PostgreSQL is an order of magnitude more complex and extensible
| than SQLite.
|
| Any comparison is basic at best.
| mannyv wrote:
| TL; DR: testing is hard, and developers aren't very good at it.
|
| It also sounds like internal documentation may be lacking, which
| isn't surprising.
| hyperman1 wrote:
| What's missing in this post is a deep analysis of what the bugs
| are and what was causing them, in a 5 times why sense. Especially
| if they all seem dumb stuff at first.
|
| There are some deep lessons about programming in this Factorio
| Friday Facts:
|
| https://factorio.com/blog/post/fff-366
|
| and I wonder if postgres doesn't look like fig.1 from this blog
| post, before the refactoring.
| api wrote:
| The "is hard" cliche is really getting on my nerves.
|
| I guess article titles are hard.
| skywhopper wrote:
| lol, of course the first comment is essentially "it would
| probably be easier if it were written in Rust".
| influx wrote:
| Just make it part of systemd, problem solved.
| avi_vallarapu wrote:
| A few things to note
|
| - Postgres documentation is one of the well maintained database
| documentations. This also means that developers, committers
| ensure changes to documentations for every relevant patch.
|
| - talk about bugs in postgres compared to MySQl or Oracle or etc
| databases. Nugs are comparatively lesser or generally rare even
| if you are supporting postgres services as a vendor with lots of
| customer. the reason is the efforts involved by a strong team of
| developers in not accepting anything and everything, there are
| strict best practices, reviews, discussions, tests, and a lot
| more that makes it difficult to pass to a release.
|
| - ultimately, more easy is the acceptance of a patch, more the
| number of bugs.
|
| I love Postgres the way it is today and it still is the dbms of
| the year and developers most loved database.
|
| I wish we have more Contributors committers, developers and also
| users and companies supporting Postgres so that the time to push
| a feature gets more faster and reasonable easier with more
| support.
___________________________________________________________________
(page generated 2024-05-02 23:00 UTC)