[HN Gopher] Ask HN: In which areas have you compared 3+ tools an...
___________________________________________________________________
Ask HN: In which areas have you compared 3+ tools and formed strong
preferences?
For example: I've used 3+ code editors on MacOS and prefer sublime
text over VScode, coteditor, xcode. I've used Chrome, Firefox and
Safari all within the past year, and I prefer Chrome and Safari.
I've used 3+ voice transcription apps, the ones I prefer are 'just
press record' and Otter.ai, I can't remember the names of the
others I used - but I downloaded a bunch of Whisper based ones and
non-whisper based ones on the iOS app store. I've used 3+
messaging apps, I like iMessage and Telegram, I prefer those over
Signal and WhatsApp. I've used 3+ interfaces for GPT, and I prefer
the OpenAI playground over ChatGPT, chatbot-ui, typingmind,
openplayground, Poe, and a few others I can't remember. I think
it's important to note that for me these are all preferences. I'm
not saying one of these tools are objectively better. I am saying
that for me they are better, and I prefer them. What are areas of
products, tools, developer tools, APIs, anything, where you've
personally used 3+ tools, and what did you prefer, and what did you
not prefer, if you can still remember?
Author : tikkun
Score : 81 points
Date : 2023-08-01 16:43 UTC (6 hours ago)
| sonofhans wrote:
| Web browser: on the desktop, Safari still smokes everything else
| for speed, efficiency, and ergonomics.
|
| Text editor, desktop: vim, still. I try new ones all the time.
| Vim is still most efficient at getting my thoughts out,
| particularly in editing.
|
| Text editor, iPad: IAWriter. Very simple and clean markdown
| editor. I tried a _ton_ of these things, and I love IAWriter. You
| point it to a cloud storage folder.
| koliber wrote:
| Note taking apps.
|
| I've used Evernote, OneNote, Apple Notes, and Notion.
|
| I prefer Notion hands down.
|
| I dislike OneNote. Poor syncing. Hard to keep formatting
| consistent. Can't link to notes easily. Web app is inferior to
| installed app.
|
| I used to love Evernote, but then it got slow, which led me to
| eval the others. This ultimately led me to Notion.
| nkjnlknlk wrote:
| Out of curiosity have you ever looked into Obsidian? Not that I
| think it's a replacement or superior but I feel that it's in
| the same space.
| tikkun wrote:
| Not GP commenter, but I've used Apple Notes, emacs org-mode,
| Evernote, Notion, Coda, Obsidian, and Google Docs. I prefer
| Apple Notes for personal notes, and Google Docs for shared
| notes, and Notion for ones that I publish as web pages.
| tomComb wrote:
| I liked notion but then it too got slow as it added enterprise
| features.
| euroderf wrote:
| Notion introduced (me to) the idea of slash commands and block-
| based editing. Now there's many others.
| agilob wrote:
| Performance testing. I'm familiar with gatling, jmeter, locust,
| but k6 is definitely the winner. Take locust if you strongly
| prefer Python over JS, TS or Go. If no preference towards Python,
| k6 is your answer.
| grrdotcloud wrote:
| The majority of software I have come across can be put into three
| categories:
|
| The first is the 'de facto', which are widely adopted for various
| reasons, mostly marketing, market share, and marketing.
|
| The second is the "alternative" options that serve as
| alternatives to the previously mentioned choices.
|
| The remaining software falls into the third category, where the
| focus is on implementing improvements to the other two
| categories.
| thih9 wrote:
| What are the benefits of categorizing software this way? Does
| this help in forming preferences?
| justinlloyd wrote:
| Databases: PostgreSQL, mySQL, MongoDB, DynamoDB, SQL
| Server, SQLite. Preference: SQL Server for big stuff,
| SQLite for small stuff or prototyping. Editors:
| Notepad++ (Linux), TextEdit (macOS), Sublime (macOS, Windows,
| Linux), Nano, Vi, Vim Preference: Sublime followed by
| vi/vim if stuck in a shell. File Finders:
| PathFinder (macOS), Directory Opus, Finder, Windows Explorer (10
| & 11), and a few different ones on Linux. Preference:
| Directory Opus on Windows, PathFinder on Mac Desktop
| Operating Systems: Windows 10 Pro, Windows 11 Pro, latest
| macOS, Ubuntu, CentOS, FreeBSD. Preference: Windows 10
| Pro, followed by macOS, and Linux a distant 3rd.
| Desktop user interfaces: Windows 10 Pro, Windows 11 Pro,
| latest macOS, KDE, GNOME, couple of others in Linux.
| Preference: Windows 10 Pro, followed by macOS, and Linux a
| distant 3rd. Calculators: Speedcrunch,
| OpalCalc, SoulVer, Windows Calculator, bc. Preference:
| SoulVer, OpalCalc, Speedcrunch. Honorable mention:
| RealCalc on mobile. One-to-one messaging:
| Telegram, WhatsApp, iMessage, SMS, Google Messages
| Preferemce: Telegram Group or work messaging:
| Slack, Discord, Teams Preference: Teams, then Slack.
| Version Control: Mercurial, SVN, CVS, Git, Perforce,
| AlienBrain (defunct now) Preference: Perforce followed by
| AlienBrain followed by Mercucrial DVCS repo websites:
| Gitlab, self-hosted gitlab, Perforce self-hosted, Perforce
| hosted, github Preference: Perforce self-hosted, followed
| by github. VM hosts: VMWare Workstation,
| VMWare Fusion, VirtualBox, ProxMox, VMWare ESXi
| Preference: VMWare Workstation followed by ESXi.
| Chrome, Firefox, Edge, Safari, Brave. Preference: Firefox &
| Firefox mobile, Chrome Search Engines: Kagi,
| Google, DuckDuckGo, Bing, a few others. Preference: Kagi
| PDF Readers: Chrome, Firefox, Adobe, Foxit, PDF Reader
| Pro, Edge Preference: Edge Languages (only
| those I've used in the past 2 years): C, C++, C#, ARM
| Assembly, Rust, Python, Ruby, Typescript, JavaScript,
| VisualBASIC. Preference: Typescript, Ruby, C#, Python
| (and I never thought I'd ever put a "scripting" language before
| my beloved C/C++/C# languages) Coffee:
| Starbucks, Trader Joe's, Illy, Groundworks Preference:
| Groundworks, with Illy in a pinch. Shells:
| fish, zsh, bash, csh, few others Preference: Fish and fish
| in Bash compatibility mode. Code Editors (just those
| in the past 2 years): Jetbrains (pycharm, rubymine,
| intellij, dataspell, Rider, etc, etc), VSCode (Linux, macOS,
| Windows), Visual Studio 2017 & 2019 & 2022, XCode
| Preference: Visual Studio 2022 with latest intellicode disabled,
| followed by Jetbrains Clouds: AWS, Azure,
| GCP Preference: AWS
|
| Now watch all the Linux lovers/Windows haters down vote personal
| preferences.
| TonyStr wrote:
| VS22 > jetbrains is surprising to me. Care to share why? I'm
| also curious why Linux is so far down and windows is your
| favourite.
| cadr wrote:
| Why SQL Server?
| flir wrote:
| I'm not saying you're wrong, but I bet you're the only person
| on HN with a preference for Teams over, say, semaphore flags.
| alex_lav wrote:
| Cloudformation, SAM, Terraform, AzureRM. Terraform by miles and
| miles, then SAM. I mostly won't work in CF anymore.
|
| Rider, Visual Studio, VSCode. Just use Rider.
|
| AWS, GCP, Azure. Never Azure. The rest doesn't matter.
|
| A more obscure ref but Airbyte, Fivetran, Hevo. Totally avoid
| Airbyte and Hevo. Begrudgingly use Fivetran.
| zxexz wrote:
| The below is obviously my subjective opinion.
|
| Operating systems: I mostly use Arch Linux personally. For work,
| though, I think it's hard to beat everyone in the organization
| being on macOS.
|
| Databases: IMO, never use anything except for Postgres for OLTP
| needs. Also, you don't need a document store - you need Postgres
| (JSONB works well if you really need to store unstructured JSON).
| If you need to search, Elasticsearch/Opensearch just...works
| well. Never use MongoDB, ever. Even if it was reliable, easy to
| maintain and reason about, why do you need it? If you need a
| lightweight local database with OLTP capabilities for managing
| application state, use sqlite3. K/V store? Redis. Never had a
| problem using Redis at massive scale, just be smart about what
| you're putting there. Also works well as a message queue and
| pub/sub (Postgres actually works well here too). I've used
| Celery, ZeroMQ, and Kafka - they all work well too, no strong
| opinions about which one.
|
| ETL/Data Processing/Workflow management: Lots of hate for airflow
| in this thread. I don't think it's as bad as people are saying it
| is, but I moved to Dagster and am definitely never going back to
| Airflow. Dagster takes a few hours to grok, but then it just
| makes _sense_ , and is a lot easier to reason about. General data
| processing - a combination of polars, pyarrow, and ray. Store
| streamable source data in parquet, stream it to arrow while doing
| initial conversions. Memory map the files, process them in polars
| - distributed amongst ray workers, ideally abstracted through
| Dagster. Sometimes you have an existing data pipeline with some
| steps in spark. That's fine - orchestrate those tasks in dagster!
|
| IDEs: I've tried them all. Nothing beats the JetBrains stack with
| the IdeaVim plugin - I end up using almost all the tools -
| PyCharm, IntelliJ, GoLand, CLion, DataGrip, WebStorm, and a few
| times I've even used PHPStorm and MPS as well. I keep VSCode
| around for...text editing mostly. Most of my bash scripts and
| config management is done through vim (vim is also the only good
| Haskell IDE I've found).
| maccard wrote:
| > Nothing beats the JetBrains stack with the IdeaVim plugin > I
| keep VSCode around for...text editing mostly.
|
| Couldn't agree more. I would like an alternative to VSCode for
| text editing/viewing that has fast startup, basic syntax
| highlighting, tabs, and an auto-saving "unnamed files" feature,
| but a license for Sublime Text feels way overkill for what
| amounts to the equivalent of a throwaway notepad
| dizhn wrote:
| Autosaving unnamed files is for some reason a feature not
| many tools implement. I think I've first seen it in Notepad++
| and it just makes so much sense. The only thing I found that
| works like that without getting into sessions etc is
| Mousepad. It's also very fast to start. Don't know about
| highlighting. It probably doesn't exist. I would also give
| Kate a look. It's more capable than it looks at first and is
| my current note taking solution with a nested folder
| structure with md files. (It has recursive folder search and
| tree view of the folders. All I need)
| jonathan-adly wrote:
| I am used 3+ Vector Databases (Pinecone, qdrant, and pgvector). I
| prefer Pgvector by miles.
| esafak wrote:
| that's bizarre. What's your QPS and corpus size?
| jonathan-adly wrote:
| It's slower, but once you are out of the toy project stage
| and you are trying to sync 2 databases - where relations get
| changed and logic gets updated, it is really hard to keep
| multiple DBs synced.
|
| Factor it dev vs. production DB setups, on boarding new
| developers and growing pains of a startup and having one DB
| as source of truth with a small performance penalty (and a
| large cost savings) - it's a no-brainer.
| fzliu wrote:
| What kind of queries over structured data do you use
| Postgres for? We're looking to see how we should broaden
| our metadata filtering to potentially support usage
| patterns for developers like yourself.
| TylerE wrote:
| I've said it before, and I'll say it again. @dang Can we please
| ditch this very hard to read light grey OP text.
|
| #828282 on #f6f6ef is only a 3.54:1 contrast ratio, well below
| the WCAG's 4.5:1 minimum for body text. Even on a pure white
| background it's only 3.84:1.
| jdjdjdhhd wrote:
| Also get rid of near invisible text for dead comments since you
| have to opt in in seeing them
| folkhack wrote:
| 100% behind this change as well.
|
| I get the reluctance to change HN and appreciate the
| consistency. But, in good faith, this one is a clear
| accessibility issue that needs to be addressed. I think a
| change to just get the contrast to minimal best-
| practices/standards would't compromise the platform.
|
| Devil's advocate against myself for a sec: HN is a site largely
| used by engineers/technical professionals. We can be an
| insufferable bunch, _especially_ in regards to change. Maybe
| the social fallout isn 't worth the change and I just would't
| have the data/context to understand that...? Just throwing it
| out there. I may just not understand.
| mock-possum wrote:
| Increment it a couple hex at a time, no one will notice til
| it's too late
| jdjdjdhhd wrote:
| Also get rid of near-invisible text for dead comments since you
| have to opt-in for seeing them
| bjornasm wrote:
| I allways assumed it was put on heavily downvoted posts to kind
| of "hide" low quality content. TIL I suppose.
| yCombLinks wrote:
| It's on all text posts. Look at the "Ask HN: Who's Hiring/
| Who wants to be Hired" posts.
| Sohcahtoa82 wrote:
| Heavily downvoted _comments_ will be lighter grey. But text
| posts are always that same light gray.
|
| I don't know why. It's awful.
| TylerE wrote:
| Even if it is, it is not an accessible solution.
|
| Edit: except it can't be, since HN doesn't actually allow
| downvotes on posts, only comments.
| fragmede wrote:
| It's very ableist and dang doesn't suffer from old people
| eyes, yet.
| samwillis wrote:
| Cars are tools - I have had a "crossover", an SUV and a station
| wagon (called an estate here in the UK), in my case all Audis.
|
| We got the crossover when we stated a family, it felt big and
| safe (compared to what I had before). However, the boot/trunk was
| too small for a growing family going on trips.
|
| We upgraded to a large SUV, felt great to drive, big a tall.
| Masses of space to load up, however hard to park in small spaces
| (this is the UK), and far too expensive to run.
|
| Now on an estate/station waggon, love it, favourite car I have
| owned. Cheaper to purchase than equivalent size suv as they are
| less fashionable, plus cheaper to run, cheaper tires for example.
| Far more fun to drive than both the others (and anything else
| I've had). Do not miss the height at all.
| smusamashah wrote:
| OneNote after trying many Markdown based tools including
| Obsidian. No other tool comes close in how fast it let's you dump
| down whatever it is in your head. It's like writing on a
| whiteboard or something. Click anywhere and write or paste. I
| have so much notes in there that now I want some kind of
| structured and linkable data support in it.
| eimrine wrote:
| I have used 3 types of money: cash, digital money and bitcoins.
| Digital money is an ugly idea, a lot of times the vendor took my
| funds out from me.
| l0b0 wrote:
| I've used Eclipse, Emacs, IDEA, jEdit, NetBeans, Vim (still do in
| a pinch), and VS Code for coding, and IDEA fits me much better
| than any of the others. Sane defaults, quickly getting up and
| running, intuitive and easily configurable keyboard shortcuts,
| usable mouse support, settings sync which Just Works(tm), the
| list goes on.
| jrvarela56 wrote:
| React state management solutions, here's my comment from last
| year: https://news.ycombinator.com/item?id=34133879
|
| tl;dr prefer Context but wish I could move everything to Mobx.
| tracker1 wrote:
| It's funny, I actually prefer Redux + Thunks the "hard way"
| (not RTK). I find it's far easier to reason with the state,
| transitions and events that way. It takes a bit more cognitive
| overhead getting started, but the complexity doesn't grow as
| features do.
| jrvarela56 wrote:
| Redux (even with RTK) has so much boilerplate it's not worth
| it anymore. I'm stuck with a bunch of legacy code but slowly
| moving it to Context.
|
| If you want actions and reducers just use the hooks versions
| for those + Context.
| tracker1 wrote:
| I find that using Context+useReducer is similar, but in the
| end you wind up with a lot of useContext and/or overlapping
| usage of different context/providers that the general
| single redux context is often better overall. Not to
| mention, that you still have to deal with the action
| generation and state in the scope of context, which gets
| even more messy with different Context usage in practice.
|
| At least that's been my own experience. I don't fault
| people for going different routes, I just find that the
| boilerplate kind of fades into the background after it's
| established.
| 2OEH8eoCRo0 wrote:
| VPN providers.
|
| I trialed a variety of VPN providers for a month rather than read
| reviews endlessly. I became a fan of Mullvad but I've have to
| switch recently because of their port forwarding changes.
| copperx wrote:
| These opinions are useless if the reasons are not stated. For
| example, you prefer iMessage over WhatsApp. Ok. Why? The UI? The
| exclusivity of the platform? Or is it just a whim?
| ajkjk wrote:
| They're not useless? If someone likes A > B and nobody likes B
| > A then you should give A a second look.
| copperx wrote:
| That's disingenuous. I don't believe you would pick something
| just because someone likes it. Almost everything that has
| been built is liked by somebody.
|
| Are flip phones worth a second look just because my parents
| prefer them over smartphones?
|
| However, if someone told me their anxiety vanished the moment
| they switched to a flip phone, they might be worth a second
| look. Maybe.
| yCombLinks wrote:
| Eh, I just look at the rotten tomato number and rarely dig
| into reviewer details. I'm looking for a decent comedy that
| most people think is decent. I usually don't need your
| thesis.
| Sohcahtoa82 wrote:
| Who said anything about _picking_ something just because
| everyone likes it? No, the statement was about _trying_
| something because everyone likes it.
|
| If everyone thinks A > B, and you seem to be the only
| person using B, then it might be worth your time to try out
| A. Sure, maybe you're happy with B, but there might be
| something you're missing that you don't know you're
| missing. If you try out A and decide you actually do like B
| more, that's fine.
| ajkjk wrote:
| Of course I wouldn't pick it _just_ because someone likes
| it, but it 's still... data. Data that I'll combine with
| other sources of data. It provides a small but non-zero
| amount of weight to one side. Aka it is not useless. If you
| see patterns in people's unjustified recommendations, it
| starts to be a lot more than non-zero.
|
| Your take appears to be: "for an idiot reader who will do
| exactly what they're told, this information is useless".
| Well! The readers aren't idiots. Easy.
| davidfstr wrote:
| I've used 3+ text editors on macOS (BBEdit, Sublime, VS Code,
| PyCharm), and prefer Sublime for my day-to-day work writing
| Python/JavaScript web apps, mainly because of fast code
| navigation and relatively fast typing speed.
|
| BBEdit I still use for multi-file search & replace since I prefer
| its interface for that. It also has the fastest typing speed.
|
| PyCharm I use exclusively for identifying & removing unused
| imports in Python.
|
| VS Code I don't use at all. Relatively slow typing speed.
| Relatively slow navigation.
| Thoeu388 wrote:
| [flagged]
| spencerchubb wrote:
| Messaging apps: Messenger
|
| Music & podcasts: Spotify
|
| TODO lists & note-taking: Text files
|
| Quick backend: Firebase
| EddyTests wrote:
| 3+ testing tools for websites - Selenium, Cypress, and
| Playwright.
|
| Selenium is a little old and verbose - the biggest issue, imo, is
| having to handle waiting for elements. Also managing webdrivers
| suck. Cypress is great for E2E tests but falls apart on edge
| cases, for example it can't handle multiple domains. Just
| requiring npm and not needing webdrivers is really nice for CI.
| Playwright is sort of a combination of both - fixes a lot of
| issues in Selenium but has the ease of use of Cypress. It is much
| less documented than Selenium though.
|
| If I was starting a new project I'd use Playwright.
| drewcoo wrote:
| > [Cypress] can't handle multiple domains
|
| It has for well over a year.
|
| https://www.cypress.io/blog/2022/04/25/cypress-9-6-0-easily-...
| EddyTests wrote:
| Not really, it was experimental and only released recently
| (end of last year?). You can't really build experimental
| features with potentially breaking changes or even a complete
| removal into a CI pipeline without a lot of risk.
|
| The main issue with Cypress handling this is, from what I
| understand, one browser per domain which can cause huge
| memory usage (which Cypress already has an issue with). Once
| you try parallelising multi-domain tests you run out of
| memory on developer laptops pretty quickly :(
|
| But yes ultimately you are right, multi-domain is available,
| it's just those edge cases that get you once you go beyond
| simpler cases.
| lmiller1990 wrote:
| Hi, I work on Cypress (but not the multi domain feature).
|
| I agree memory usage is an issue, I sure hope we can fix
| that. It's a hard problem with the frontend execution heavy
| model Cypress uses, along with the DOM snapshots consuming
| a ton of memory.
|
| There is a `experimentalMemoryManagement` feature that
| might be of interest to you [0]. The reason this one is
| experimental is the implications are not fully understood
| yet, and as a mature software product, we can't just throw
| out a big feature and "see what happens", so to speak.
|
| I understand not wanting to use experimental features, but
| this might be useful to your existing Cypress projects,
| even if you aren't using it for your new projects anymore.
|
| [0] https://docs.cypress.io/guides/references/experiments
| monkpit wrote:
| I agree with this, after you get past the simple happy path
| testing, working with Cypress is mostly just wrestling with
| Cypress and its limitations.
|
| Most of the limitations are around 2 things - Cypress has its
| own JS runtime separate from the node process which causes
| weird (non-intuitive) behavior. The other thing is the hand-
| wavy way Cypress tries to pretend the code is not async. You
| can't just write JavaScript and have it work the way you
| expect, you have to write Cypress-flavored JavaScript, which
| sets up lots of footguns for junior engineers and causes you to
| have to be extra vigilant on code reviews.
|
| I have worked heavily with Cypress for years, and if I were
| starting a new project I would 100% choose Playwright.
| tracker1 wrote:
| I'm with you on this... I'd just assume use simple
| test/assertion library along with Playwrite/Puppeteer. May even
| push for straight up Deno as the test base and runtime for that
| matter.
| politelemon wrote:
| Operating systems. I use Linux, Mac and Windows regularly. I've
| decided that I prefer Linux, tolerate Windows and regret Macs.
| eagleislandsong wrote:
| As someone who prefers Linux, tolerates Windows, and has never
| used MacOS, I'm curious to hear why you regret MacOS.
| Sohcahtoa82 wrote:
| Not the person you asked, but I'm in the same boat. I prefer
| Linux, but tolerate Windows because I'm a gamer, and I can't
| fucking stand MacOS. I'll give you my reasons.
|
| - Can't disable mouse acceleration via the UI. You have to
| use the terminal to enter a command to change a setting that
| sometimes gets overwritten on a system update. You can't
| disable scroll wheel acceleration at all without using 3rd
| party software.
|
| - I hate the dock bar. I want each window of an app to be its
| own entry on the bar so I can easily switch from one window
| to a specific window with a single click. In MacOS, starting
| a second instance of an application that doesn't have a
| native method to do so is a total pain in the ass. Going to
| the Launcher to start the program just shows the existing
| instance of the application.
|
| - Shitty defaults in general. Scroll wheel direction is
| backwards. Scrollbars auto-hide. The screen color shifts
| throughout the day.
|
| - MacOS is based on UNIX but always feels like it's trying to
| hide that from you. The "root" user isn't capable of
| everything. There are directories in your home directory that
| even root can't access.
|
| - Home/End keys move the cursor to the beginning/end of the
| document, not the beginning/end of the current line. As
| someone who uses those keys a lot in Windows, that behavior
| is aggravating. There IS a way to change this, but it's very
| non-obvious, and doesn't work in all programs.
|
| - Muscle memory of using CTRL+X/C/V to cut/copy/paste will
| screw you up as a Mac uses their Command key which is where
| Alt is on a PC keyboard.
|
| - Finder sucks. You can't cut/paste a file to move it, but
| you can copy/paste a file to copy it. To move a file in the
| Finder, you have to open two windows and drag it. The Delete
| key doesn't delete files.
|
| Often times, things feel different just for the sake of being
| different and not because the designer thought it could
| actually improve UI/UX. Apple seems to very much tend to
| subscribe to the mentality that removing features is a
| feature.
| maccard wrote:
| Version Control.
|
| Professionally, I've used git, perforce and PlasticSCM.
| Personally, I've also used SVN and Mercurial. (I also toyed with
| pijul's claims of being fast). I think Perforce sucks in many
| ways, but I think it's objectively a better tool than git.
| Centralized, immutable history, atomic incrementing change
| numbers are superior (IMO) to a decentralised, signed, hash based
| model. I think git's branches are a superpower, and the PR-style
| workflow in github is objectively superior to whatever trash P4
| has put out there.
|
| CI Tools - I've used Jenkins, TeamCity, GH Actions, Buildkite and
| Electric Commander. I strongly prefer Buildkite over all of the
| others. It strikes the right balance between Jenkins and GH
| Actions, mostly by avoiding a bunch of the "legacy" that comes
| with Jenkins.
|
| "Programming Languages" - This is a spicy one. I flip-flop
| between C++, Kotlin and Go these days depending on the task.
| Despite the warts, I really really like go. It's a statically
| typed language, built in (and opinionated) package manager,
| wicked fast compile times, an order of magnitude faster runtimes
| than python, OOTB cross compilation support for full statically
| linked binaries, (and it has a jetbrains IDE). I wish it had
| _some_ extra features (sum types, real enums, and a slightly
| wider range of the basics in the standard library), but overall
| I'm very happy with it!
| tom_ wrote:
| You can (and many do) purge items from the Perforce history.
| maccard wrote:
| Indeed, and it's useful. But, (and this is the most important
| part), it's a role based feature, not "standard day to day
| workflow", and it doesn't require everyone to handle as part
| of their mental model
|
| A force push on a git repo is pain for everyone involved
| jdjdjdhhd wrote:
| "I've used Chrome, Firefox and Safari all within the past year,
| and I prefer Chrome and Safari."
|
| Those are all Safari under the hood aren't they?
| Ilasky wrote:
| I've tried multiple translation apps for real-time translation
| between English and Korean, including Google Translate, Apple
| Translate, Microsoft Translator, and Papago (by Naver).
|
| Surprisingly, Microsoft translator is the best for real-time
| translation. It's still rough, but it's able to keep up in real-
| time. Google doesn't have the transcribe mode for the Korean-
| English language pair, and neither does Apple or Papago. Really
| surprised those other services don't have it
| esafak wrote:
| Any professional engineer choosing a solution has to do this and
| discuss the trade-offs. For example I was involved in a migration
| from Tensorflow and pyTorch and Go to Java at one big company.
|
| Personally, I'm playing with new browsers and search engines.
| SigmaOS, Arc, Firefox. Dropped Chrome, gave up on Safari.
|
| Search engines: phind, chatgpt by API, kagi, you, perplexity
|
| Not sure how useful this question is...
| raydiatian wrote:
| GCP over azure and AWS. I got to greenfields my company into the
| cloud, but as such I felt compelled to cost/benefit comparison
| the three main cloud providers. I also looked into Heroku and
| Serverless as alternatives.
|
| It's a close race between AWS and GCP, but we wanted serverless
| because we didn't want the hassle of touching anything kubernetes
| related. Unfortunately, AWS Lambda had some goofy limitations,
| like requiring every method to be post and only supporting single
| routes per container (this was my understanding at the time, it
| may have changed). On the other hand, GCP Cloud Run is built on
| top of Knative, which allows you to do serverless services.
|
| Also, fuck Jeff Bezos.
| leokennis wrote:
| Podcasts apps on iOS: I prefer Castro over Overcast, Pocket Casts
| and Podcasts.app.
|
| Castro's concept of show - episode - inbox - queue is unbeatable
| for me because it's uncomplicated but still powerful. Sideloading
| is solid. Audio enhancements are high quality.
| Blackthorn wrote:
| Horizonal bandsaws. I'm sorry but nothing compares to the
| American built ones. If only they could make one even smaller
| than the smallest Ellis saw.
|
| Welders. It's hard to beat the convenience and ease of a MIG
| machine. This does not apply to flux core wires that don't use
| shielding gas though. I've never found one as good and easy to
| use as plain old gas shielding.
|
| DAWs. Reaper is basically the perfect program as far as I'm
| concerned.
| cratermoon wrote:
| I've used wooden pencils, mechanical pencils, ballpoint pens, gel
| pens, fountain pens, felt-tip pens, rollerball pens, dip pens,
| and more. For pencils, I've used harder and softer leads, thicker
| and thin.
|
| For everyday writing, I prefer a good fountain pen that takes
| international standard cartridges. Nothing too lightweight, I
| need some heft, and generally a fine nib and good ink. I write
| left-handed, so it needs to absorb and dry quickly enough so that
| I don't smear.
|
| For marking up things, like taking notes in the margins of a
| paper or book, I prefer a pencil a bit softer than the standard
| #2. Right now I have a Uni Mitsubishi Hi-Uni Pencil - 2B, well
| worth the price. Of course, you can't have a pencil without and
| eraser and a sharpener. For the eraser a white Pentel Hi-Polymer,
| and to sharpen, a KUM 2-hole long point AS-2.
| jiehong wrote:
| Web servers: I've used Apache httpd, nginx, caddy and haproxy.
| Caddy is my favourite in local and in production, but haproxy is
| more powerful if no need to serve files.
|
| Java web frameworks: spring, spring boot and quarkus. I'd go with
| quarkus or maybe spring boot.
|
| Java unit test assertions: junit, assertj, Hamcrest. Assertj is
| much nicer than the rest. Use it with junit5 and jqwik too.
|
| Interactive shell: bash, sh, zsh, fish, powershell. I recommend
| fish with fzf.
|
| Scripting shell: bash, sh, zsh, fish, powershell. I'd go with
| powershell or bash.
|
| Text editor: vim, emacs, sublime text, notepad++, gedit, geany,
| textmate. Sublime text first. (I recommend Sublime Merge as well
| for git)
|
| Workflow engine: conductor, camunda, temporal. I'd go for
| conductor if you don't want to write code, temporal for code and
| camunda for the ui.
| maccard wrote:
| > Interactive shell: bash, sh, zsh, fish, powershell. I
| recommend fish with fzf.
|
| > Scripting shell: bash, sh, zsh, fish, powershell. I'd go with
| powershell or bash.
|
| For me, the value in these is that both are the same thing. If
| I'm having to flip between bash and zsh conventions when
| iterating/smushing things together, I'd rather just uese
| python.
| bitwize wrote:
| If Scheme implementations count as a tool, then (Tigger voice)
| comparing implementations by the dozen is what Schemers do best!
|
| I'm partial to Guile and Gambit. Guile because it has
| comprehensive POSIX support and is an awesome glue language.
| Gambit because it has a compiler that emits fast code and an easy
| C FFI.
|
| I prefer Emacs to all other text editors, but I frequently use
| whatever vi version is hanging around for quick edits and config
| changes.
|
| VCS, the order goes Darcs, git mercurial, the rest.
|
| Image editors, I dunno. GIMP, Krita, and GrafX2 I use at
| different times for different things.
|
| 8-bit micros: C64, TI-99/4A, VIC-20, everything else.
| AlexITC wrote:
| - Postgres over mysql/mongo/sqlite, I'm yet to come up with a use
| case where Postgres doesn't fit well for a while, while mysql is
| considerably better than before, it just bit me again where I had
| to run `mysql_tzinfo_to_sql` before tz conversions work, guess
| what? others didn't had to do this.
|
| - Github-actions over circleci/travisci/etc, while I miss
| features from time to time, github-actions adoption allows to
| easily find most workflows you can need, having it with github
| itself is also handy.
|
| - Integration tests with a real database mounted by docker, some
| people complain that this is slow but its worth it given the
| confidence it provides, in-memory databases or fake databases
| could be appealing due to the speed but they are annoying to
| maintain and never behave like production.
|
| - DigitalOcean over AWS/GCP/Azure, for most projects, DO is
| enough, its simple to use and predictable pricing is something I
| could never get from other clouds.
| anavette wrote:
| I've used 3+ dictionary apps and I prefer this Webster's Writer's
| Dictionary app: https://apps.apple.com/us/app/websters-writers-
| dictionary/id...
|
| The definitions, while slightly archaic, are evocative of the
| etymological valences in ways that other dictionary apps
| (Merriam-Webster, Oxford Dictionary, Dictionary.com) lack. The
| WWD also has a pleasingly clean UI, smooth interaction, and no
| ads.
|
| Special mention of Etymonline's etymology app.
| majewsky wrote:
| Do you have any experience using Wiktionary, and if so, where
| does it fall on your spectrum? I personally use Wiktionary a
| lot because it seems adequate from what I tell and it covers
| all the multiple languages that I'm interested in.
| cratermoon wrote:
| You can install the Webster 1913 dictionary as one of the
| dictionaries in MacOS and then use it anywhere the dictionary
| is available, including the cmd-crtl-D shortcut.
| https://github.com/cmod/websters-1913
|
| I agree 100% about the etymologies.
| commandersaki wrote:
| Compared Monodraw, asciiflow, and Vim DrawIt plugin and prefer
| Monodraw.
| ignite wrote:
| I didn't even know that was a thing! Cool.
|
| Reminds me of the ascii art video codecs!
| Mister_Snuggles wrote:
| I've got a few.
|
| Text editors: I've used emacs, pico, nano, jed, vim, AE
| (UniData), ED (UniVerse), Sublime Text, VSCode, BBEdit, and
| countless others that I've forgotten. Sublime Text wins for
| simple edits of single files, vim wins when I'm ssh'd somewhere
| or at a terminal, and VSCode wins when I'm editing a collection
| of related files or doing development. ED and AE have a special
| place in my heart.
|
| Databases: PostgreSQL, MySQL, UniVerse, UniData,
| OpenQM/ScarletDME, Sqlite, Oracle, MSSQL, Access. For relational
| databases, PostgreSQL for personal use and Oracle when someone
| else is paying for it. For MultiValue databases (UniVerse,
| UniData, OpenQM/ScarletDME) I like them all, but UniVerse was my
| first one and will always be special.
|
| Browsers: I use Chrome, Edge, Firefox, and Safari roughly
| interchangably. Each one works better (for me) than the others in
| different contexts.
| purplecats wrote:
| > I've used 3+ code editors on MacOS and prefer sublime text over
| VScode, coteditor, xcode. Check out IntelliJ Ultimate, it's
| phenomenal. Startup time is slow though.
|
| > I've used Chrome, Firefox and Safari all within the past year,
| and I prefer Chrome and Safari. Agreed. Sometimes I need
| different browsers for whatever reason, and for that you can use
| different Chrome releases (like Dev channel).
|
| > I've used 3+ interfaces for GPT, and I prefer the OpenAI
| playground over ChatGPT, chatbot-ui, typingmind, openplayground,
| Poe, and a few others I can't remember.
|
| ChatKit is really nice UI (if you are open to modding the css
| yourself, you can get a really slick interface pretty easily).
| Makes chatting pleasurable.
|
| Typingmind UX is horrendous.
|
| featuresets are different though
|
| > I've used 3+ voice transcription apps I've been experimenting
| with a pipeline to use Recup and run it though my local Whisper
| instance so all data is on-premise
|
| > I've used 3+ messaging apps, I like iMessage and Telegram Then
| you would love Chatkit. Also, telegram has web interface which is
| really useful (though Whatsapp does as well).
|
| > I think it's important to note that for me these are all
| preferences. I'm not saying one of these tools are objectively
| better. I would go as far as to say objectively better based on
| certain metrics. But I suppose the bias in choosing which metrics
| makes it a "preference".
|
| > What are areas of products, tools, developer tools, APIs,
| anything, where you've personally used 3+ tools, and what did you
| prefer, and what did you not prefer, if you can still remember?
|
| * Alfred over spotlight, Launchy.net, LaunchBar * GPT-4 over all
| the other AIs including OpenAI's offerings
| JoshTriplett wrote:
| Desktop environments: I've used Windows, macOS, GNOME1, GNOME2,
| GNOME3, multiple versions of KDE, twm, wmaker, and a half-dozen
| others. I really enjoy GNOME3 and the degree to which it simply
| _isn 't a point of concern_ and Just Works. It needs a little bit
| of non-default configuration, but so does everything, and I do
| that bit of configuration via my git homedir.
|
| Laptops: I've used Sony, Dell, and IBM-then-Lenovo ThinkPads;
| ThinkPad every time. Very eager to try Framework the moment it
| has a touchpad with real physical mouse buttons, though!
|
| Browsers: I've used Firefox, Chromium, and GNOME Web, and it's no
| contest: Firefox, every time, for both features _and_ needing
| meaningful competition in browsers /engines. With uBlock Origin,
| of course.
|
| Ad blockers: I used, sequentially, Adblock, Adblock Plus, and
| uBlock Origin. Adblock Plus and uBlock Origin are both good, and
| uBlock Origin seems subjectively faster and doesn't have the
| "acceptable ads" contradiction-in-terms garbage.
|
| Search engines: I've used Google, DDG, and a dozen others over
| the years, and stuck with DDG. Very happy with bangs, no ads, and
| generally the feeling of less invasive results.
|
| Email servers: I've used various ISPs over the years, I've used
| Gandi, I've run my own, and I now use Fastmail. Highly recommend
| Fastmail.
|
| Mail clients: I've used Thunderbird, Evolution, mutt, K-9, and a
| couple of webmail clients, and I prefer the combination of mutt
| and K-9 to deal with the sheer volume of mail I deal with.
|
| Editors: I used Emacs extensively for years, then switched to
| Vim, and have also tried various IDEs over the years. Stuck with
| Vim. My argument: optimizing _basic text operations_ provides a
| substantial win, everything else (and there 's plenty of
| "everything" to be had) is gravy on top.
|
| Version control systems: I used, in order, CVS, SVN, tla/baz,
| Mercurial, and git. I found git both more usable _and_ more
| powerful. The staging area is a great feature, `rebase -i` is a
| superpower, and having a clear idea of the underlying data model
| feels much better than having it be opaque.
|
| Build systems: I've used a _huge_ number of these over the years,
| including Make (and many variations), autotools, non-autotools
| configure scripts, CMake, Meson, Ninja, Scons, . /build.sh and
| many variations on bespoke shell scripts or Python scripts,
| _several_ language-specific package managers including
| setuptools, pip, cabal, npm, Cargo, Debian packaging
| infrastructure, and a couple dozen others. My general conclusions
| are to avoid CMake, to use your language 's package manager for
| building if it's remotely possible to integrate with Linux distro
| packaging, that opinionated tools are generally nice if you like
| the opinions and terrible if you don't, and if your language is C
| or C++, try to use the simplest thing that could possibly work
| and handle dependencies, including raw Make.
| Delk wrote:
| > DDG [...] no ads
|
| DDG does have ads, though? Perhaps they're less common and less
| intrusive than on Google, and only appear for searches that can
| meaningfully have ads related to them (e.g. if I search for
| "thinkpad" I get one). But they exist.
|
| Unless your ad blocker is blocking them. But, to be honest, I
| personally rather let DDG show the ads. I want them to have
| their income.
|
| I also generally use DDG first, although Google is better for
| me at local results or non-English searches, so I sometimes do
| those on Google. It has also sometimes seemed like more complex
| searches with OR clauses or something similar work better in
| Google search.
|
| Also, while Google isn't particularly good at returning exactly
| what I search for nowadays (it used to be better, nowadays it
| seems to guess too much or return what Google wants you to
| want), I don't feel like DDG is better at that either, and
| sometimes I seem to need to run e.g. searches for error
| messages on Google instead of DDG because of that.
|
| But for most general searches, DDG does fine.
| JoshTriplett wrote:
| With or without an adblocker, DDG gives you a checkbox in
| their preferences to disable ads.
| redman25 wrote:
| Web backends. I've used Flask, Django, Sinatra, and static site
| generators. Flask and Sinatra are great for simple APIs, anything
| more and you'll wish you had the structure of a larger framework.
| Django for database backed frontends. Static site generators for
| simple sites that have no need for persistence. Static sites can
| actually go further than you would expect at first glance, for
| instance, Obama's campaign website was backed by static HTML.
| botulidze wrote:
| Spent the past few months on:
|
| Observability platforms - Elastic / DataDog / Splunk / new era
| platforms
|
| Incident management and alerting - PagerDuty / OpsGenie /
| FireHydrant
|
| Status Pages - Atlassian / PagerDuty / new era platforms
|
| Would be really keen to read someone's experience on Jira on-prem
| / Cloud / alternatives considered.
| modestraen wrote:
| I would love to hear your opinion on observability platforms,
| this is something I am starting to investigate.
| karmakaze wrote:
| Most of what you've listed I would consider rather mainstream
| surface-like things, which could be swapped out for another
| without much impact. When I think tools, I think frameworks,
| databases, programming languages.
|
| Also listing that you have a strong preference for one over
| others isn't helpful without reasoning as to why.
| olalonde wrote:
| Not really an apple to apple comparison but I've used many
| programming language package managers (e.g. npm, pip, maven,
| bundler, cargo) and cargo is hands down the best one I've worked
| with.
| hawski wrote:
| After using 3+ tools I often come to the conclusion that I don't
| really like any of them. I see the trade offs and choose
| something based on often superficial reasons.
|
| I use Linux instead of Windows or macOS, because it allows me to
| tinker and I feel more free. But I do use ChromeOS, because
| machines are cheap, the experience is good and I get access to
| Linux thanks to Crostini.
|
| While I use ChromeOS I don't really like Chrome and prefer
| Firefox. I've used Brave and Edge, but thinking how they are all
| Chromium with a twist is not that compelling for me. But Firefox
| has a killer feature for me with Simple Tab Groups, Containers
| and full uBlock Origin.
|
| With editors I used many, but I'm mostly stuck with vim, because
| I invested in it in the past and it is almost always there or
| easily obtainable. Other than it VSCode is good, but I feel dirty
| using it.
|
| From terminals I've used many, but find rxvt-unicode not getting
| in my way the most. Though often I just use xfce4-terminal,
| because it is there.
|
| From Linux DEs I've used most. KDE/Plasma seems always buggy for
| me. GNOME is too strange. dwm and friends are very nice, but
| sometimes I don't feel like setting it up on a new machine/vm.
| Budgie was nice, but was too much tied to Solus at the time.
| Xfce4 is just there, reliable and pretty enough.
| eitally wrote:
| I've used all sorts of sit/stand desks and chair & swiss balls in
| all sorts of combos, but I've learned over time that I prefer a
| static height desk and a padded piano bench to sit on. Sitting on
| a bench rather than a chair with a back encourages much better
| posture [for me]. It also encourages me to get up and move around
| more because it's trivially easy to get up & sit back down from
| any angle.
|
| Virtual meeting platforms: Webex is by far the worst. Zoom is
| amazingly clunky given how much money they have to spend on it,
| but at least it generally works across various desktop & mobile
| environments. Meet is the most basic, but it "just works", and
| the features on web & mobile are identical (unlike some of the
| options). It also seems to have been undergoing the most
| consistent feature development over the past 18mo. Teams is
| perfectly fine on desktop if you're already bought into O365, but
| it's hot garbage from ChromeOS or if you're a guest.
|
| Collaboration/productivity software for enterprise: O365 contains
| too many products, is too complicated, and afaict, nobody
| actually knows which is the right tool for the job. If it was
| just Word/PowerPoint/Excel/Access/Outlook + Teams it would be
| fine, but add in Sharepoint/Viva/OneNote/OneDrive/Forms/PowerBI/P
| ublisher/Project/Bookings/Planner/Visio and nobody knows which
| tool is the right one for the job. Google Workspace can't hold a
| candle to Word/PowerPoint/Excel but arguably is much easier for
| normal people to grok. Trying to find files at an O365-using
| company is nearly impossible.
| majewsky wrote:
| > I prefer a static height desk and a padded piano bench to sit
| on.
|
| I have a similar setup, though with a height-adjustable desk.
| Instead of a piano bench, I got a simple stool from Ikea [1],
| which apparently they classify as "standing support". Works the
| same for me as you describe with the piano bench.
|
| [1] https://www.ikea.com/us/en/p/nilserik-standing-support-
| black...
|
| > Trying to find files at an O365-using company is nearly
| impossible.
|
| 100% agree. Microsoft's work has created millions of jobs in
| the "Sharepoint archeology" sector. David Graeber must be
| rotating in his grave.
| Sohcahtoa82 wrote:
| SharePoint is the quintessential example of feature creep run
| rampant. It's like MS didn't know what SP was supposed to be,
| so they let it be everything, and as a result, it's clunky and
| slow.
| slotrans wrote:
| OLTP databases. I've used Oracle, MySQL, and Postgres in anger
| (in that order, historically), plus DynamoDB and MongoDB if you
| want to count those. I have to admit Oracle has some technical
| superiority but the pricing is insane. MySQL is a joke. MongoDB
| has zero use cases. Just Use Postgres.
|
| OLAP databases/query engines. Redshift, Athena/Presto, Snowflake,
| BigQuery, ClickHouse. It's complicated, actually. I guess my
| strong opinion is that Snowflake is massively overrated and it
| _will_ perform worse and cost you more than you expect.
|
| DAG frameworks. make, Luigi, Drake, Airflow, DBT, Prefect,
| NextFlow, others I'm forgetting. Airflow is hot garbage. It's
| frankly an indictment of Data Engineering as a specialty that
| we've formed a consensus around such a poorly-designed piece of
| software. In fairness, no one has really gotten this right yet. I
| like Luigi but it has its own shortcomings and continued
| development is an open question. Sadface.
| tough wrote:
| > MongoDB has zero use cases. Just Use Postgres.
|
| I kinda am on that bandwagon but at a new job we're using
| mongoose and mongo for our database, because the consultors
| that implemented the webapp before us decided so.
|
| How do I convince mgmt that we might want to switch that? When?
| Is it even worth it or possible now that it's already our
| default?
| burkaman wrote:
| "Just use Postgres" is advice for choosing a database for a
| new project, it does not apply when you're already running
| something else in production successfully. In general I don't
| think it would be worth switching unless Mongo is causing
| significant tangible problems in your specific environment
| that Postgres would solve.
| tracker1 wrote:
| I think it deeply depends on how much variation in code is
| actually there to support the database(s). Along with how
| much data, how many collections, and how would you handle the
| migration and/or scaling with your proposed alternative? Is
| it worth stopping all feature development for weeks/months to
| migrate? How profit driven or constrained are you?
|
| There's a lot that I like and dislike with MongoDB. As a
| developer, it can be a relatively nice experience. As an
| administrator, I hate it beyond words. It really depends on
| your needs, entrenchment and the skills of your staff.
| btown wrote:
| For one, compare https://jepsen.io/analyses/postgresql-12.3
| and https://jepsen.io/analyses/mongodb-4.2.6 - similar
| vintages, entirely different levels of reliability.
|
| From a migration perspective, the presence of JSONB fields in
| Postgres, and much of the modern tooling around them, mean
| that you can even put a highly nested document structure
| directly into Postgres and have a clear migration path
| without rewriting code.
|
| (Do note however that Postgres' JSONB overrides the key order
| of objects to have canonical object identity - it thus avoids
| pitfall #1 in https://devblog.me/wtf-mongo but legacy code
| may rely on that behavior.)
|
| MongoDB was great when it worked; I've still never
| experienced as fluid a workflow as Meteor enabled, where
| either a client update or a batch process could update a
| document in MongoDB and it would immediately propagate to
| every other interested client. RIP - while there's an active
| community, the Meteor team went on to do Apollo GraphQL,
| whose real-time support is a shadow of that original
| developer experience.
|
| That said, on that same project, I can say anecdotally that I
| was bitten _hard_ by some of the disaster recovery and
| divergent-oplog issues mentioned in the meme-y
| https://aphyr.com/posts/284-jepsen-mongodb - granted, the
| software has improved significantly since then. But I'm of
| the opinion that a company developing database products can
| only shift its internal culture so much; such a "cowboy"
| mentality, one that led to releasing a database product with
| these glossed-over problems and unreliable defaults, never
| truly goes away.
|
| Rewrites always require a cost-benefit analysis. Modern
| MongoDB (you are running a modern version, right?) may have
| addressed enough concerns that it's the right decision to
| stick with. But even for document database needs, there's no
| reason not to use Postgres for green-field projects.
| programmarchy wrote:
| Depends on your data and how it needs to be modeled and used,
| imo. If it's sparse and interconnected then obviously SQL
| makes a lot of sense. But for data that is very document-like
| e.g. 10Ks you'd get from SEC EDGAR, then document based is
| suitable. I've found even with relational databases,
| sometimes you need to denormalize the data a bit to achieve
| performance benchmarks for complex queries, and there's a
| tipping point where you may want to sync your data into
| something document-based like Elasticsearch to get improved
| search capabilities, too.
| Palmik wrote:
| Can you share for what kind of tasks you use DAG frameworks
| such as temporal?
| saulpw wrote:
| Re OLAP: 100% agreement on Snowflake being a poor performer.
| BigQuery is Google-scale I guess which might be necessary but
| feels way over-complicated for most uses. I've only dabbled
| with ClickHouse but at every turn I've been impressed with
| their offering.
| gjsman-1000 wrote:
| I'm surprised that naming a poorly-performing database
| "Snowflake" didn't seem like an obviously bad marketing
| decision...
| mushufasa wrote:
| have you tried mage ai for DAG/elt?
|
| I haven't yet used it, we are evaluating it though, and I'd
| love to hear from HN'ers that tried it
| hodgesrm wrote:
| > I guess my strong opinion is that Snowflake is massively
| overrated and it will perform worse and cost you more than you
| expect.
|
| Are there specific use cases or experiences that prompt you to
| say this? I've seen a lot of examples (such as web analytics or
| SEIM) where teams have built very capable stacks on ClickHouse
| or similar analytic databases. Basically if you have a focused
| use case it's often possible to built a custom stack with open
| source on Kubernetes that outperforms Snowflake along axes like
| p95 response, cost-efficiency at scale, and data ownership. It
| would be interesting to hear more about your experience.
| dharmab wrote:
| Could you link some resources on critiques of Airflow? I have
| some colleagues using it and knowing of any impending footguns
| would be helpful.
| tetha wrote:
| Color me very interested about footguns in Airflow as well.
|
| We're currently considering it as a self-hosted process
| engine to automate technical business processes and to
| coordinate a few automation systems like jenkins. Crap like,
| trigger a database restore via system A, wait until complete,
| update tickets, trigger some data migration from that
| database via System B, update tickets. Maybe bounce things
| back to human operators on errors and wait for fixing /
| clarification. Trigger a bunch of deployments in parallel and
| report on success.
|
| Systems in this space are either (a) huge, hugely expensive
| enterprise applications designed to consume all business
| processes, which is a bit overkill for our current needs
| (Camunda, SAP, Stackstorm, ...), (b) overfitted onto a very
| specific data analysis setup (aka: if you don't have hadoop,
| don't touch it) or (c) overly simplistic and offering no real
| benefit beyond investing into self-hatred, guiness and making
| jenkins work.
|
| Airflow seemed like a bit of a decent middle ground there for
| workflows a bit beyond what you can sensibly do via jenkins
| jobs.
| weitendorf wrote:
| An inherent problem with data engineering dag frameworks is
| that they are not directly integrated with the nodes, they live
| above the nodes. The nodes don't know about the DAG and
| depending on the software they're running they'll have
| different semantics for checking if something is stuck,
| cancellation, etc.
|
| I think there's a lot of room for innovation here. Given 1+
| data streams or ingestion locations, a bunch of SQL scripts, a
| DAG to orchestrate the scripts, and 1+ data destinations there
| are many different execution models that could be used but
| aren't. You've only specified the pipeline semantics rather
| than implementation, so smart tooling should be able to
| automatically implement patterns like streaming or intermediate
| queueing without much further input. IMO that's what DAG
| frameworks could be: "compilers" for your data pipeline, rather
| than orchestrators. There's progress in the area but nothing
| that quite gets there yet AFAIK
| esafak wrote:
| Here is another workflow orchestrator for you to try: Flyte!
| ldjkfkdsjnv wrote:
| Can confirm that Airflow is a terribly designed system. So
| inflexibile and gotchas that only arise after a ton of
| development. AWS Step Functions are infinitely better and more
| simple.
| streblo wrote:
| I've put a lot of time into Airflow and feel similarly that
| it's a huge pain and a risk to rely on it. I've replaced it
| with Temporal (https://temporal.io/) and while I don't have the
| breadth of experience with the frameworks you listed, I do
| think Temporal is a great replacement for Airflow.
| ye-olde-sysrq wrote:
| I worked in Data Engineering for 6 years before recently
| deciding to at least take a siesta to write some "normal"
| software as a product engineer.
|
| I really like data engineering in general - writing spark jobs,
| manipulating huge amounts of data, concerting pipelines,
| writing SQL... I just like it all a lot. I like SQL - there, I
| said it!!
|
| But damn do the rough edges just burn you out. Airflow is one
| of those edges. IMO part of the problem is that everyone wants
| to hold it in a slightly different way, and so you end up with
| Airflow being this opinionless monster thing that lets you use
| it any way you want as long as it "works". And everything built
| on top of this opinion-less mess is of course also kind of a
| mess. And plus at a certain scale, Airflow itself needs to be
| maintained as a distributed system.
|
| Spark is another one of those things with lots of rough edges.
| Some of it might've been the places I worked at were holding it
| all wrong, but Spark and all its adjacent context are so damn
| complicated you either have a small group of experts writing
| all spark code and you have to heavily prioritize what they
| work on, or you have a bunch of non-data-engineer folks writing
| spark jobs and kind of making it work sorta but it's all as
| inefficient as possible while still making it work, and it's
| really brittle as they don't know all the ways they need to
| think about things scaling.
|
| Notably, I got really tired of people coming to our team and
| going "we wrote this big spark job, it worked fine up until it
| didn't, can you please help us fix it, it's mission critical".
| And it's just an OOM because they're pulling data into memory
| and have just been bumping -Xmx every 2 weeks for six months.
| or perhaps even worse, they were doing some wonky thing that
| wouldn't scale and then now they're running into weird errors
| because they used the feature so wrong (like, having a small
| dataframe with 1-2 partitions that then gets used in a
| subsequent stage that's HUGE and all the executors absolutely
| swamp the poor node or two that's hosting those partitions).
|
| Anyway, it's easy to write a spark job that works at the time
| of writing, and really hard to make a spark job that will work
| 6 months from now.
|
| Add to the fact that Spark is a giant money firehose, data
| engineering departments are constantly asked to sacrifice
| reliability to increase efficiency (just run everything hotter!
| who needs slack or buffer space?) and it just makes all the
| above issues worse.
| [deleted]
| sohnakukkar wrote:
| How much did you pay for Oracle? Is it a one time payment or
| annual?
| oneplane wrote:
| Over the years I've had many of those but I often ended up
| preferring some for some tasks but overall using most of them.
|
| Examples (including dead products and random tools you might not
| always put in the same league): Desktop
| Environments:
|
| Aqua over GNOME3, GNOME3 over DWM/Windows, DWM/Windows over KDE,
| KDE over non-DWM/Windows Terminal Emulators:
|
| Terminal.app over gnome-terminal over iTerm2 over urxvt over
| xterm over Windows Terminal over CMD.exe Editors:
|
| Sublime Text over Atom over vim over BBEdit over VSCode over nano
| over Notepad++ IDEs:
|
| IntelliJ over Xcode over NetBeans over KDevelop over Eclipse over
| Visual Studio Browsers:
|
| Safari over FireFox over Chromium over Chrome over Edge over
| Edgeium Shells:
|
| bash over zsh over csh over powershell over command.com
| Databases:
|
| Postgres over FoundationDB over MySQL over MongoDB over SQLite
| over Neo4J IaC:
|
| Terraform over SaltStack over Ansible over Chef over Puppet over
| PS DSC Public Clouds:
|
| AWS over GCP over Azure Communication
| Collaboration:
|
| Slack over IRC over Matrix over Signal over smoke signals over
| tin cans with a string over not communicating at all over MS
| Teams Workload Orchestration:
|
| Kubernetes over Nomad over CloudRun over ECS over systemd units
| over Swarm
| joenot443 wrote:
| What makes you prefer Terminal.app over iTerm2?
| oneplane wrote:
| Mostly the default availability and native stability and
| compatibility with very Mac OS X and macOS release so far. I
| don't really need the terminal emulator to do much (i.e. I
| don't need to have it integrate with tmux or screen, do
| things with the prompt or 'react' to what's happening in the
| shell.
|
| While I have a couple of personal devices where I could go
| and customise everything as much as I like, it wouldn't port
| over to anyone I work with or any system I write software
| for, or any presentation I give, documentation I write etc.
|
| Being more portable gives be better results overall than
| being specifically tailored to my preferences. I do default
| to light foreground and dark background colours wherever I
| go, but that is supported everywhere as far as I know, and
| has been the default on many platforms for long enough. (that
| said, I sometimes consult for complex/crazy situations and
| whenever you spot an xterm with black-on-white you know it's
| pain 'o clock).
| gazby wrote:
| Where IaC is distinct from configuration management, what would
| be your preference for that?
| oneplane wrote:
| I would use Terraform for anything that is provisioning and
| probably SaltStack for everything else.
|
| There are some scenarios where there is no 'one tool for
| everything', like creating a complex OCI image, a VM image
| using Packer, a VM using Vagrant, configuring a network
| appliance or configuring an end-user workstation.
|
| Example: super complex OCI images that attach to a CI
| pipeline would be better served by native packaging tools
| (like Jib for Java projects), but re-packaging of existing
| applications might depend on how green the build is. You
| could do a bunch of shell scripting in a Dockerfile, but once
| that gets big enough you'll start to need to use functions
| and perhaps even function libraries that you source in, but
| at that point you're doing a re-invention of something like
| Ansible and SaltStack, so depending on existing modules,
| knowledge, time, experience etc. you might use those instead.
|
| Compare that to non-packaged-deliverable things like network
| switches and routers; those tend to be long-lived (non-
| ephemeral) systems, and they tend to have various
| inconsistent APIs between vendors, models and generations.
| But you may want to exchange configuration inputs and outputs
| with other systems so your Cloudflare rules, AWS ALBs, AWS
| SGs, on-prem Palo Alto firewall and on-prem Cisco switch
| agree on the configuration, and in such a way that it can be
| validated and audited constantly and consistently. Not a
| single named vendor does it all, especially not in a
| consistent and useful way. But as long as there are providers
| for Terraform, you can (with Terraform).
|
| Perhaps the best way to describe how I balance it (or try to)
| is: the more modern the API, the more declarative I'd want to
| manage it in code. Cloud APIs tend to sit on the most-
| declarative end of the spectrum and end-user workstations on
| the least-declarative (or: most-imperative) end.
___________________________________________________________________
(page generated 2023-08-01 23:02 UTC)