[HN Gopher] My Knowledge Lakehouse
___________________________________________________________________
My Knowledge Lakehouse
Author : tabokie
Score : 113 points
Date : 2024-01-13 09:33 UTC (1 days ago)
(HTM) web link (tabokie.github.io)
(TXT) w3m dump (tabokie.github.io)
| fireynis wrote:
| I like the idea, and I do something similar but this has a lot of
| rules and feels more complex than potentially necessary. I
| personally use Obsidian.md as the tool for my Zettelkasten
| method. It provides back links and the like. I also create
| engineering journals for projects.
|
| The only rule I have is to avoid unlinked notes.
| dinkleberg wrote:
| There are some good ideas in here.
|
| Personally I've abandoned all of my complex attempts at
| productivity and knowledge systems and have recently moved to the
| ever reliable txt file. You can format it however you want, use
| fun ascii art, or just plain paragraphs.
|
| I've got a daily.txt where I prepend a new days section every day
| and put in whatever I want over the course of the day. I also
| create other files for courses I'm going through or projects as
| needed. The nice thing is using my editor I can make a file
| reference and hit gf and it'll go right to a referenced file. But
| this "system" will keep on working regardless of what tools or
| system I happen to be using.
| ahmedfromtunis wrote:
| It's clearly a nice and simple way to write data, but what
| about the other way around? How do you manage to find
| information when you need it?
| dinkleberg wrote:
| Read data? lol it is usually just a write only system (aside
| reviewing the current info from the week).
|
| But since it is all in one place I just do search queries
| using my editor to find what I need. I don't use tags, but it
| would be very simple to just append some #tags in my content
| and search for those.
| gkbrk wrote:
| CTRL-F or grep seem like they would work well
| wharvle wrote:
| I use Apple Notes and don't bother to organize anything
| unless it's a whole bunch of related material that I'm going
| to use together--notes for an rpg session, say. All the rest,
| I just make sure that I give it a title that I'll recognize
| when I see it, and include words that I'll likely use if I go
| looking for it with search.
|
| Been doing this for years, haven't lost anything yet. It
| works so well that time organizing it would have been wasted.
| falseAss wrote:
| i've been doing the exactly same thing. Some risk is the
| data safety - years of log is attached to the icloud
| account is concerning.
| wharvle wrote:
| There exist programs to export them to markdown or what
| have you. Dunno how well they handle embedded media. I do
| a lot of copy-pasting screenshots or embedding PDF
| pages... or entire pdfs.
|
| [edit] "why screenshots?"
|
| 1) To record gui workflows, walkthrough-style.
|
| 2) to record whole screens of values from guis while
| preserving formatting perfectly (think: cloud dashboard
| vital stats screens for various resources)
|
| 3) to record short message exchanges from ephemeral
| messaging with all the formatting intact with zero extra
| effort. (Think: feature discussion in a periodically-
| cleaned chat channel; I can always turn it into text
| later if I need to, recording with screenshot is fast)
|
| 4) plus now that it's almost as easy and reliable to
| copy-paste from images as from regular text, on macOS and
| iOS, why not?
| tomcam wrote:
| Having trouble with Apple notes syncing too slowly. Wife
| can update the shopping list but I won't see it for 24+
| hours. Unfortunately my life runs on it. Problem may be
| that my Notes file has become a 15G monstrosity?
| ukuina wrote:
| > 1.1221 Sometimes the primary tool is not available. An "always-
| on" secondary should take its place.
|
| > 1.1222 Sometimes log from different tools or locations need to
| be "merged", "persisted".
|
| These two are the main reasons I've stuck with Google Docs: It's
| available everywhere, and everything's always in-sync. Google
| already has all my info of value, so the incremental trust
| necessary to the threshold of log visibility is minimal.
|
| I'll add one more requirement: I don't want to leave an
| unencrypted on-disk footprint containing my notes. This means I
| can access the entirety of my notes on any machine and the only
| thing I have to worry about is keylogging/screenscraping.
|
| Google Doc is lacking in many respects, though. Linkability is
| nonexistent. Docs longer than 100 pages really struggle with
| latency on mobile. Searchability is bad (!) because you have to
| open the doc first to see the matches from within.
|
| Really wish there was a self-hosted alternative with sync and
| encrypted storage that didn't result in sync errors. I've tried
| DEVONthink, Obsidian, LogSeq, Google Keep, Notion, NotesNook,
| GoodNotes, Samsung Notes, Loop, OneNote, Apple Notes, Org Mode,
| plain text files, and probably a dozen others... I'd say
| NotesNook is the best so far, with DEVONthink a close second,
| but, nothing beats the reliability, omnipresence and privacy of
| Google Docs.
| zilti wrote:
| Syncthing + org-mode is the winning combo for me
| sebtron wrote:
| > It's available everywhere
|
| Unless you are offline
| diggan wrote:
| Or somehow triggers their anti-spam system on service Y while
| you really care about service X, but they're all bundled
| under the same company Z, so being blocked on one service
| impacts your usage of all other services under the same
| company...
| vineyardmike wrote:
| I think apps generally work offline which satisfies the OPs
| requirement of eventually consistent merging.
| seanwilson wrote:
| What about using Visual Studio Code with your notes in multiple
| Markdown files with automatic syncing to Google Drive via the
| desktop Google Drive app?
|
| Searching multiple files, editing and organising files is quick
| and simple on desktop, and you can fallback to Google Doc on
| mobile when needed (Google Drive app lets you view .md files,
| there's no way to edit?). You can install extensions as well
| (like for doing inline maths) and if you already use Visual
| Studio Code for coding it's one less thing to learn.
|
| I'm not familiar with Obsidian, which gets mention a lot. What
| does it improve on compared to the above if you don't need a
| mobile app or complex linking between Markdown files?
| jbiggley wrote:
| +1 for Obsidian or other platforms that support Markdown
| format natively. Being platform locked is terrible for
| knowledge management and Markdown has made me less concerned
| with future knowledge access in a post-Obsidian world. (Yes,
| I said it! There will be a post-Obsidian world.)
| MonaroVXR wrote:
| I'm using it and syncing my notes to GitHub, these are
| technical notes
| wfhBrian wrote:
| A couple years ago I switched from Google Docs to Obsidian.
|
| Unfortunately, as other have mentioned, Google Docs omits
| some seriously impactful features.
|
| All it would've taken, at the time, was
| collapsible/foldable headings for me to stick with Docs.
|
| But since then, I've grown to appreciate the millions other
| things Obsidian has to offer, like the ease of developing
| plugins which, to me, make Obsidian feel like it's an OS
| within my OS.
| sesm wrote:
| If you don't need to sync with mobile, then adding your
| `notes` directory to IDE projects is a great solution. I've
| been doing that for a long time.
|
| Eventually I switched to Obsidian for mobile support (syncing
| with free 'Remotely Save' plugin using S3). There are 2 other
| features of Obsidian that I came to appreciate over time:
|
| 1. Daily Notes
|
| 2. Calendar via Full Calendar plugin that uses your daily
| notes as one of event sources. So I can freely mix my Google
| Calendar 'official' meetings with my personal timeslot
| allocations for current day.
|
| It is possible to have both of this features in IDE too, but
| with Obsidian they come for free.
| inferense wrote:
| try acreom, free e2ee sync that works out of the box with a
| local-first setup.
| ssss11 wrote:
| Joplin is great try it. I self host mine, it syncs between
| devices, encrypted notes.
| _boffin_ wrote:
| Thanks for posting this. Currently building out something that is
| consolidating events from numerous sources and then automatically
| processing of those.
|
| This post helped to give me some additional ideas on the
| processing of those events.
| threecheese wrote:
| Interesting, and something I've been thinking about. What is
| your architecture? Using any available workflow products for
| "retrieve/receive event from X, figure out what do, then do
| that in Y"?
| csbartus wrote:
| Very cool ideas! Very simple concept, indeed!
|
| It helped to pin down my own approach, which is a very similar
| workflow, with these differences:
|
| - The log entries are flashcards with ID
|
| - The structure / tree follows the scientific method, aka the
| glue is not ad-hoc.
|
| - So far I need no lists, because I only learn things and I'm not
| tracking any activity
| simonw wrote:
| This year I'm trying a new mechanism based around GitHub Issues
| (which I've been using for personal knowledge management for a
| few years already) - I create a new "planner" issue every day,
| then use that to make notes about what I want to do and what I've
| got done.
|
| Hitting Command+N in Firefox on my Mac opens up my planner issue
| for the day, or creates it if one doesn't exist yet. I wrote up
| how that system works (a tiny bit of GitHub Actions + Pages
| magic) here: https://til.simonwillison.net/github-actions/daily-
| planner
| civilized wrote:
| This post probably has some kernels of value, but it suffers from
| the XY problem: it's a note taking system which provides the
| solution, but not the problem it's solving. Let's invert that and
| focus on the problem first. Below are the problems my personal
| system solves, followed by the solutions. My solutions may not
| fit your situation, but if you see yourself in the problems,
| maybe you'll be inspired to find the solution that works for you.
|
| 1. Formatting easily becomes a distraction, but a _little_
| formatting is vital, so I use markdown to write my plans and
| notes.
|
| 2. Mixing streams from different projects is confusing, so each
| project has its own workspace - either a folder or a distinct
| prefix on the file name.
|
| 3. Work or ideas often become irrelevant for a while as plans
| change, but I hate feeling like I might lose work that might be
| valuable later. So I have a "dump" file or folder where I can
| dump such things. Entries are typically dated so they can be
| referenced by other files (see below). I typically use a level 1
| or 2 markdown section header to delimit entries.
|
| 4. I often have a "log" markdown file where I append descriptions
| of indisputably important project developments (as opposed to
| maybe-important-later stuff that goes in the dump file). Entries
| here are dated and delimited similarly to the dump file.
|
| 5. There is a "plan" file that I try to keep clean and concise.
| This is where I go to remember what I have resolved to do next.
|
| 6. A common issue with plan files is that they get bloated with
| discussion of current or past context, as you try to figure out
| what to do and how to do it. When this happens, the discussion is
| moved to the "log" or "dump" files and replaced with a "link"
| describing how to find it again (e.g. "see log date 2024-01-12
| for more on this").
|
| 7. There is a master plan file that coordinates all the projects.
| Bloat in this file is moved down to individual project files and
| replaced with a link.
|
| To recap: markdown; project workspaces; log and dump with dated
| entries; concise plan file; move bloat out of plan into log or
| dump and replace with a link; master plan file with similar bloat
| management strategy.
|
| I have used this system for five years. It has developed over
| time, but all elements have been in use for at least two years.
|
| I am a staff data scientist leading a team of a half dozen people
| that is overhauling the core analytics pipelines and models at a
| Fortune 100, and 80% of my day is IC work. I have this job not
| because anyone asked me to do it, but because I proved I could do
| it on my own time. I work normal hours, parent of two kids,
| working spouse. I don't have time to putter around. This system
| is a big part of how I make it work.
| sureglymop wrote:
| I just create one markdown file per day. I wrote a custom
| obsidian plugin that displays the "timeline" as if it was one
| open file and creates the files in year/month/week subfolders.
|
| But I don't use linking and all the other complexity. But if I
| ever needed it I could.
|
| Then it's synced with next cloud and also available on my phone.
|
| I like this the most because just as explained, the time line is
| the most accurate representation of reality. And I can very
| quickly scroll back to see what I did last week or do full text
| search.
|
| Simple and useful!
| philips wrote:
| Can you explain the timeline further? I don't understand- just
| a list of links?
| sureglymop wrote:
| The idea is to create one markdown file per day but to be
| able to open them all in one editor view. Then one can scroll
| through all the days conveniently like a timeline. Something
| like this [0] plugin although I did create my own (more
| hacky) version for myself.
|
| [0]: https://github.com/Quorafind/Obsidian-Daily-Notes-Editor
| canadiantim wrote:
| Why next cloud for sync?
| sureglymop wrote:
| Because I already use it for everything else such as contacts
| (CardDAV), calendar events and tasks (CalDAV) which syncs
| nicely with my android phone. It's not that hard to host if
| not many people use the instance, has always worked well for
| me.
| toddmorey wrote:
| I love the approach. Have you published the "timeline" plugin
| or is code available?
| sureglymop wrote:
| I haven't yet because its hacky at the moment and I want to
| clean it up first (once I have some spare time). But there
| are somewhat similar plugins already published, e.g.:
| https://github.com/Quorafind/Obsidian-Daily-Notes-Editor
| Zetobal wrote:
| Reads a lot like logseq.
| sureglymop wrote:
| That was actually the inspiration for the journal view but I
| like obsidian more. One thing I do miss from logseq is also
| its PDF annotation feature. Although that also comes with
| quite a bit of complexity.
| hnthrowaway0328 wrote:
| I had a revelation that my ideas are worthless and the notes I
| took are also going to be worthless once I get familiar with X.
|
| So I completely stopped taking notes. A to-do list plus a Google
| Drive for my son and our stuffs are good enough. The only problem
| is that we are not comfortable to put sensitive information on GD
| so probably build a NAS for that.
| Zetobal wrote:
| Can work in a fast paced environment but if you work on
| projects or ideas with hard numbers for years its not
| applicable or your brain capacity is off the charts.
| hnthrowaway0328 wrote:
| Yeah 100%, I forgot to mention the above is just for my
| private projects so a lot of notes went into as comments.
|
| For work we do use Confluence but again the quality of our
| notes are not great. From my experience the most important
| notes are the ones that tag a PoC if issue X flares up.
| toddmorey wrote:
| My experience has been the opposite. I used to envy people with
| amazing memories like it was a super power. Then I realized if
| I got into the habit of just writing stuff down, I'd
| effectively have that same superpower. It's the one thing that
| I can honestly say has changed my life.
|
| I use obsidian.md for notes, which I have on phone and desktop.
| Nothing fancy. You just need something that's always with you,
| has great search, and ideally allows you to link notes
| together.
| hnthrowaway0328 wrote:
| I don't really have amazing memories. I just found out most
| of my stuffs are worthless and do not worth being preserved.
| cfr2023 wrote:
| An interesting read and a good share.
|
| Every time I see that other people are contemplating these
| subjects, I notice that they hit on many of my own considerations
| and miss many of the big picture things I consider essential.
|
| I think this will evolve into tribalism of mental models for
| organizing information. More vi vs emacs energy to go around in
| the decades to come.
| riffic wrote:
| this is effectively a very geeked out zettelkasten method.
|
| https://en.wikipedia.org/wiki/Zettelkasten
| moonlion_eth wrote:
| Anytype all day every day
| openquery wrote:
| This is the second piece of content I've ever seen using the same
| numbering system as Wittgenstein's Tractatus, the first of course
| being the Tractatus itself.
___________________________________________________________________
(page generated 2024-01-14 23:00 UTC)