[HN Gopher] Show HN: Seen - Virtual list rendering with 1M+ notes
___________________________________________________________________
Show HN: Seen - Virtual list rendering with 1M+ notes
Edit: Thank you so much for taking a look at Seen! Yes, I know,
it's really slow. And scrolling does not render notes instantly.
And the title is a bit clickbait-y. I'll try my best to improve on
it though. This is just a preview, the most basic version of Seen,
and it's getting there. But, oh God, I almost got a heart attack
when I opened up Vercel Analytics and saw ~1,500 page views, up
from 10 or 12. Thank you again! It's such a weird experience
showing something to the world for the first time as a 16-year-old
and seeing so many people look at it. Hello HN! I've been working
on creating a new note-taking app called Seen. Right now, it's
really just a preview: virtual-list rendering ~1,000,000 notes in a
masonry layout, while trying to minimize CPU and memory usage as
much as possible. Seen currently has: - A client-side search
feature - Creating, updating, deleting the topmost note (locally;
no changes are saved to any server) - Automatic repacking of the
masonry layout when the window width changes - A lot of bugs, I'm
sure I reached a point where I thought I was ready to show the
Seen to the world, and this is it. It's pretty basic, I know, but
I'm really proud of it. Seen was born out of an experiment to see
just how fast things can become if you use the right techniques,
and will (hopefully) achieve this dream sometime in the near
future. My vision for Seen is to create something that can handle a
lot of notes while using the least amount of resources. If you've
got any suggestions or bugs, they're welcome in the comments! A
little about how Seen works: during the first visit, Seen caches
note heights for the specific window width. Seen also caches the
HTML output for the Markdown notes. It caches height by the key
SHA256(title + note content), and the Markdown-to-HTML renders by
the key SHA256(note content). It uses this cache every other time
instead of recalculating, meaning that if most of the content
remains unchanged, Seen can render things pretty fast. Caches are
saved to IndexedDB. When window width changes, the amount of text
that can fit in a line also changes. So, Seen caches heights for
specific window widths as well, putting the width in the key.
Scrolling triggers an event handler that calculates the current
viewport and finds & renders all notes that are visible (or
partially visible) with a buffer of a 600px. When a note is
deleted or inserted, the entire layout is re-rendered. I plan on
optimizing this to only re-render the specific column, which is
what happens anyway when a note is updated. Updated notes create
new caches (as they should).
Author : _bittere
Score : 36 points
Date : 2025-02-10 09:22 UTC (1 days ago)
(HTM) web link (seen-v2.vercel.app)
(TXT) w3m dump (seen-v2.vercel.app)
| lelandfe wrote:
| If I tap and hold the scrollbar on my iPhone and scroll around,
| I'm looking at a blank page the whole time because the virtual
| list is waiting for me to finish.
| isoprophlex wrote:
| they're _virtually_ all there, though! :^)
| _bittere wrote:
| Oh yeah, that's a known bug. You're right, the scroll event
| handler is waiting for you to finish scrolling. I'm looking
| into it though!
| gkbrk wrote:
| Is it fast by default? Every time I scroll up and down I'm
| looking at empty space while the notes load in. And it's not just
| initial network delay, even if I scroll over already loaded notes
| it takes time to display them.
|
| I think it would be a lot faster to do this with native HTML.
| _bittere wrote:
| Native HTML? As in actually rendering the 1M divs into the DOM?
| Etheryte wrote:
| Out of curiosity, why SHA-256? I would wager using a non-
| cryptographic hash would be faster, no? In this context you don't
| need any of the crypto properties of it, so that could be an easy
| performance improvement for your cache.
|
| It seems currently the scroll struggles when you jump around,
| e.g. when you click to the bottom of the scroll bar area, the
| scroll will jump twice, first as you click and the second time as
| it renders. This is not a big issue if you continuously scroll,
| but is very noticeable if you use page down or click on the
| scroll bar. At least on my system, there is a noticeable delay
| between scrolling and the items being rendered on screen, that
| might be a related issue?
|
| Those notes aside, I could definitely see value for this in cases
| when you want to get a glance overview or similar of a large
| number of notes, it's a cool idea.
| _bittere wrote:
| No specific reason: it's a great hashing algorithm, and the
| first that came to mind. I'll look into other hash functions.
| Thanks for the suggestion!
|
| Yes, I believe the two are related (and related to the scroll
| event handler). I'll look into that too!
| cmgriffing wrote:
| FNV-1a might be worth looking into. It's pretty fast
| billconan wrote:
| how do you cache heights without rendering? It seems to be a
| chicken-egg problem, you need to know the heights to test
| visibility, but without rendering, you can't know the heights? Or
| heights are obtained by invisible rendering in a hidden div?
| bloomingkales wrote:
| Working with fixed heights makes virtualized lists more
| predictable, easier to implement.
| _bittere wrote:
| While that is true, it's not guaranteed that all notes will
| be of the same height. Some notes might have more content,
| some might have less; which is why we need all the fancy
| JavaScript rendering.
| _bittere wrote:
| What Seen does is render notes off-screen first, then pack them
| up in the container. So yes, it does use a hidden div in a way.
| Some CSS styling does not let you measure the height, which is
| why Seen uses an off-screen div.
| billconan wrote:
| Thank you for the tips. the off-screen div can't set display
| to none right? otherwise the browser will skip rendering?
| tobyhinloopen wrote:
| It is not even rendering 1,000,000 notes; just those visible on
| screen. The illusion breaks pretty quickly as soon as you scroll
| a bit. I feel like it could be much faster.
|
| Since you can't CTRL+F them, they're not rendered.
|
| Also... who wants to render 1M notes? Maybe if you can CTRL+F
| them...
| _bittere wrote:
| Yup, that is true. And yes, it could really be much faster.
| This is an early preview though, and I'm working on it!
|
| Also, Seen does have search. The problem with making it work
| with Ctrl+F would be to render all 1M notes (or at least some
| "shadow" version of them), which would really slow everything
| down. But again, maybe there is some other way - speed is one
| of the major concerns, and I'm working on it!
| nickzelei wrote:
| Hm, this is cool but I'm not seeing the speed. I loaded this on
| my phone and did one scroll flick with my thumb. Immediately saw
| white screen and took multiple seconds to render the items. iOS
| iPhone 15 pro. I've run into this same problem before using
| tanstack virtual table and it's usually do to a few things:
|
| - Too much on band calculation blocking the renderer when the new
| items come into view.
|
| - Not enough preloading of the items off screen
|
| - Incorrectly calculating the virtual scroll height which causes
| inefficient re drawing during render.
|
| Not sure what stack you're running but it seems the table could
| use some optimization.
| _bittere wrote:
| You're right, those are some trade-offs I've had to make. Not
| using Tanstack Virtual Table though, wrote a custom
| implementation with some help from our friend ChatGPT. I'll
| look into making it render faster!
| G_o_D wrote:
| i have created one such in past, though not <1s but one note -
| 167239 character long, total 662 notes takes 10s on initial load
| from server and rendering then its fast, CRUD operations doesnt
| lag, + filtering live search also fast enough fiktering 662
| notes, only 2s
| spiddy wrote:
| This reminds me of https://news.ycombinator.com/item?id=42342382
| on the blog https://eieio.games/blog/writing-down-every-uuid/
| they mention how they tackled various challenges such as
| rendering.
| Kuinox wrote:
| I have a 1kEUR PC and this takes half a second to display
| anything when i scroll.
| _bittere wrote:
| You're right The speed is a major issue for everybody. I'll try
| my best to improve it. Thanks for checking Seen out btw!
| shultays wrote:
| No offense but I feel like this is an overengineered solution
| that misses the problem. It may efficiently render millions of
| notes I will not take in my life but it is extremely ugly with
| the slow loading/populating the screen as you scroll. And scroll
| goes crazy if you try to drag and scroll.
|
| I would prefer a note taking app that can render maybe 100 notes
| but doesn't have that scrolling behaviour.
| _bittere wrote:
| Thanks for the feedback! Yes, that is true; the scrolling is a
| big issue right now. Working on it though. Thank you for
| checking out Seen!
| new_user_final wrote:
| Similar-ish project shared on HN. https://everyuuid.com/
| helb wrote:
| I get zalgo-like glitches (overlaying notes) when resizing the
| viewport width in Firefox. Looks kinda cool though:
| https://i.vgy.me/JQsqZO.png
| mousetree wrote:
| Cool name! (CTO of seen.com here)
___________________________________________________________________
(page generated 2025-02-11 23:01 UTC)