[HN Gopher] VT330/VT340 Sixel Graphics
___________________________________________________________________
VT330/VT340 Sixel Graphics
Author : ecliptik
Score : 53 points
Date : 2023-05-14 07:25 UTC (1 days ago)
(HTM) web link (vt100.net)
(TXT) w3m dump (vt100.net)
| kps wrote:
| For those who take the weekend off, yesterday's Sixel discussion:
| https://news.ycombinator.com/item?id=35936331
| ape4 wrote:
| There are definitely lots of interesting uses once this reaches
| critical mass. A "high definition" progress bar, showing photos
| of course, thumbnails when doing an `ls`, graphical web browsing,
| plotting charts, nicer looking gtop, things we haven't even
| dreamed of.
| prmoustache wrote:
| If it had to reach critical mass, it would have done so ages
| ago. This is 45y old tech.
| wkat4242 wrote:
| When this tech was invented it was less relevant than it is
| now. For the following reasons:
|
| - It required special expensive terminals, the VT330 (mono)
| or VT340. Those were not ubiquitous, the VT320 was the bread
| and butter type. Later VTx20 models also didn't gain support
| for it (usually the lower range models got premium features
| from the previous ranges like multiple connections / pages
| but this one didn't 'trickle down'). So there wasn't so much
| incentive for developers to support it as the installed base
| was very low.
|
| - It required a long time to send all the information,
| because terminals were still on slow serial connections. The
| VT320 could only do up to 19200 bps which was not even super
| fast for text at 80x25! Sixel images would take ages to
| transfer. Even though we had different expectations of
| response speeds then, sixels were slow by the standard of the
| day.
|
| - Colour highlighting (or even monochrome "bold"/ultrabright)
| was not even common in most commandline tools back then, even
| LS. Hell, on HP-UX in the 90s I couldn't even use the cursor
| keys in the default shell for command history and line
| editing! Mod cons we take for granted now were yet to be
| invented then.
|
| - Thumbnail processing as the parent suggests would take a
| lot of space and processing speed. My quota at college was a
| few megabytes. In fact I hardly had any images. I think I had
| just one, a nice Enterprise-D render that I used as
| wallpaper. It's not like we had thousands of images to sift
| through like we do today. Of course the web changed that
| rapidly but at that time text terminals were already on the
| way out.
|
| Some apps used it for stuff that was worth the wait. Like the
| print preview in word perfect. But many apps didn't bother
| because the above constraints just made them infeasible.
| Sixels only really took off after their original terminals
| were long gone. And really in this day and age we should be
| able to come up with something better.
| kevin_thibedeau wrote:
| You can already make detailed bars with Unicode block drawing.
| tyingq wrote:
| The Tektronix vector graphics that are also built into xterm is
| an neat side journey if Sixel is interesting to you.
| YZF wrote:
| I've used a few Tektronix terminals back in the day (1980's)
| with an IBM mainframe and some graphing library (the name
| escapes me). What's old is new again?
|
| We could build some pretty nice GUIs in the terminal maybe ;)
| kragen wrote:
| unfortunately sixel seems to be seeing a kind of revival by
| retrocomputing enthusiasts who want sixel support to become
| ubiquitous
|
| excerpt from my comments yesterday at
| https://news.ycombinator.com/item?id=35940188:
|
| > _some technologies, like the unix shell, smalltalk, and tcp
| /ip, make hard things easy and easy things possible. others, like
| retrocomputing, code golf, and malbolge, make easy things hard
| and hard things impossible. sixel makes easy things hard and hard
| things impossible, ...._
|
| > _that 's a fun way to spend my time when i choose to (e.g., in
| https://asciinema.org/a/390271 i did real-time 3d graphics in a
| unicode terminal emulator with braille) but it's not how i want
| my primary user interface to my computer to work_ ...
|
| > _sixel is oulipo programming; encoding your graphics in sixel
| is like writing a novel without the letter 'e'. btw, a wonderful
| article about oulipo programming is
| https://100r.co/site/working_offgrid_efficiently.html_
|
| > _sixel is an art project, not an engineering project_
| flohofwoe wrote:
| > unfortunately ...
|
| Is there a better alternative for pixel graphics in the
| terminal? If yes I'm all ears. The way Sixels encodes pixel
| data sucks, but currently it seems to be the only way to render
| pixels in the terminal (and that's implemented in more than one
| terminal application).
| kragen wrote:
| you seem to be attempting to define 'terminal' as 'vt100
| emulator', which unfortunately condemns you to only
| unreliable solutions like sixel that are too inefficient to
| provide smooth graphical interaction and full of
| compatibility problems. you can plot a graph from a remote
| machine, sure, but if you want to use your mouse to zoom in
| on it, you just bought a one-way ticket to jank city
|
| but i've used graphical terminals like ncd x terminals since
| 01994, and they provided smooth graphical interactions even
| over shared 10-megabit ethernet. current xorg and xwayland do
| a great job of emulating those terminals, with most of their
| deficiencies fixed, and ssh has built-in support for
| transparently passing along the needed credentials
|
| xpra works even better, supporting h.264 compression and
| reattaching to applications you've detached from
|
| the big problem with x11 is that trying to write a program to
| do something simple like display an image or draw a bar graph
| ensnares you in five pages of bureaucracy
|
| to solve that problem i wrote yeso:
| https://gitlab.com/kragen/bubbleos/blob/master/yeso
|
| here's a program to display a png with yeso:
| #include <yeso.h> int main(int argc, char
| **argv) { ypic img = yp_read_png(argv[1]);
| if (!img.p) return 1; ywin w = yw_open(argv[1],
| img.size, ""); yp_copy(yw_frame(w), img);
| yw_flip(w); for (;;) yw_wait(w, 0); }
|
| it also runs on the linux framebuffer without x-windows
|
| here's a color generalization of munching squares in yeso in
| python: import yeso def
| munch(t, fb): for y, p in enumerate(fb):
| for x in range(fb.size.x): p[x] = (t &
| 1023) - ((x ^ y) & 1023) if __name__ ==
| '__main__': with yeso.Window(u"CPython munching",
| (256, 128)) as w: t = 0 while
| True: with w.frame() as fb:
| munch(t, fb) t += 1
|
| it uses python context managers so you don't forget to
| yw_flip()
|
| there's also a luajit binding
|
| the example programs above are non-interactive, like
| virtually everything sixel that isn't full-screen; here's a
| lua program to test mouse interaction:
| local yeso = require 'yeso' function bounds(a,
| b) if a < b then return a, b else return b, a end
| end yeso.Window("mOuso", {256,
| 256}):run(function (w) local m, dragStart =
| {x=128, y=128} while true do
| for ev in w:events() do if ev.isMouse
| then m = ev.p
| if ev:button(1) and dragStart == nil then
| dragStart = {x=ev.p.x, y=ev.p.y}
| elseif not ev:button(1) then
| dragStart = nil end
| end end
| w:frame(function (f) f:fill(0x7f7f7f)
| if dragStart then local minX, maxX
| = bounds(dragStart.x, m.x) local
| minY, maxY = bounds(dragStart.y, m.y)
| f:sub({minX, minY}, {maxX-minX, maxY-minY}):fill(0x336699)
| end end) w:wait()
| end end)
|
| as for embedding interactive graphics in a text stream, i'm
| still trying to figure out the bubbleos solution to that. but
| i'll get there, and the result will be usable over ssh, while
| sixel probably hasn't gotten there even for low-bandwidth
| interactive localhost cases and probably never will
| hello_computer wrote:
| > but i've used graphical terminals like ncd x terminals
| since 01994, and they provided smooth graphical
| interactions even over shared 10-megabit ethernet
|
| The problem is that with the predominance of toolkits like
| GTK & Qt in the *nix application space, which pre-rasterize
| everything for easier cross-platform compatibility, X
| forwarding is nowhere near as efficient as it once was.
| Under present circumstances, the h.264 route is probably as
| good as it gets.
| hulitu wrote:
| Real users don't use GTK and QT. Real users use Xt,
| Xaw(3d) and Motif.
| hello_computer wrote:
| Real programmers don't sully their hands with libraries
| or even surface handles, preferring to rowhammer the
| primary video buffer instead.
| kragen wrote:
| xt and xaw are in the same retrocomputing penalty zone as
| sixel: what is astonishing is not that programs written
| using them do so little and work so badly, but that they
| ever work at all
|
| motif is not quite as bad, hiding some of the worst parts
| of xt, but it's still unnecessarily clumsy
|
| unfortunately, unlike sixel, there's already enough
| software written that depends on them that we have to
| keep supporting them
| kragen wrote:
| the h.264 route used by xpra is dramatically better than
| remote x-windows has ever been; x11 forwarding has never
| been great, for reasons including bufferbloat,
| compatibility problems, and excessive round trips, even
| if it's not nearly as bad as sixel
| hello_computer wrote:
| i believe MS RDP has also taken that approach
|
| https://techcommunity.microsoft.com/t5/security-
| compliance-a...
| kragen wrote:
| interesting, i thought rdp was a vnc-like protocol with
| minimal compression, but i don't think i've actually
| looked at it
|
| looks like you're right:
|
| > _With RDP 8.1 we introduced an AVC /H.264 mixed mode
| which in addition to using RemoteFX Media Streaming,
| extended support for AVC/H.264 to images as well, while
| text is compressed using a proprietary Codec. This mode
| is used by Windows RT devices running Windows 8.1 and
| some 3 rd party RDP implementations._
| akkartik wrote:
| kragen, what's the current status of BubbleOS?
|
| Also, I'm curious what you think of Arcan (https://arcan-
| fe.com)
| kragen wrote:
| i haven't done anything on it in years but this
| discussion is stimulating
|
| was it you that linked me to arcan the other day? i've
| been studying it. i think it's interesting
| hello_computer wrote:
| https://news.ycombinator.com/item?id=35951733
| kragen wrote:
| base64-encoded graphic files are obviously a much better
| idea than sixel but graphics really benefits from animation
| and interactivity
| vidarh wrote:
| I don't think the issue you describe there is very relevant for
| typical Sixel usecase. Yes, it's not efficient to play full
| screen high frame rate animations that way, but the more
| typical use would be things like being able to pipe log data
| into a command line tool and get a plot out inline in the
| terminal, or similar type of uses where you'd otherwise use
| characters.
|
| You can do all kinds of things to make something better, but
| the two things that matters for this usecase is being able to
| do it over ssh _and_ wide enough support that you can start
| relying on it. Better performance or more features only matters
| if both of those are met.
| kragen wrote:
| ssh has supported x11 forwarding since at least 01996, and
| i've frequently forwarded vnc over it as well
|
| i think the problem here is that x11 is shitty at 'inline'
| (things tend to open annoying new windows) and a modern
| terminal cosplaying a vt100 is shitty at graphics
|
| sixel doesn't solve that problem; your terminal emulator is
| still shitty at graphics
| vidarh wrote:
| Being shitty at graphics doesn't disqualify it for what
| most people want to use Sixel for, so it doesn't matter.
| X11 isn't an option not just because it opens windows but
| because it'd have been too much hassle even _if_ it was
| inline for the type of simple things people want Sixel for.
|
| Just something that's marginally better than emulating
| bitmaps with unicode characters is sufficient, as long as
| it's inline and simple to output.
| kragen wrote:
| in https://news.ycombinator.com/item?id=35951674 you can
| see that with yeso i've reduced the hassle of viewing a
| png in x-windows to ten lines of c, and you don't have to
| worry about terminal emulator incompatibility,
| interleaved background-process output, error recovery, or
| dithering
|
| (you do have to worry about bugs in yeso, of which there
| are several)
|
| if you don't like c, in cpython it's four lines of code
| with yeso from yeso import * w
| = Window('hello, world', (406, 220))
| w.frame().copy(read_png('admu_shell.png')) while
| True: w.wait()
|
| this requires a little bit of tweaking to work in pypy
|
| also it works on the linux framebuffer as well as in
| x-windows
|
| not being inline is a harder problem to solve
| hello_computer wrote:
| If I were retired, I'd write a terminal that passes an extra
| bi-directional pipe (fd 3?) to programs for an interactive
| jupyter-like REPL experience. From that pipe, programs could
| request a canvas from the terminal (mmapped file?, shm?,
| another pipe?), and the terminal could send per-canvas
| mouse/key actions back to the program. Obviously, I haven't dug
| into the details yet, but it's on the list.
| kragen wrote:
| that's an interesting approach
|
| i think you could probably get pretty far by creating a new
| x11 socket with fresh xauth for each new program launched;
| then, instead of opening new floating overlapping windows for
| them, embedding the display of whatever windows they create
| in your notebook, just below below the command that created
| them
|
| a bytestream protocol has some real advantages, including
| being able to forward it over a network socket, including an
| ssh-forwarded socket, or providing a gui for whatever little
| embedded electronic gadget you build and connect to your
| terminal over spi or usb serial
|
| i had previously rejected it as too inefficient for the
| localhost use case, but in
| https://news.ycombinator.com/item?id=35693474, and some later
| protocol-design ideas i haven't shared yet, i convinced
| myself that you can probably make it efficient enough for
| anything that doesn't use custom shaders
|
| is this something you'd be interested in talking about in a
| less hostile environment
| hello_computer wrote:
| Is the email on your site still good?
| kragen wrote:
| yes, but sometimes i fail to notice email among the spam
|
| unless you ran across the pobox address, which has been
| dead for... a while
|
| the canonical.org address is the correct one
| msla wrote:
| Library you can use to generate these images:
|
| https://github.com/saitoha/libsixel
|
| Plenty of links to other projects.
| Per_Bothner wrote:
| Sixel has the one advantage of being mplemented in xterm and a
| modest number of other terminals. Otherwise, it's a pretty bad
| format: Inefficient. Unclear and inconsistently implemented
| specification. All images have to be a multiple fof 6 pixel rows,
| which may not align with either image height or character height.
|
| Some terminal implement some other protocols, but attempts to
| specify a standard have failed. There are some tricky issues,
| such as: When does an image or part of an image get erased? Can
| you write text on top of an image and if so how are they aligned?
| What happens if you write an image on top of existing text? On
| top of an existing image? How does scrolling affect things? What
| happens to the image on window resize or zoom? Can you reliably
| update part of an image?
|
| DomTerm (https://domterm.org) supports images in two ways: For
| sixels, it creates a canvas element as an "underlay". As it is
| earlier in he rendering order, following text can be written on
| top of the image.
|
| DomTerm also supports general HTML, including IMG elements. (Of
| course it is safety-scrubbed.) This is quite flexible, but it
| doesn't overlap regular text (without some contortions). DomTerm
| allows different lines to have different height, which is
| beneficial for "printing" an image without having to pad the
| output to an integral number of rows.
|
| Neither of these DomTerm protocls are suitable for a portable
| protocol that can be widely implemented. Two other protocols that
| I'm aware of are the iTerm2 protocol and the Kitty protocol. Both
| are bit on the complicated side, but a subset of one of those
| might be an acceptable "standard".
| hello_computer wrote:
| Kitty / Wezterm / iTerm2 / mintty and a few others already
| support base64 encoded gif/jpg/png, which are more space
| efficient, and easier to splat as b64 blobs than to dither and
| sixel-encode. Sixel needs to die.
| wkat4242 wrote:
| Funny thing, as popular as sixels are now, they were pretty
| useless back in the day. It would take ages for a picture to
| appear on a VT terminal with those slow serial speeds (VT320
| doing 19200 baud max! - not sure about the 330/340). Especially
| with colour. I'm pretty sure the later ones (I have a VT520
| myself) didn't even have it anymore!
|
| I know word perfect for Unix used it for the print preview but I
| don't know any other app from those days that used them at all.
| The same with the double width or height fonts the VT series
| offers. I've only ever seen them working when running vttest.
| They could be very useful, but in practice I've never seen them
| actually used for something other than testing.
|
| These days with pretty much unlimited speed sixels actually
| become useful. But they don't support full 24 bit colour so
| another technique is probably better for today's needs. For
| example it would be great to be able to define a terminal region
| and send images and video to it with full hardware acceleration
| and compression (H.264 or similar). I think there's still a good
| usecase for terminals, but holding on to sixel feels a bit too
| much of a kludge to me. If they were super abundant and popular,
| sure. But they never actually were.
|
| I think this is also why kitty chose to build a custom protocol
| instead of implementing sixel.
| JdeBP wrote:
| I wonder about the popularity. There are a number of examples
| in the long Microsoft Terminal issue discussion about sixels,
| but very little of it seems to be day-to-day tasks. Whereas
| people are getting inventive with Unicode in the little ways,
| from clever progress bars and spinners to tiny but importantly
| everyday things like arrows in shell prompts.
|
| I just myself wrote a pseudo-window login dialogue (a
| replacement for the login(1) program for framebuffer terminal
| emulators that can do Unicode) using MouseText from the Apple
| IIc, which got added to Unicode version 13 in 2020.
|
| * https://tty0.social/@JdeBP/110373938216006335
| wkat4242 wrote:
| Yes the spinners that are made of braille are nice. And
| graphs.
|
| But unicode support is pretty ubiquitous now. It works great.
| Despite advanced unicode (ligatures, emoji) not really
| playing nice with monospaced fonts, most terminals handle it
| very well.
| wkat4242 wrote:
| By the way yes it's relatively popular, out of the various
| options. Sixels are more popular than Kitty's protocol and
| what iTerm2 has.
|
| Though more and more console libraries are supporting
| multiples of these which will help get more reach for all of
| them.
| crispyambulance wrote:
| I find it disappointing that an article about Terminal graphics,
| does not, you know, include an image of what it even looks like!
|
| Is that just uninteresting to people that might want to use it?
| MisterTea wrote:
| It's not an article. It's documentation.
| JdeBP wrote:
| It's not an article. It's a digitized version of one section of
| a chapter of a book from May 1988. Which is in turn volume 2 of
| a 2 volume manual set, with the diagrams in volume 1.
| teddyh wrote:
| I guess that Wayland users have to get their remote graphics
| _somehow_.
| kijiki wrote:
| Huh? Does: waypipe ssh remotehost guicmd
|
| not work for you?
| rwmj wrote:
| If only Wayland worked like MGR and was just an extended set of
| graphics primitives sent over stdin/stdout!
| kragen wrote:
| this would be wonderful
| CharlesW wrote:
| How is this proprietary standard designed for dot matrix printers
| a better choice than using standards-based image formats with
| their associated rich ecosystem of tools? In which use cases is
| converting a lossy JPEG image to Sixels considered a win?
|
| https://iterm2.com/documentation-images.html
| altairprime wrote:
| Was this reply intended for a comment? The documentation page
| linked doesn't make any claims about or comparisons to other
| solutions, to the best that I can determine, and so it's
| unclear to me what statement this is in reply to.
| CharlesW wrote:
| > _Was this reply intended for a comment?_
|
| No, I'm just asking why there's interest around Sixel when
| there are protocols (like iTerm's Inline Images Protocol)
| that use standards-based image formats.
|
| I'm not a heavy terminal user and don't see much value for
| images in my CLI, so one very possible explanation is that I
| don't understand the use cases where this "interesting"
| format (without a guaranteed pixel aspect ratio, as I
| understand it) shines.
| flohofwoe wrote:
| Sixels is supported by more than one terminal implementation,
| while the example you posted only works in iTerm2 (and IIRC
| only accepts a filename as input, but doesn't offer a way to
| send image data directly as a stream of characters through
| stdout).
|
| I would welcome a proper pixel graphics / framebuffer standard
| for terminals though.
| kps wrote:
| I know that at least Konsole, mlterm (which I use), and
| wezterm support iTerm's method.
|
| Unfortunately iTerm's method does not have a googlable name
| like 'sixel'.
| kps wrote:
| _Technically_ it may not be, but xterm has supported it
| approximately forever, and xterm is the de-facto standard
| minimum terminal emulator.
| CharlesW wrote:
| That's very helpful to know, thanks! So Sixels are a
| reasonably-popular de facto standard already, which I did not
| understand.
| kps wrote:
| They date to DEC terminals of the early '80s. DEC was
| involved in developing X Windows (competing with other
| windows systems like Sun's NeWS) and xterm supports many of
| the DEC terminal extensions to ANSI X3.64.
| vidarh wrote:
| Because unlike the alternatives, Sixel has relatively
| widespread support, and that matters far more.
| bawolff wrote:
| Everyone uses terminals to get away from complex graphics, then
| people reinvent (uncover? I know this is old tech) a way to do
| that but poorly...
___________________________________________________________________
(page generated 2023-05-15 23:02 UTC)