Post B4hqaZPuJ8zRHp4lCi by dabeaz@mastodon.social
(DIR) More posts by dabeaz@mastodon.social
(DIR) Post #B4hqaObCWoUJjfH7mS by dabeaz@mastodon.social
0 likes, 1 repeats
It's come to my attention that John Bradley, author of the "xv" tool has died. I never knew John, but I used "xv" a LOT in the early days of my career as a graduate student working in computational physics. In many ways "xv" might have been more pivotal to my early career than even Python.In this thread, I'm going to offer my tribute to "xv" and say a bit about its role in my 1990s "vibe"-coding project that definitely did NOT involve any AI.This will probably meander a bit. 1/n?
(DIR) Post #B4hqaQ6QwB46Oota6K by dabeaz@mastodon.social
0 likes, 0 repeats
xv was an integral part of day-to-day research activities. Basically, all we did all day was solve differential equations. This involved a ton of simulation, plotting, data analysis, and visualization.xv was this really modest tool that did one thing really well--make it easy to look at images. You could browse images, adjust color maps, flip between images, and so forth.If you walked by a colleague's workstation on any given day, you'd see like 20 instances of xv running at once. 2/n?
(DIR) Post #B4hqaQxFlnJf2dblQm by dabeaz@mastodon.social
0 likes, 0 repeats
I'd recall days where a group of us would be huddled over some workstation flipping between images in xv. Somebody might even hold a ruler up to the monitor to measure distances between features of two images as we flipped back and forth. Of course, there was the problem of finding "that image" from last month. Is it in the "old2" directory or the "new3" directory? Or the "new_old2" directory. Ah hell. So, you'd fire up xv to go searching around for it. 3/n?
(DIR) Post #B4hqaSNAUvdjRIjySm by dabeaz@mastodon.social
0 likes, 0 repeats
I was always amused when I'd see video of scientists at NASA JPL or some similar place talking about a space mission. In the background, you could see that they've got like 50 instances of xv running on their workstation. "Ah xv, I know this!" 4/n?
(DIR) Post #B4hqaUMt4wbTd0cmIq by dabeaz@mastodon.social
0 likes, 0 repeats
Needless to say, xv was one of those tools that was all about "doing the work." It wasn't flashy. It wasn't freaking Photoshop. It was effective. And it worked on my SPARCStation with 16MB of RAM. Seriously, we used the hell out of xv in our research group. 5/n?
(DIR) Post #B4hqaVFTnyGwMKANOa by dabeaz@mastodon.social
0 likes, 0 repeats
Anyways, sometime around 1992, computational power had increased to a point where we started to do a lot more of our simulation work in 3D. For example, running molecular dynamics simulations on a 250x250x250 atom cube of some material.This had nothing to do with xv, but it had a huge impact on all of the other tooling related to data visualization. Specifically, dealing with large datasets, interactive exploration, and even being able to reason about what you were looking at. 6/n?
(DIR) Post #B4hqaW0yxMGmjeOJRA by dabeaz@mastodon.social
0 likes, 0 repeats
Of course, nobody was able to visualize a 15 million point 3D dataset on their SPARCStation. Naturally, the "CS" solution to this problem was to throw infinite money at it. So, at some point, a shiny million dollar SGI showed up over the computer center and you could go use that (which we did).Of course, they put it behind glass in the computer room where people could gawk at it along with the nearby Connection Machine. 7/n?
(DIR) Post #B4hqaWrnmyWLNT6Ulc by dabeaz@mastodon.social
0 likes, 0 repeats
So, sure you could go use the SGI, but doing so meant a) you had to actually had to leave your office b) You had to go outside in the blazing heat (or a midday thunderstorm), and c) you had to bring a sweater because you had to sit next to it in the freaking machine room.Of course, this all assumed that the machine wasn't already being used for a HS student demo or someone playing "driving simulator." Or showing off that dumb Unix file browser thing from Jurassic Park. 8/n?
(DIR) Post #B4hqaYAcwTAnQ8v2kS by dabeaz@mastodon.social
0 likes, 1 repeats
And if you brought your colleagues along for an SGI visit, half of the conversation was like "WHAT?!?!"Because, as mentioned, you're all sitting in the computer room next to a giant supercomputer that needs cooling. But, it all looked pretty sweet. I will grant you that. The kind of setup that would make a funding manager take one look and say "it'd be even sweeter looking if we had a bigger SGI than Livermore." 9/n?
(DIR) Post #B4hqaYlqi4wnHaKllY by dabeaz@mastodon.social
0 likes, 0 repeats
Anyways, while all of this is going on, a more diminutive, yet still kind of cool looking SGI shows up in the corner of the post-doc/grad student bullpen. A mere $250K machine. It too can play Driving Simulator.It sort of works but a) you now had to FTP gigabytes of data from the computer center to local storage over 10Mbit ethernet. b) There's never enough local storage. c) Visualization takes forever. d) Nobody else can use it while it churns. It kind of sucks. 10/n?
(DIR) Post #B4hqaZPuJ8zRHp4lCi by dabeaz@mastodon.social
0 likes, 0 repeats
But, the resulting pictures look kind of awesome. Bad ass really. Realistic lighting. Anti-aliasing. All of the molecules look spiffy--like rendered billiard balls. Stuff, you'd definitely want in a PowerPoint presentation to be shown on the big screen at a Supercomputing conference. Why, what are we talking about again? Oh yeah, xv. Let's get back to xv... 11/n?
(DIR) Post #B4hqaaPaboKWN7w1Lc by dabeaz@mastodon.social
0 likes, 0 repeats
Actually, before I do that, let me add that this "big SGI in the machine room thing" eventually morphed into a "let's make a holodeck for doing VR!" thing. Yeah, because that's exactly what scientists want. Freaking VR! To use when you're able to sneak in sessions between the HS student tours and you're prepared to take the afternoon off afterwards to sooth the resulting migraine.Dear metaverse, 1996 called and wants its dumb idea back. Thanks. I digress. 12/n?
(DIR) Post #B4hqabLj7epnHR8Rxw by dabeaz@mastodon.social
0 likes, 0 repeats
This is getting kinda ranty. Fortunately, I have to drive someone to O'Hare. So, I'm going to calm down while I do that and then I'll be back to say more about how xv fits into this whole story. Stay tuned. 13/n?
(DIR) Post #B4hqacHVep3UAeAb20 by dabeaz@mastodon.social
0 likes, 1 repeats
So, xv had a special "polling" mode that you could use to make it continuously watch a file. If you were viewing the file and the file changed, then xv would update the displayed image.This was the secret sauce for making real-time dashboards.You'd have your physics code dump some kind of output file every so often. Then, you'd stick pgplot or some other tool on a crontab to make an image. Have xv poll the output file.It worked great and involved almost no coding. 14/n?
(DIR) Post #B4hqad8KURJ2oSsmMS by dabeaz@mastodon.social
0 likes, 0 repeats
It was generally understood that no workstation would ever be idle in the group--someone was always solving differential equations somewhere. Usually for tens to hundreds of hours at a time.So, when I say you'd sometimes see someone running 20 copies of xv, they were often running it as a dashboard. Spin up your physics code on 20 machines and then spin up 20 copies of xv to monitor progress. It was pretty hacked together, but it worked decently well. 15/n?
(DIR) Post #B4hqae7eoQMXsfZkx6 by dabeaz@mastodon.social
0 likes, 0 repeats
In light of this, we got to thinking... "what if we could just make our big supercomputing code directly produce output images?" If we could do that, maybe we could wire up xv on the back-end of it and turn it into a cheap "visualization platform."The only problem with this is that we were working on the Connection Machine. I wasn't aware of a vis/graphics library that we could use for this. And whatever we used, it would have to be extremely lightweight due to memory concerns. 16/n?
(DIR) Post #B4hqafCenJxLESvGNs by dabeaz@mastodon.social
0 likes, 0 repeats
In physics, you often see people making "crazy" approximations. For example, "sin(x)" is basically the same as "x" for certain values of "x". A trick you can sometimes use to simplify some gnarly integral. In terms of graphics, we were mostly making rendered spheres. A first-order approximation to an antialiased rendered fancy sphere is a plain 2D solid-colored circle with a single pixel black outline around it.So, maybe we should just do that. 17/n?
(DIR) Post #B4hqag3TcwCtsHdRiK by dabeaz@mastodon.social
0 likes, 0 repeats
About 5 years earlier, I had done a senior thesis project where I explored the Bresenham ellipse drawing algorithm (and also circle-drawing algorithm). I even presented this in my first ever conference talk as a student paper at a math conference in Topeka, KS.So, I had that code laying around. I figured I could adapt it into something where I rendered flat 2D circles into a 3D z-buffered 8-bit pixel array. This could be easily coded in C using just basic arrays. 18/n?
(DIR) Post #B4hqagr6ePuEMCr54S by dabeaz@mastodon.social
0 likes, 0 repeats
So, I add that to our physics code. The problem is how do I turn that into an image? I don't know anything about image formats. However, there it was, xv. We had the source code for xv. xv could write GIF images. So, I simply copied the GIF encoder out of xv, tweaked it a bit, and dropped it in our physics code. A bit of code reuse. 19/n?
(DIR) Post #B4hqaiTmc6RDORxTzk by dabeaz@mastodon.social
0 likes, 0 repeats
Now, maybe I should feel kind of bad about doing that. But, xv lifted its GIF encoding code from somewhere else too. The original source had comments crediting it to Michael Mauldin and David Rowley from 1989. But, then there was another comment below that where they credit it to an IEEE Computer Article from 1984. So, it's not really the same as installing a dependency, but definitely in the spirit of early open source. 20/n?
(DIR) Post #B4hqajVEoBCCZFe9tw by dabeaz@mastodon.social
0 likes, 0 repeats
With the physics code now able to render "spheres" and make images, we just need to solve a certain transport problem. The Connection Machine was physically in a different building and on a different network. You couldn't really log into it in the traditional sense.However, you could open up socket connections. So, the physics code was modified to open a network socket that was connected to a small "server" program running on your desktop machine. 21/n?
(DIR) Post #B4hqakM3dnRlD4MLEO by dabeaz@mastodon.social
0 likes, 0 repeats
The idea is that the server program running on your desktop would watch for images coming from the CM-5 and write them to a file. Behind that, you'd have xv running in polling mode. Suddenly, we've slapped together this visualization dashboard. The whole thing was coded up in about a week. Nothing super fancy. Mostly standard parts with a bit of cut/paste. 22/n?
(DIR) Post #B4hqalDES5ytrzEo76 by dabeaz@mastodon.social
0 likes, 0 repeats
Now, you might ask how well something like this worked?In the SGI era, producing a single image from simulation data might take 5-10 minutes. You'd first have to FTP the data to the SGI. Then, you'd need to launch the vis tool. And it would render for awhile. And you'd sit there. Assuming that you could even get access to the machine of course.This new approach reduced all of that time down to about 5 seconds. And it worked from the comfort of your office. 23/n?
(DIR) Post #B4hqam6X8UDWdV6yJM by dabeaz@mastodon.social
0 likes, 0 repeats
People looked at this arrangement and didn't quite know what to make of it. Was it the dumbest coding project ever? Maybe. But, it was hard to argue with the results.We started adding more visualization capabilities into the physics code. This included a minimal "interactive" CLI mode where you could drive the visualization in more interesting ways.Essentially you could type various commands and as long as you were willing to wait about 4-5 seconds, xv would update its output. 24/n?
(DIR) Post #B4hqan31d10NYuTgTw by dabeaz@mastodon.social
0 likes, 0 repeats
The "interactive" mode grew more features over time and eventually turned into something that looked a bit like a scripting language interpreter.I demoed this setup to all sorts of people including, at one point, the US Deputy Secretary of Energy--who honestly had no idea that were probably looking at some extremely primordial incarnation of a computational notebook. They were unimpressed. 25/n?
(DIR) Post #B4hqaoHx10XRPUT7Nw by dabeaz@mastodon.social
0 likes, 0 repeats
Anyways, around this time I read a short blurb about Python that Paul Dubois had written in "Computers in Physics.""Oh, that sounds cool. Maybe I should use that for my CLI!"And that was that... a small little thing that started by hacking around with xv. Anyways, thanks for xv, John. 26/end.