http://ascii.textfiles.com/archives/5443

ASCII by Jason Scott

Jason Scott's Weblog

Archiving in the Time of Streaming -- February 4, 2023

Few things are harder to start than a narrative of the thing you "do"
and what all the ramifications of it are. For people who don't make
it the center of their lives, your thing is already hopelessly
complicated and gaining enough of a foothold to feign interest is
olympic-level effort. For people who do make it the center, your
description fills them with a never-abating dread that you're going
to get the explanation "wrong", or that you're going to give the
Outsiders a bad impression.

So let's begin at the beginning, again.

My parents' divorce, taking place in the realm of the beginning of
the 1980s, was not in any way friendly and in fact rather
contentious. I wasn't yet a teenager, and I was the oldest, and most
notable for the purposes of this story, my mother gathered up her
children and scooted off to first a hotel and then a location with
family, without telling my father where she had taken us. In modern
parlance, the term is "child abduction", although I'm sure my mother
didn't think that's what she was doing. Ultimately, connections were
made, a nasty divorce proceeding happened, nobody was shot, stabbed,
restrained, or attacked.

But in many presentations I've given, I mark the moment of being
scuttled out of the family home into the great unknown of Outside The
Neighborhood as being when my focus and awareness of the world
fundamentally changed, because as she put things into the station
wagon and tearfully told her kids to get ready to go, she also asked
us to take what we needed. Which is a difficult question for someone
in the realm of ten years old.

I needed a blanket and my dog.

Woven into the simple declaration take what you need are a host of
world and perspective-changing understandings of what you and need
are. Let's not overplay the trauma aspect of it all, and focus on the
concepts being dropped on me and my siblings, which first and
foremost is a crumbling of rock-solid foundations. Foundations of
family, to be sure, but also possession, consistency and location.
Home was no longer an immutable realm, but split into multiple
locations, one inevitably a favorite, with the other a shadow of the
concept of home. Possessions stopped becoming things you could walk
out of sight from and be a hundred percent sure (maybe 90-95 percent
sure) they'd be there when you came back. And most effective on me
was the idea that lack of effort to maintain the talismans and
protocols of representation would result in a void. Put more simply:
You're the keeper of yourself and of what matters. There is no
consistency that will protect you.

[18161968638_ec90bf0bee_c]

Like a lot of knowledge, this came at some level of heavy price,
although I again stress the price is one many others pay at much
greater cost. The fact remains that both my parents bent over
backwards to provide opportunities, to ensure comfort and food and
connection as best they could, and while I can sit back in my fifties
and leave notes as to my parents' actions in their thirties, my own
thirties were spent in hacker conventions, shooting films and
stumbling through my own home life, so I'm quite the unqualified
judge.

This is all to say that when Chris Boufford showed me an acoustic
modem in his grandparents' spare room and how, by calling another
number in Mount Kisco and putting the receiver of the phone into a
cradle you'd suddenly get words on the screen, I had lived a life up
to that point where two thoughts came almost simultaneously:

This is amazing.

I need to save this before it inevitably disappears.

I couldn't have known that in the years afterwards, I would in fact
collect so many artifacts from this thing, this concept of the
Bulletin Board System, that I'd meet many of the people responsible
for it existing, that this pile of artifacts would get a name and a
branding, and that I'd collect it all so hard that I'd end up being
associated with the concept of it for a lifetime. But I definitely
understood, taught as I was during that painful childhood lesson,
that inaction would be tantamount to approving its destruction and
disappearance.

Calling as I did to many BBSes in my teen years, I'd focus on the
textfiles, the message bases, the downloadable files, because they
felt, I think, like special missives from beyond my little life and
easily kept on floppy disks. When disks were the main way to store
computer data for home users, you would encounter two kinds of
people: folks who kept a small, special set of floppies representing
what they needed, and others who had vast, terrifyingly large
collections representing not what they needed but what they thought
should be held in trust. I've gone through a lot of these collections
over the years, and have seen rooms full of these things, hundreds
and sometimes thousands, representing homemade bunkers of data. I had
my own bunker, and while it was only a few dozen, the relative
smallness of textfiles meant that each disk could hold many, many
examples of these artifacts I thought deserved whatever long life I
could provide.

[EfKWBo3WkAEXCsO-1024x678]

At what point does a hoarder of data, driven by a sense of loss and
of fear of same, turn from a mere accumulation of piles and end up
with something resembling an archive?

I can point to various choices I was making in my teenage years with
textfiles as the whole endeavor being more than just a private copy
of things I liked: creating capsule descriptions of the textfiles for
my own BBS, giving them unique extensions (HUM, PHK, HAC, PRO) to
classify them in general genre headers, and attempting to keep the
authorship and context of the files in the form of "buffers", just
saving all the output of a BBS to keep a record of where the files
came from. I can find in my stacks actual essays I'd written about
what these files were, and throwing my own writings in amongst
everyone else's so my own works wouldn't be lost.

My college years were where all of it could easily have come to
naught. New city, new goals, explorations and discovering my new set
of interests could have led to my younger days and their collections
being scattered to the wind. In a series of lucky maneuvers and
chances across that period, I did not lose my textfiles and floppies
and printouts, and they persisted for about 8 years in my hands and
in the hands of a friend, David Weinstock, who kept things I was
"done with" and critically asked me if I wanted them back. And by the
time he asked, I did want them back. My own collection and archive,
itself, came at the same risk of entropy and disappearance but the
spinning wheel fell on "save" and I had them all again.

In my twenties is when I start creating The Warrens.

I don't have a lot of handy photos of all the Warrens, and maybe
that's a compilation I need to add back here as I find them again.
But over and over again, I turn wherever I'm living, or a single room
within it, into a cramped, filled-to-the-brim, often deeply
concerning space of materials. These will be favorite books, personal
collections of memories, computer hardware and software, and an
ever-growing set of amusements and pieces into my own functioning
workspace. "Work", in many cases, being the day-to-day activities of
a geek browsing online things or playing with some sort of toy or
tool, but surrounded by all the possibilities and options at arm's
length should my shifting focus switch to a new attraction.

I create Warrens a lot. Casting my net backwards, I count six of
them, and each one a tiring memory to me, as I consider how much
effort it took to build them up, and then inevitably pull them apart.

[Jason-Scott-1]
[3065759650_c8722c2a23_k-1024x683]
[2474798212_1c11371308_o]

While it's fundamentally silly to think each Warren was going to be
the absolute last, there was definitely an approach and plan with
each one to improve what came before. Bins instead of piles, thematic
groupings instead of simple shelves of one kind of medium, and so on.
Ultimately, though, they all have had flaws and they've all had a
lifespan. My life changes and the Warrens soon collapse like a circus
tent and travel to the next stop.

The site called TEXTFILES.COM caused me to regard not only my
collection but the contributions of others, and the resulting
documentary that I shot about bulletin board systems put me in a lot
of homes with a lot of similar Warrens, and somewhere along that
continuum, I found myself constructing an awareness of the types of
items being collected by me and others, and giving them
classifications. In more and more cases, I started to take on others'
collections as well, which forced me to think about it even harder.

Here's what a few decades of this cobbled together in my mind:

When we end up with our physical and digital piles of material,
there's a couple grand classifications that help parse what we've
got, and for some folks, they need this to process the next steps to
take, especially if they're overwhelmed. And those classifications
are things that are you, things held in trust, and things held in
indifference.

Things that are you tends to be stuff that you've created, be it
writings, photos or saved data that represents projects or memories,
and which is relevant as your trail of effect through your lived
life. E-mails you've written, images you've made, and the inevitable
works we consciously or unconsciously create as people. These items
are not necessarily precious, but they are often rare - you have the
only copy or item, only you maintain it. And in fact, only you may
see any value in it or understand what it is.

Things held in trust are items that may or may not have deep meaning
or relevance to you, but which you acquired from without - the
downloaded programs, or bought books, or a six-foot statue you won an
auction for years ago. You didn't make these things, and there may be
many copies of them, or you again might have the only one: but if
they're not part of your functioning life, then I consider them "held
in trust", as a caretaker keeps maintaining a garden or structure,
towards some future.

Things held in indifference are all those pieces of life that acquire
around a certain personality - ranging from discarded envelopes from
packages you got, to motherboards and loose screws from machines long
gone and pamphlets from trips and travel that you shoved into a
suitcase and then forgot about. Some personality types (say, someone
who remembers that time long ago he had to give up everything to take
what he needed) might impulsively acquire things and then forget
about them almost immediately. These collections can overlap with the
items held in trust or the things that are the person's own
creations, as well.

When I give advice to people on what to do when they wake up one day
and realize they have 2,000 CD-ROMs or piles of magazines they'll
never read, or stacks of VHS tapes of shows they bought 20 years ago
and now will never watch, is help them reach the "end of the story".
People want a story, and they want the story to have a happy ending.
I advise them on how to frame that story:

..but then, after asking a number of message boards and confidantes
about what to do about this multi-gigabyte zip of Wojacks, the
collector uploaded them to a website, finally resting knowing this
long-gathered precious trust of meme juice would survive another
generation. The End.

..but in a shocking twist, it turned out there was a weirdo working
for an Archive of the Internet who wanted these stacks of CD-ROMs and
floppies, and they offered a home which immediately cleared up that
part of the garage, allowing the lawnmower to finally be stored
inside, The End.

...having discovered that there are plenty of National Geographic
issues to go around and there was no need to keep them around, our
hero contacted a local old folks' home and donated them to the
residents' library, where they were happily passed around and enjoyed
for years to come. The End.

[20220213_131251-1024x485]

What is now past a decade working for the Internet Archive has meant
that I'm working in both physical and digital concerns, and each one
has challenges and its own peculiar qualities. Millions of items are
being shipped to large warehouses controlled by Internet Archive, and
millions of files and "items" are being added to the online presence.
In some cases, they live in both places, existing in boxes in pallets
in shipping containers in rooms in a building, and also inside a .zip
file inside an identifier inside a search result on a website.

I've concocted ideas, then, on Archiving.

It's probably as good a time as now to say that I am not universally
beloved as a figure or authority. I am not a professionally trained
archivist, but I've spent my entire life somewhere in the discipline,
and it is not hard, if you seek them out, to find people who consider
my very existence in the field to be a cavalcade of gaps in judgement
by the world and by, perhaps, destiny itself. Why, in a world
overflowing with top-notch expertise by individuals educated by some
of the finest academic programs and concerns, would this street-wise
dandy be considered the one to listen to?

Well, for one thing, I'm fuckin' hilarious. But I also think it's
because I come to a lot of my conclusions and efforts from the point
of view of ad-hoc need and not because I read somewhere that it's
where I should be putting my resources towards. I made a documentary
about bulletin boards because I was concerned many of these people
would die and there'd be no record of them and their perspective; and
I was right. Doing this work put me in touch with lives and people
who had collections that lacked a specific interest by established
institutions, and so I was the one who helped keep them around, or
even take them personally. And when the time came for me to join
forces with Internet Archive, I was already strongly my own thing and
it was a partnership, not a subsumption.

And so from this situation comes a pile of general credos and rules
of thumb I've picked up in my travels:

Where possible, save the original. Where possible, digitize the
original or maintain a digital copy. Ephemera and transient content
is just as important to maintain as products and projects. Digitize
at the highest resolution and fidelity possible, but realize you're
never going to get it perfect and keep the originals around, if you
can. Make digital copies as widely available as possible, all the
time, so it finds its value to people seeking it.

It seems pretty basic stuff, but some of it is hotly contested and
virtual ink spilled by the gallon about process, style and
considerations along the way. It's what works for me, and on the
whole, it's been a succcess.

In this world-view, one of the critical parts of the whole aspect of
"archiving" is making that digital copy of something physical or
analog, using tools and equipment to do so. Naturally, "Born Digital"
items merely need to be kept around and maintained, but items that
are sitting in another medium or container need to make the leap over
the Air Gap into virtual/digital reality and that's where it gets
complicated.

[image-2]

I've been asked, in all manner of ways, what the most difficult part
of the process is - is it tracking down items to work on, or finding
the right order, or devising which video container codec is best for
a ripping of a VHS tape, the DPI of a paper scan, or which equipment
stack is best for the job?

No, none of that.

It's the crushing loneliness.

It's the functional experience of facing down a pile of things that
are in one format, and doing whatever steps are taken, over and over,
to convert them into another form: the loading of the papers into the
feed reader, the stacking of CD-ROMs into a ripping device, the
constant movements of putting tapes into tape players and turning the
capturing software on and off, typing in the filenames with metadata
information as it's done. Doing it endlessly, facing down hundreds
and sometimes thousands of components in a single "collection" with
lots of potential for mistakes, do-overs, unexpected failures, and
all the bumps in the road for what seems like a very straightforward
task.

It's slowly grinding through a backlog of promises and easily-said
agreements to turn This into That, and then finding hours, days, and
weeks of your life drained out of you, resulting in barely enough
data to fill a percentage of a modern hard drive.

The secret-not-so-secret is a lot of this work falls under "it should
be paid for", because it requires just enough mental capacity as to
not be automation-ready, but the minute-to-minute joy of it is
absolutely minimal, repetitive, and only enjoyable in the rear view
mirror looking back at all the stuff you did. The occasional bright
gem of something truly interesting and weird won't make up for the
hundreds of times you'll be getting a necessary but basic item
hoisted into digital, and after enough time, you just wear out.

When I started digitizing VHS tapes en masse, I did a bunch of
research and asking a number of people how they approached the task,
and an interesting theme of conclusions came out: Most were working
with a specific set of items, and most of them burnt out after 20 to
50 tapes. Almost nobody went past that amount, even when they had
many more to do.

My solution, then, was to Stream.

For the years I've been doing the fundamentally boring VHS and
U-Matic tape ripping as part of my projects, I've almost always had a
stream going on Twitch. It's at https://twitch.tv/textfiles and it
has ranged from a non-camera showing of what was being digitized to a
full-on just-short-of-a-televised-show experience, while I move
through piles of cassettes stacked to the ceiling from donated sets.

In this way, I've digitized (at this juncture) over two thousand
videotapes, with many more to come.

[FbNDYwmX0AAJHaG-768x1024]

The initial work was being done out of my actual apartment, which
made sense until it really, really didn't.

The nature of this sort of project is spare parts, awaiting cardboard
boxes, and a mildew smell that starts to hit you when you walk in. At
some phase of life, this is tolerable, but just like separating
things that are you from things held in trust, it's better to have a
dedicated workshop away from a living space.

So, I started renting an office.

It's in one of those facilities where they have dozens of rental
rooms and has a set of group amenities like a kitchen, copy room, and
even meeting rooms. Obviously, it cost more than just stuffing
everything into my home, but the separation has turned out to be
particularly healthy, both in terms of knowing what lying around is
my own stuff, and what is destined for long-term storage after being
digitized.

After consulting with my friend Kyle, I re-imagined the entire
"streaming" approach to be focused on the image, and having it both
look good, and look informative. The result is striking:

[4ce8cf3b90b6ae19-1024x561]

People have asked what the huge monitor is, behind me. I'll answer
that one straight off - it's a cheap LCD TV, purchased for my
apartment and long-since superseded by others but still working
enough to look fine on a camera. The lighting is from two $25 LED
lights designed for the purpose, along with a webcam aimed down the
maw of the U-Matic tape device, since I have to keep the top off
anyway (constant cleaning). The camera recording me is a mirrorless
DSLR (a few hundred dollars) in constant monitor mode, and sending it
all to a HDMI-to-USB Camlink.

Some time in the future, let me go into further detail of The Setup
and the Toolchain, which a certain segment of audience can't get
enough of, and another can't stand a smidgen of.

Instead, let me say that what's obvious, looking in the context of my
full life with this endeavor, is that I've built yet another Warren.

[20220203_171014-1024x485]

Cameras and framing are very deceiving. The room is tiny, barely 70
square feet. The day I toured the facility and was sent the stack of
paper I needed to sign up for a year of long-term residence, they
included a "typical" picture, which is either my office or the one
next to it, and the difference is striking:

[image-4]

Because of the equipment, it runs very hot in there. Because I've got
all the projects going on from so many sources, it's also a bit
noisy. Filters on my studio microphone prevent my audience from
hearing the never-ending humming.

Composing the dull, generic room that I was given into the
cyber-scape of fluorescent dreams that now appears on the Twitch
stream has been a multi-year project. Tapes and other products move
in, get processed, boxed up, shipped out. Streams have been a few
minutes or many hours, depending on what I'm focused on and what time
permits. And because I have a dedicated space, I can be very loud,
very intense, and be able to speak freely on subjects without
worrying I'm ruining anyone's living conditions or sleep. It has
worked spectacularly.

But again, the real purpose of this Warren is to share - to share
with people online (thousands of them, over the years) with what I'm
up to, to have conversations or debates through chat and phones, and
to be able to conduct myself in a way that doesn't feel like a prison
sentence, even if the space I'm functioning in resembles a jail cell
a little too much.

I know this set of decisions and designs is not for everyone. Not
everyone wants to yammer constantly while doing their job to a
shifting, weird audience of onlookers. Not everyone feels they need
multi-colored lights and a massive background video to conduct
themselves, but not everyone is processing thousands of videotapes
all their waking hours, with a dreary consistency that would have
long-ago wrenched all joy and delight from the occasional
discoveries. Even with my motivations to archive and share being
life-long, and my individual cramped spaces being laboratories that I
use to experiment and improve my processes, it turns out that
isolation didn't give me focus - audiences do.

[Fbhbu_tWIAAw_3F]

Here, in the contemporary time of my archiving life, really an
archiving lifetime, is me now trying to turn the promise of endless
stacks of media and materials into digital form, to make them
reachable to the world, before something, and there's more than a few
somethings up to the task, takes me out of the game. It's a life born
of a tragedy, but that tragedy caused perspective, and that
perspective has given me an awareness of how much has been done and
how much is left to do.

From my cramped Warren launches hundreds of recorded moments, and
maybe, with the help of a kind set of eyes, I'll get a lot more of
the work ahead done.

The twitch stream is at https://twitch.tv/textfiles. See you there.

Share this:

  * Share
  * 

  * Email
  * Facebook
  * 
  * Reddit
  * Twitter
  * 
  * 

---------------------------------------------------------------------

Categorised as: Uncategorized

---------------------------------------------------------------------

2 Comments

 1. Archiving in the Time of Streaming -- Pixel Envy says:
    February 5, 2023 at 6:20 am
   
    [...] Archiving in the Time of Streaming [...]

    Reply
 2. [81de04ef] dd_if=devzero_of=devsda says:
    February 5, 2023 at 3:10 pm
   
    And this is for all who want to archive something: For Great Old
    Ones' sake, treat your metadata well! This will return to You!
    There is so much interesting material in the IA I constantly roll
    upon when surfing the links for similar things... A bit later, I
    want to find it again and I cannot because the only metadata is
    like "CD dump A9331".

    I have a library of books about my hobby. A big one, but I have a
    list of them, tags, data, location and my own notes. I keep them
    in some old, GTK2-based freeform database, because currently
    nobody does freeform databases anymore. This saved me so many
    times when I wanted to find stuff.
    If I shoot a scan of such book (I made a book scanner from an old
    wardrobe, Pentium 4 PC and two cameras), the scan lands in the
    special directory which is linked to this database too. I have a
    set of scripts for searching this menagerie to find all what I
    want. If I need to find grayscale photos in all scanned books
    related to microcomputers I have, but published between 1988 and
    1990 by a specific publisher, I do one shell script command, a
    few seconds of my Core 2 Duo thinking and I get a slideshow to
    browse and find what I need. It's bad that I can't do that with
    an analog books - some time ago I was doing a research about
    history of a specific computer which never left prototype stage,
    and I remembered that I have seen a photo of it in a book, I
    remember the publisher and more-or-less years, but this narrows
    down to a few dozens of books, half of which I don't have and
    they are unobtainable from libraries.

    What I'm going to say, the archiving itself is being solved, and
    there are no huge obstacles in it. However, the amount of
    incoming bytes is increasing and the method of navigating among
    them is still more or less the same, and it starts being
    inefficient. It would be a great invention if someone could make
    some good searching method for visual content - images,
    illustrations, videos. This would require some kind of visual
    input, maybe metamodel, maybe totally another input device, some
    kind of "hashing" function, but tolerating all these noises,
    skews, scalings and rotations images have, and then some
    comparing method.

    Reply

Leave a Reply Cancel reply

---------------------------------------------------------------------

    Jason Scott's weblog of computer history, punditry and trivia,
    from the creator of the BBS Documentary, GET LAMP, and proprietor
    of the textfiles.com family of sites. Available to chit-chat at
    jason@textfiles.com.

    Search for: [                    ] [Search]
    [sjshot1]Jason's PresentationsThe BBS DocumentaryThe Archive Team
                                GET LAMP

    Recent Posts

      + Archiving in the Time of Streaming
      + Infinite Inboxes of Infinity
      + Is This Thing On?
      + The Full BBS Documentary Interviews are Going Online
      + The Undiscovered

    Archives

      + February 2023
      + January 2023
      + August 2020
      + January 2018
      + September 2017
      + May 2017
      + March 2017
      + February 2017
      + December 2016
      + November 2016
      + September 2016
      + August 2016
      + June 2016
      + May 2016
      + March 2016
      + February 2016
      + January 2016
      + November 2015
      + September 2015
      + August 2015
      + July 2015
      + May 2015
      + April 2015
      + February 2015
      + January 2015
      + December 2014
      + November 2014
      + October 2014
      + September 2014
      + August 2014
      + July 2014
      + June 2014
      + May 2014
      + April 2014
      + January 2014
      + December 2013
      + November 2013
      + October 2013
      + September 2013
      + August 2013
      + July 2013
      + May 2013
      + April 2013
      + March 2013
      + January 2013
      + December 2012
      + November 2012
      + October 2012
      + September 2012
      + July 2012
      + June 2012
      + May 2012
      + April 2012
      + March 2012
      + January 2012
      + December 2011
      + November 2011
      + October 2011
      + September 2011
      + August 2011
      + July 2011
      + June 2011
      + May 2011
      + April 2011
      + March 2011
      + February 2011
      + January 2011
      + December 2010
      + November 2010
      + October 2010
      + September 2010
      + August 2010
      + July 2010
      + May 2010
      + April 2010
      + February 2010
      + January 2010
      + December 2009
      + November 2009
      + October 2009
      + September 2009
      + August 2009
      + July 2009
      + June 2009
      + May 2009
      + April 2009
      + March 2009
      + February 2009
      + January 2009
      + December 2008
      + November 2008
      + October 2008
      + September 2008
      + August 2008
      + July 2008
      + June 2008
      + May 2008
      + April 2008
      + March 2008
      + February 2008
      + January 2008
      + December 2007
      + November 2007
      + October 2007
      + September 2007
      + August 2007
      + July 2007
      + June 2007
      + May 2007
      + April 2007
      + March 2007
      + February 2007
      + January 2007
      + December 2006
      + November 2006
      + October 2006
      + September 2006
      + August 2006
      + July 2006
      + June 2006
      + May 2006
      + April 2006
      + March 2006
      + February 2006
      + January 2006
      + December 2005
      + November 2005
      + October 2005
      + September 2005
      + August 2005
      + July 2005
      + June 2005
      + May 2005
      + April 2005
      + March 2005
      + February 2005
      + January 2005
      + December 2004
      + November 2004
      + October 2004
      + September 2004
      + August 2004
      + July 2004
      + June 2004
      + May 2004
      + April 2004
      + March 2004
      + February 2004
      + January 2004
      + December 2003
      + November 2003
      + October 2003
      + May 2000

(c) 2023 ASCII by Jason Scott | powered by WordPress
Entries (RSS) and Comments (RSS).

 

  

Loading Comments...
 
Write a Comment... [                    ]
Email (Required) [                    ] Name (Required)
[                    ] Website [                    ]
[Post Comment]