https://thenewstack.io/a-visit-to-the-physical-internet-archive/ TNS OK VOXPOP DORA for Dev Productivity Are DORA metrics effectively used at your organization to measure developer team productivity? Yes, DORA is used effectively. 0% DORA is used but it is at best only "somewhat effective." 0% DORA gets talked about but it isn't actually used. 0% I don't know about DORA, is she an explorer? 0% Thanks for your opinion! Subscribe below to get the final results, published exclusively in our TNS Update newsletter: [ ] SUBMIT TNS DAILY We've launched a new daily email newsletter! You can now receive a free roundup of the most recent TNS articles in your inbox each day. Register now, never miss a story, always stay in-the-know. [ ] SUBSCRIBE [ ] Search More Results ARCHITECTURE Cloud Native Ecosystem Containers Edge Computing Microservices Networking Serverless Storage ENGINEERING AI Frontend Development Software Development TypeScript WebAssembly Cloud Services Data Security OPERATIONS Platform Engineering Operations CI/CD Tech Life DevOps Kubernetes Observability Service Mesh CHANNELS Podcasts Ebooks Events Newsletter TNS RSS Feeds THE NEW STACK About / Contact Sponsors Sponsorship Contributions PODCASTS EBOOKS EVENTS NEWSLETTER ARCHITECTURE ENGINEERING OPERATIONS Cloud Native Ecosystem Containers Edge Computing Microservices Networking Serverless Storage Oxide Launches the World's First Commercial Cloud Computer Oct 26th 2023 3:10am, by David Cassel Zombie Resources Eat up Your Cloud Budget Oct 20th 2023 7:01am, by Aditya Gune True Portability Is the Killer Use Case for WebAssembly Oct 19th 2023 8:30am, by Liam Crilly The Architect's Guide to the Modern Data Stack Oct 11th 2023 11:07am, by Brenna Buuck 3 Ways Observability Matters in Cloud Native Testing Oct 11th 2023 7:52am, by Ken Hamric Why Capistrano Got Usurped by Docker and Then Kubernetes Oct 25th 2023 1:53pm, by David Eastman The Moby Project Post-Kubernetes: 3 New Releases in 2023 Oct 24th 2023 8:41am, by Loraine Lawson K8s Is Not the Platform - Or Is It and We All Misunderstood? Oct 20th 2023 6:36am, by Stefan Schimanski Docker at 10 -- 3 Things We Got Right, 3 Things We Got Wrong Oct 16th 2023 7:52am, by Sam Alba Docker Launches GenAI Stack and AI Assistant at DockerCon Oct 10th 2023 12:25pm, by Loraine Lawson Edge AI: How to Make the Magic Happen with Kubernetes Sep 14th 2023 4:00am, by Saad Malik Cloud to Edge AI with a Mobile Database Platform Aug 28th 2023 8:42am, by Mark Gamble Dev News: React Still King, Vercel AI Tools, Netlify Connect Jun 17th 2023 4:00am, by Loraine Lawson Creating an IoT Data Pipeline Using InfluxDB and AWS Jun 5th 2023 10:25am, by Jason Myers Dell Intros New Edge, Generative AI, Cloud, Zero Trust Prods May 31st 2023 11:00am, by Chris J. Preimesberger Easier, Faster, Better: Real-Time Microservices Oct 24th 2023 8:56am, by Angel Camacho Scaling Environments with OpenTelemetry and Service Mesh Oct 17th 2023 11:13am, by Anirudh Ramanathan A Showdown Between Express.js and Fastify Web App Frameworks Oct 11th 2023 9:57am, by Loraine Lawson 5 Best Practices for Securing Your API Gateway Sep 29th 2023 11:51am, by David Sudia The Basics of Event-Driven Architectures Sep 22nd 2023 7:45am, by Jim Moffitt Cilium CNCF Graduation Could Mean Better Observability, Security with eBPF Oct 13th 2023 4:00am, by B. Cameron Gain Performant and Programmable Telco Networking with eBPF Aug 11th 2023 10:00am, by Bill Mulligan Create a Samba Share and Use from in a Docker Container Jul 29th 2023 6:00am, by Jack Wallen CIOs, Heed On-Premises App and Infrastructure Performance Jul 5th 2023 1:21pm, by Gregg Ostrowski Hasura Launches New Data Network for APIs Only Jun 29th 2023 9:36am, by Chris J. Preimesberger Netlify Launches Composable Web Platform for Enterprise Devs Oct 19th 2023 9:00am, by Richard MacManus 4 Factors of a WebAssembly Native World Oct 12th 2023 9:00am, by Liam Crilly Raising the Serverless Bar: Infrastructure APIs Unleash More Value for Enterprises Oct 10th 2023 10:00am, by Tyson Trautmann The Security-First Mindset to Unlocking the AWS Opportunity Aug 9th 2023 8:14am, by David Melamed 3 Reasons Why Teams Move Away from AWS Lambda Jul 18th 2023 10:00am, by Jonathan Michaux Optimize Your Observability Spending in 5 Steps Oct 6th 2023 11:00am, by Joshua Scott How Goldsky Democratizes Streaming Data for Web3 Developers Oct 2nd 2023 8:46am, by Yaroslav Tkachenko NoSQL Data Modeling Mistakes that Ruin Performance Sep 27th 2023 9:00am, by Felipe Cardeneti Mendes Address High Scale Google Drive Data Exposure with Bulk Remediation Sep 26th 2023 10:00am, by Adam Gavish Battling the Steep Price of Storage for Real-Time Analytics Sep 22nd 2023 6:00am, by B. Cameron Gain AI Frontend Development Software Development TypeScript WebAssembly Cloud Services Data Security How Retool AI Differs from LangChain (Hint: It's Automation) Oct 25th 2023 5:00am, by Richard MacManus Machine Learning for Real-Time Data Analysis: Training Models in Production Oct 24th 2023 6:00am, by Jelani Harper The GenAI Data Developer Experience: Performance Optimization Oct 23rd 2023 8:00am, by Scott M. Fulton III Dev News: Angular's New Loading, an AI Compiler, and JS Runs Wasm Oct 21st 2023 4:00am, by Loraine Lawson The AI Evolution Will Happen Faster Than Computing Evolution Oct 20th 2023 12:58pm, by Alex Williams How Retool AI Differs from LangChain (Hint: It's Automation) Oct 25th 2023 5:00am, by Richard MacManus How to Use Astro with a Sprinkling of React Oct 24th 2023 7:00am, by Paul Scanlon Dev News: Angular's New Loading, an AI Compiler, and JS Runs Wasm Oct 21st 2023 4:00am, by Loraine Lawson Netlify Launches Composable Web Platform for Enterprise Devs Oct 19th 2023 9:00am, by Richard MacManus More Ways to Slim Down 'JavaScript Heavy' Web Development Oct 19th 2023 5:00am, by Loraine Lawson Measure Developer Joy, Not Productivity, Says Atlassian Lead Oct 26th 2023 3:00am, by Jennifer Riggins Going Real Time in AdTech: A Batch-to-Streaming Journey Oct 25th 2023 12:39pm, by Abhishek Jain PHPFusion Vulnerabilities: Reminder Why You Need CMS Security Oct 24th 2023 12:26pm, by Matthew Hogg Despite the Hype, Engineers Not Impressed with DORA Metrics Oct 24th 2023 11:21am, by Lawrence E Hecht Environment Replication Doesn't Work for Microservices Oct 24th 2023 11:00am, by Nocnica Mellifera Dev News: Udemy's New Docker Program, Plus TypeScript Beta Oct 7th 2023 5:01am, by Loraine Lawson The Angular Renaissance: Why Frontend Devs Should Revisit It Sep 26th 2023 8:15am, by Loraine Lawson Dev News: A 'Nue' Frontend Dev Tool; Panda and Bun Updates Sep 16th 2023 4:00am, by Loraine Lawson Dev News: Svelte 5 vs. VanillaJS and Google's Project IDX Aug 12th 2023 8:00am, by Loraine Lawson Dev News: Dart Frog Now Stable, Redwood Updates Bundler Aug 4th 2023 7:00am, by Loraine Lawson Why WebAssembly Will Disrupt the Operating System Oct 23rd 2023 11:23am, by Charles Humble True Portability Is the Killer Use Case for WebAssembly Oct 19th 2023 8:30am, by Liam Crilly Rust Is Surging Ahead in WebAssembly (for Now) Oct 19th 2023 8:02am, by B. Cameron Gain 4 Factors of a WebAssembly Native World Oct 12th 2023 9:00am, by Liam Crilly WasmCon 2023: A Conversation about the Future of WebAssembly Oct 6th 2023 6:31am, by Jenn Gile and Matthew Yacobucci Why the Cloud Makes Forecasts Difficult and How FinOps Helps Oct 26th 2023 11:46am, by Heather Joslyn Netlify Launches Composable Web Platform for Enterprise Devs Oct 19th 2023 9:00am, by Richard MacManus Primer: Cloud Development Environments, or CDEs Oct 19th 2023 7:23am, by Alexander T. Williams 3 Signs It's Time to Move off the Cloud Oct 19th 2023 6:37am, by Steve Martinelli How to Get Your Organization Started with FinOps Oct 18th 2023 8:46am, by Heather Joslyn Combining the Power of Text-Based Keyword and Vector Search Oct 26th 2023 6:10am, by Megan Grant Going Real Time in AdTech: A Batch-to-Streaming Journey Oct 25th 2023 12:39pm, by Abhishek Jain Easier, Faster, Better: Real-Time Microservices Oct 24th 2023 8:56am, by Angel Camacho Machine Learning for Real-Time Data Analysis: Training Models in Production Oct 24th 2023 6:00am, by Jelani Harper Build Your Own Chat App with Quix and Redpanda -- Part 2 Oct 23rd 2023 9:56am, by Tomas Neubauer Can eBPF Keep Your Kubernetes Cluster from Getting Hacked? Oct 25th 2023 10:00am, by B. Cameron Gain eBPF Offers a New Way to Secure Cloud Native Systems Oct 25th 2023 8:40am, by B. Cameron Gain Securing CI/CD Pipelines: A Comprehensive Approach Is Vital Oct 25th 2023 6:06am, by Noah Simon PHPFusion Vulnerabilities: Reminder Why You Need CMS Security Oct 24th 2023 12:26pm, by Matthew Hogg Don't Shift Security Left; Integrate It Everywhere Oct 24th 2023 6:53am, by James Sanders Platform Engineering Operations CI/CD Tech Life DevOps Kubernetes Observability Service Mesh Measure Developer Joy, Not Productivity, Says Atlassian Lead Oct 26th 2023 3:00am, by Jennifer Riggins Q&A: Patrick Debois on the Past, Present and Future of DevOps Oct 24th 2023 10:07am, by Jennifer Riggins Measuring Key KPIs and Platform Engineering Success Oct 23rd 2023 7:02am, by Pravanjan Choudhury K8s Is Not the Platform - Or Is It and We All Misunderstood? Oct 20th 2023 6:36am, by Stefan Schimanski 9 Steps to Platform Engineering Hell Oct 18th 2023 11:12am, by Luca Galante Open Source in Numbers: The Terraform License Change Impact on Contribution Oct 26th 2023 11:34am, by Eyar Zilberman Oxide Launches the World's First Commercial Cloud Computer Oct 26th 2023 3:10am, by David Cassel Going Real Time in AdTech: A Batch-to-Streaming Journey Oct 25th 2023 12:39pm, by Abhishek Jain Stop Setting CPU and Memory Requests in Kubernetes Oct 25th 2023 12:00pm, by Reid Vandewiele OpsCanvas: Infrastructure as Code for Smaller Shops Oct 25th 2023 4:00am, by Susan Hall Securing CI/CD Pipelines: A Comprehensive Approach Is Vital Oct 25th 2023 6:06am, by Noah Simon Continuous Release: Move Faster without Breaking Things Oct 19th 2023 10:55am, by Karishma Irani No Workflow Is an Island Oct 13th 2023 7:09am, by Mark Fussell Is a Career as a DevSecOps Engineer in Your Future? Oct 12th 2023 12:03pm, by John Ross Continuous Integration Is Just a Script Runner Oct 12th 2023 7:43am, by Emanuele Libralato Startup Jungle: The Art of Networking at Tech Conferences Oct 26th 2023 8:47am, by James Leaverton A Visit to the Physical Internet Archive Oct 26th 2023 7:19am, by Richard MacManus How to Be a Better Ally in Open Source Communities Oct 25th 2023 10:25am, by Jennifer Riggins Why Women Are Underrepresented in Open Source Oct 25th 2023 6:57am, by Barbano Gonzalez Crack a 1999 NSA Cryptography Standard and Win a Bounty Oct 22nd 2023 6:00am, by David Cassel Combining the Power of Text-Based Keyword and Vector Search Oct 26th 2023 6:10am, by Megan Grant Why Capistrano Got Usurped by Docker and Then Kubernetes Oct 25th 2023 1:53pm, by David Eastman Stop Setting CPU and Memory Requests in Kubernetes Oct 25th 2023 12:00pm, by Reid Vandewiele Securing CI/CD Pipelines: A Comprehensive Approach Is Vital Oct 25th 2023 6:06am, by Noah Simon OpsCanvas: Infrastructure as Code for Smaller Shops Oct 25th 2023 4:00am, by Susan Hall Why Capistrano Got Usurped by Docker and Then Kubernetes Oct 25th 2023 1:53pm, by David Eastman Stop Setting CPU and Memory Requests in Kubernetes Oct 25th 2023 12:00pm, by Reid Vandewiele Can eBPF Keep Your Kubernetes Cluster from Getting Hacked? Oct 25th 2023 10:00am, by B. Cameron Gain Easier, Faster, Better: Real-Time Microservices Oct 24th 2023 8:56am, by Angel Camacho The Moby Project Post-Kubernetes: 3 New Releases in 2023 Oct 24th 2023 8:41am, by Loraine Lawson What Is OpenTelemetry? Oct 17th 2023 6:28am, by Jess Lulka Cilium CNCF Graduation Could Mean Better Observability, Security with eBPF Oct 13th 2023 4:00am, by B. Cameron Gain 3 Ways Observability Matters in Cloud Native Testing Oct 11th 2023 7:52am, by Ken Hamric Cloud Native Observability: Fighting Rising Costs, Incidents Oct 11th 2023 7:18am, by Heather Joslyn How TNS Migrated to Google Analytics 4 Oct 10th 2023 8:28am, by Steve Albright Scaling Environments with OpenTelemetry and Service Mesh Oct 17th 2023 11:13am, by Anirudh Ramanathan Kubernetes 1.28 Accommodates the Service Mesh, Sudden Outages Aug 18th 2023 10:08am, by Joab Jackson Don't Force Containers and Disrupt Workflows May 25th 2023 3:10pm, by Alex Williams Linkerd Service Mesh Update Addresses More Demanding User Base Apr 11th 2023 6:17am, by Joab Jackson How to Create Zero Trust Architecture for Service Mesh Mar 27th 2023 7:00am, by Joe Fay 2023-10-26 07:19:17 A Visit to the Physical Internet Archive Tech Life A Visit to the Physical Internet Archive In a behind-the-scenes tour of the Internet Archive, founder Brewster Kahle explained how it preserves both digital and physical media. Oct 26th, 2023 7:19am by Richard MacManus Featued image for: A Visit to the Physical Internet Archive All photos by Richard MacManus 10/23/2023: DORA for Dev Productivity Try our new 5 second poll. It's fast. And it's fun! DORA for Dev Productivity Are DORA metrics effectively used at your organization to measure developer team productivity? Yes, DORA is used effectively. DORA is used but it is at best only "somewhat effective." DORA gets talked about but it isn't actually used. I don't know about DORA, is she an explorer? I HAVE AN OPINION We'd love to hear what you think. While I was in San Francisco for the AI Engineer Summit earlier this month, I took the opportunity to visit the Internet Archive -- the actual physical archive in the California town of Richmond, about twenty minutes drive from San Francisco. I'd bought a ticket to "go behind-the-scenes at the physical archive" on Wednesday, Oct. 11, and I arrived just before the start time of 6 p.m. I was glad I hadn't arrived any earlier, since the location of the physical archive was (unsurprisingly) a warehouse in an industrial part of Richmond. There didn't seem to be anything else to do in the area. I had instructed the Uber driver to drop me off at a car park with an Internet Archive sign. But as I looked around, I couldn't see a public entrance to the warehouse. There were a few other confused-looking internet history nerds standing around, so we awkwardly introduced ourselves and discussed whether we were in the right place. Eventually, a couple of people at the end of the street, about 200 yards away, spotted us and waved us over. Internet Archive physical archive, Richmond, CA. Internet Archive physical archive, Richmond, CA. It turned out a group of people had already made themselves comfortable inside the main building, drinking complimentary cokes, beers or mineral water, and eating finger food. The crowd was a mix of older people (perhaps from the generation that worked in Silicon Valley during the 1960s and 70s) and younger geeks (my guess is that many were either librarians or professional webheads -- me being an example of the latter). When the tour began about half an hour later, thirty or forty people gathered in front of an enthusiastic red-shirted man with thinning gray hair. He was of course the founder of the Internet Archive, Brewster Kahle. At first, I was surprised he would be conducting the tour himself, but it soon became clear that Kahle lives and breathes the mission of the Internet Archive. He began by showing us the shipping containers full of old books and other materials, while reeling off some facts ("the Internet Archive is a nonprofit library; we started it 27 years ago, 1996."). Brewster Kahle in front of containers Brewster Kahle in front of containers (actual, physical ones). Later in the tour, Kahle eagerly demonstrated the book-scanning machine, pointed out stacks of boxes gifted to the archive (full of books, videos, disks, records, cassettes, and other media), and stood to the side proudly while his film archivists told us how they convert vintage home videos into high-res digital files. It was a fascinating look into the day-to-day operations of the Internet Archive, which is staffed by a number of friendly and probably liberal-minded Californians -- including Brewster's son, Caslon. What the Internet Archive Stores The Internet Archive is perhaps most well known for its Wayback Machine, which debuted in 2001 and has been archiving web pages since 1996. "We collect about a billion URLs every day, just kind of an astonishingly large number," said Kahle during his tour. "And it's now two and a half trillion URLs in the Wayback Machine collection -- these old web pages. And it's queried about six or seven thousand times a second." But the physical archive, as its informal name suggests, is a repository of physical media -- books, catalogs, old computer disks, film, records and cassette tapes, and much more. When a new piece of media comes in, the Internet Archive staff first decide whether it's a duplicate of something they already have -- a process they call "deduping." If it is a dupe, it's discarded or given away. If not, it is digitized and then the physical item is stored. (As an aside, the Internet Archive says it only makes available digital copies of a book if it owns the physical copy.) IA film scanners A specially made vintage film scanner at the Internet Archive. "We've been digitizing books now since the early 2000s," said Kahle, "and we ended up building our own book scanners." He added that IA digitizes "about a million books a year" and they've digitized in the order of 7 or 8 million books in total (on its about page, the IA says it has "41 million books and texts", so the majority of those must be text items other than books). As for music, it's a media type that has historically had multiple formats -- LPs, CDs, cassettes, MP3, etc. Kahle was particularly enthusiastic about 78 RPM records, which he said were around from about 1900 to 1950. "There are maybe 2 or 3 million of them," he said, "[and] we've digitized about 450,000." Boxes of media items Boxes of media items, overseen by a cardboard cutout of Darth Vader. "We're trying to basically do all the media types," continued Kahle. "And what I've been finding is that the time that [...] things have been becoming obsolete, it's happening faster and faster. [...] Not only do you not have access to the same things; even if you have access, it's not presented to you in such a way that you actually use it." Note: If you're interested in donating items to the Internet Archive, check this web page for a list of media types it is currently accepting. How the Internet Archive Keeps Going Someone in the tour group asked Kahle how often the IA needs to buy new servers, to store this constant influx of new media. "Continuously," he replied. "We buy a new rack pair -- because it always comes in a pair -- every two months [or] three months. [...] In one rack, you can put around five petabytes now." Internet Archive server racks Two previous generations of Internet Archive storage machines; on the left, the StorageTek 9710 from the 1990s, and on the right the first generation PetaBox (2004). Of course, the IA has been in the news this year because of legal attacks from both the book publishing industry and the music industry (the latter regarding the 78 RPM records project). Kahle made several sniping comments about these legal challenges during the tour, but it was clear it had taken a toll on the IA. "That's still going through the courts," he sighed, regarding the book publishers' lawsuit, "and it's incredibly expensive." So how does the IA survive? Kahle said that the IA runs mainly on donations, from 110,000 individuals averaging about $5 per person, as well as "foundations giving us serious amounts of money." The IA also offers subscription services to libraries and other organizations. "We also survive by, well, not spending a lot," he added. "I mean, you notice the servers don't have any air conditioning, right? If it gets hot, we just open the windows. So, it's green. But it's also inexpensive." streets of Richmond Outside the physical Internet Archive in Richmond, CA. A fun night out for an internet history nerd! Created with Sketch. [1c7152c0-r] Richard MacManus is a Senior Editor at The New Stack and writes about web and application development trends. Previously he founded ReadWriteWeb in 2003 and built it into one of the world's most influential technology news sites. From the early... Read more from Richard MacManus SHARE THIS STORY TRENDING STORIES SHARE THIS STORY TRENDING STORIES THE NEW STACK UPDATE A newsletter digest of the week's most important stories & analyses. [ ] SUBSCRIBE The New stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy. ARCHITECTURE Cloud Native Ecosystem Containers Edge Computing Microservices Networking Serverless Storage ENGINEERING AI Frontend Development Software Development TypeScript WebAssembly Cloud Services Data Security OPERATIONS Platform Engineering Operations CI/CD Tech Life DevOps Kubernetes Observability Service Mesh CHANNELS Podcasts Ebooks Events Newsletter TNS RSS Feeds THE NEW STACK About / Contact Sponsors Sponsorship Contributions FOLLOW TNS roadmap.sh Community created roadmaps, articles, resources and journeys for developers to help you choose your path and grow in your career. Frontend Developer Roadmap Backend Developer Roadmap Devops Roadmap (c) The New Stack 2023 Disclosures Terms of Use Privacy Policy Cookie Policy FOLLOW TNS TNS DAILY SUBSCRIBE * *