https://www.os2museum.com/wp/the-secret-history-of-atapi/ OS/2 Museum OS/2, vintage PC computing, and random musings [os2floppy] Skip to content * Home * About + Wanted List * OS/2 History + OS/2 Beginnings + OS/2 1.0 + OS/2 1.1 + OS/2 1.2 and 1.3 + OS/2 16-bit Server + OS/2 2.0 + OS/2 2.1 and 2.11 + OS/2 Warp + OS/2 Warp, PowerPC Edition + OS/2 Warp 4 + OS/2 Timeline + OS/2 Library o OS/2 1.x SDK o OS/2 1.x Programming o OS/2 2.0 Technical Library + OS/2 Videos, 1987 * DOS History + DOS Beginnings + DOS 1.0 and 1.1 + DOS 2.0 and 2.1 + DOS 3.0, 3.1, and 3.2 + DOS 3.3 + DOS 4.0 + DOS Library * NetWare History + NetWare Timeline + NetWare Library * Windows History + Windows Library * PC UNIX History + Solaris 2.1 for x86 - Looking for High Sierra The Secret History of ATAPI Posted on November 24, 2021 by Michal Necasek The other day I asked myself a seemingly trivial question: What was the first ATAPI CD-ROM drive and when was it available? Given that ATAPI was a major technology which instantly obsoleted all proprietary CD-ROM interfaces and made SCSI much less desirable, one might expect that there would have been some press releases touting the advantages of the new technology, articles describing the whys and wherefores, but... nope. There is nothing. [yH5BAEAAAAALAAAAAABAAEAAAIBRAA7][PB180025-640x480]A rather old IDE CD-ROM drive with curious jumper settings In 1993, CD-ROM drives used either SCSI or one of several proprietary interfaces, the major amongst those being Matsushita/Panasonic, Mitsumi, Philips, and Sony. In 1995, the proprietary interfaces were effectively gone and most new CD-ROM drives used the ATAPI interface. Something clearly happened in 1994, but exactly what, when, and how--that's something of a mystery. There are reports that Philips showed their CDD-300 CD-ROM drive with "new IDE interface" (e.g. ATAPI) at the CeBIT '94 expo in March 1994. When the drive started shipping is unclear, but it was certainly in 1994. Further digging showed that NEC was another early vendor of ATAPI CD-ROM drives. Establishing the timeline is again very difficult, but in June 1994 there was a Linux kernel patch to support the NEC CDR-260. Its author seems to have been completely unaware of the existence of ATAPI, but that's perhaps not surprising since the ATAPI 1.2 specification was only finalized on June 10, 1994, although a draft was available in February of that year. Too Old to Work? There is a possibly misleading Microsoft KB article (KB131499) claiming that the "NEC CDR-38, NEC CDR-250, CDR-260, CDR-260R, and CDR-260GW CD-ROM drives are incompatible with the ATAPI specification version 1.2". Another Microsoft KB article (Q121547) brings more detail: The NEC CDR-260 drive "follows the ATAPI A1.8 specification, which is a pre-release specification that has not been ratified by the ATAPI council". The article further adds: "NEC also manufactures the CDR260R CD-ROM drive, which does conform to ATAPI 1.2 specification and works with Windows NT version 3.5. Starting in November 1994, Gateway 2000 began shipping the CDR260R rather than the CDR260 in its computers." Note that the two articles contradict each other: KB131499 claims that the NEC CDR-260R is not compatible with ATAPI 1.2, whereas Q121547 explicitly states that it is. The NEC CDR-260 was apparently an OEM drive and no real documentation exists. What is interesting is that photos of an NEC CDR-260 made in July 1994 show a jumper selecting between "8-bit bus" and "16-bit bus". ATAPI is inherently 16-bit just like ATA... then again there are specialized 8-bit ATA variants. Now, if there were CDR-260 drives built the first half of 1994, that would explain why those drives weren't compatible with the ATAPI 1.2 specification which was officially published in June 1994. It's also no wonder that Windows NT wasn't the only operating system that didn't like those drives. NeXTSTEP 3.3 release notes explicitly list the NEC CDR-260 and CDR-273 as unsupported. Linux source code offers some clues too. The NEC CDR-260 is clearly mostly a standard ATAPI drive but several relatively minor differences prevent it from working with normal drivers. Examining NEC_IDE.SYS Another indirect source of information is the NEC_IDE.SYS device driver for DOS (required for MSCDEX). The OS/2 Museum has an NEC_IDE.SYS file dated 2-18-94, size 21,669 bytes. This driver does not work with modern ATAPI drives, but shows something interesting: [yH5BAEAAAAALAAAAAABAAEAAAIBRAA7][atapi-necide-feb94-640x356] NEC_IDE.SYS from February 1994 Rev A1.7 quite likely corresponds to a pre-release ATAPI specification, but notice how there is absolutely no mention of NEC, only Oak Technology OTI-011. Analyzing the driver shows that it cannot work because it issues the ATAPI Soft Reset command (code 08h) and expects that when the command completes, the drive will set the DRDY and DSC bits in the ATA status register, just like an ATA hard disk would. The ATAPI 1.2 or even ATAPI 2.6 specifications are not 100% clear on this point, but suggest that the bits should probably not be set. ATA /ATAPI-6 is unambiguous and says the DRDY and DSC bits will be clear, just like after ATAPI device power-up, in order to "hide" ATAPI devices from old ATAPI-unaware drivers for ATA hard disks. A newer version of NEC_IDE.SYS from April 1995 works: [yH5BAEAAAAALAAAAAABAAEAAAIBRAA7][atapi-necide-apr95-640x356]Updated NEC_IDE.SYS compliant with ATAPI 1.2 It's notable that the newer driver handles the standard ATAPI behavior (status register bits clear after ATAPI Soft Reset) as well as the old NEC drive behavior (DRDY and DSC set). It is highly likely that the driver was designed to handle both old pre-standard drives and newer ATAPI 1.2 compliant CD-ROM drives. But again... why does a more or less standard ATAPI driver for an NEC CD-ROM drive only mention Oak Technology OTI-011 and nothing else... Oak OTI-011 So what exactly was the Oak Technology OTI-011 chip? There is not much information available. Unusually, in place of datasheets, bitsavers only has several patents related to the OTI-011 chip. That is not great, but does show fairly clearly what OTI-011 did. [yH5BAEAAAAALAAAAAABAAEAAAIBRAA7][PB180016-640x481]Oak Technology OTI-011 chip made in January 1994 Note that in the late 1980s, Oak Technology developed OTI-012, a CD-ROM controller chip with no IDE/ATA support. The OTI-011 is a successor chip (no, I don't know why the Oak numbering went backwards) developed in 1992-1993 which combined the functionality of OTI-012 with a built-in ATA interface, suitable for implementing an ATAPI CD-ROM. The OTI-011 required an external microcontroller and DRAM and was expected to be connected to a DSP which delivered the raw bits from a CD-ROM. The OTI-011 decoded sub-channel data and raw sector data, and was capable of performing the additional error correction and detection needed for CD-ROMs. The host interface of OTI-011 implemented ATA task file registers in hardware and set up ATA/ATAPI command processing for the attached microcontroller. The OTI-011 chip itself implements very little of the ATA protocol itself; the external microcontroller's firmware does the hard work. The patents describe what the OTI-011 chip did in terrible mangled lingo, but don't provide any background information. Normally there would be no public information about the development of a chip... but in this case there is, thanks to lawsuits that Oak Technology (later Zoran) initiated in the late 1990s and early 2000s. There is an U.S. International Trade Commission document which supplies a surprising amount of historical information, and also establishes a fairly detailed timeline, since that was required for establishing the validity of US Patent 5,581,715 (aka '715 patent). We learn that Oak Technology achieved "considerable commercial success" with the OTI-011 chip because it solved an important problem (attaching a CD-ROM drive to the ATA bus) before others. Oak Technology, namely Phil Verinsky, conceived of the OTI-011 chip "no later than April 21, 1993" while "the date of the ATAPI specification is June 10, 1993. There is no evidence that the ATAPI specification or any earlier version of it was communicated to Oak before the April 21, 1993, conception date." Note that the official ATAPI 1.2 specification is dated June 10, 1994, but the above refers to Western Digital's "ATAPI CD-ROM Standard: Proposal for Working Draft" document from June 10, 1993. We learn that in early 1993, Mitsumi worked with Oak Technology on a CD-ROM drive attached to the ATA bus but not utilizing ATAPI. It appears that this project didn't really go anywhere. In April 1993, there was a sales agreement between NEC and Oak concerning the not-yet-completed OTI-011 chip. It is almost certain that NEC was Oak's first customer and likely the first user of the OTI-011 chip. This fits with other known facts, namely that NEC shipped ATAPI CD-ROMs before the ATAPI 1.2 standard was finalized. The OTI-011 chip taped out on July 22, 1993, was quickly corrected and taped out again on July 29, 1993. Given that information alone, it's near certain that OTI-011 chips were made in 1993, but it's much less clear if any CD-ROM drives based on those chips were shipped in 1993. We also learn the versions and dates of several unpublished early ATAPI specification drafts: * Revision A1.4, dated Aug. 17, 1993, 141 pages * Revision A1.5, dated Aug. 28, 1993, 158 pages * Revision A1.6, dated Sep. 10, 1993, 174 pages * Revision A1.7, dated Oct. 18, 1993, 176 pages * Revision A1.7B, dated Nov. 2, 1993 Note that revision 1.7B was almost certainly the last one before SFF took over the ATAPI project. Under the SFF heading, ATAPI 1.0 and 1.1 drafts were made available but very quickly replaced. Revision 1.2 was the first widely used ATAPI standard. NEC CDR-260 The OS/2 Museum was able to obtain a functioning NEC CDR-260 drive with a February 1994 manufacturing sticker (February 1994 was when the draft of ATAPI 1.2 was made available). Sure enough, the drive is... interesting. That starts with the drive jumpers which according to the label provide an option to switch the IDE interface into an 8-bit mode (see photo at beginning of article). What good that might be... nobody knows, since there is no documentation. [yH5BAEAAAAALAAAAAABAAEAAAIBRAA7][PB180026-640x480]NEC CDR-260 ATAPI drive made in Feb '94 Looking inside the drive, one very unsurprisingly finds an Oak Technology OTI-011 chip (lower right corner): [yH5BAEAAAAALAAAAAABAAEAAAIBRAA7][PB180013-640x480]NEC CDR-260 electronics Linux detects the CDR-260 drive, but is a bit confused about the drive's name. It turns out that the model name is not byte swapped in the data block returned by IDENTIFY PACKET DEVICE, but Linux expects that it will be, so it ends up getting scrambled. This is the exact same problem that certain old IDE drives had. Unsurprisingly, the problem has been noticed before. It is worth mentioning that the ATAPI specification makes no suggestion that the ASCII strings embedded in the IDENTIFY PACKET DEVICE response should be byte-swapped, but "everyone" expects them to be. What's much worse is that after inserting a CD-ROM, the drive "falls off the bus" with many errors and many reset attempts. Linux is effectively unable to read anything. But the drive is not dead. In DOS, the NEC_IDE.SYS driver (dated 08/ 11/1994, size 24,824 bytes) works just fine. The drive can read CD-ROMs without trouble. Interestingly, the previously mentioned older NEC_IDE.SYS driver (dated 02/18/1994, size 21,669 bytes) appears to find the drive but then says that it's not ready. The working NEC_IDE.SYS driver appears to detect this particular model, or rather detects NEC (by definition ATAPI) drives with firmware version lower than two; the drive in question reports firmware revision 1.01. Such old NEC drives are then treated specially by the NEC_IDE.SYS driver. A survey of various operating systems confirms that yes, the NEC CDR-260 is unusual. As mentioned above, Linux has workarounds for it, but clearly not sufficient. Solaris mentions the CDR-260 explicitly, and special-cases the CDR-260 and a few other drives to only transfer one sector at a time. The ATAPI driver in OS/2 has by far the most complete support for these drives. The author clearly had the elusive ATAPI revision 1.7B specification in hand. The OS/2 driver translates a number of SCSI commands and converts the SCSI mode page information returned by the drive from the ATAPI 1.7B format to the published ATAPI 1.2 specification. There's also the following comment: The 1.7 spec indicates that the completion status only relies on the DRQ bit being 0. If we are in 1.7B compatibility mode and the DRQ bit is 0, set the reason variable to IR_COMPLETE. I suspect that might be the why Linux doesn't work with this drive and why Solaris only reads one sector at a time. From reading the ATAPI mailing list, it is very clear that as late as December 1993, some important low-level details of ATAPI were still in flux. If the NEC CDR-260 drives really conform to the pre-SFF revision 1.7B of the ATAPI specification, it's not surprising that software written for ATAPI 1.2 and later has trouble with them. The NEC CDR-260 drives were also found in NEC's own PC-98 machines. This blog post (Korean) shows a drive with a "CDR-260(PI)" designation on a Japanese label and much the same electronics as the one pictured above. The OTI-011 chip visible in the blog photos was made in week 46 of 1993; there is a high likelihood that NEC shipped the CDR-260 drives before the end of 1993, though perhaps only in Japan. Other Early Entrants As mentioned above, Philips/LMS was another early supplier of ATAPI drives. Examining DDATAPI.SYS driver from August 1994 that shipped with CM-207 drives (and likely others), it is apparent that the driver is very unlikely to work with generic ATAPI CD-ROMs. The DDATAPI.SYS driver expects that the Sector Number register (typically at I/O port 1F3h or 173h) changes its value in a certain way. Yet the ATAPI specification leaves said register more or less completely unused and does not define any behavior for it. That is a hint Philips started implementing their ATAPI drives well before the ATAPI specification was finalized. It is possible that the Sector Number register has some vendor specific meaning on Philips drives, but it's also possible that the register had some defined function in unpublished drafts of the ATAPI specification. Note that Philips did also produce integrated ATAPI CD-ROM controllers (e.g. SAA7380), but there is no evidence that such chips existed before 1996. Cirrus Logic was, together with Oak Technology, an early ATAPI chip vendor. The CL-CR3400 may have been available in 1994, but unlike the OAK OTI-011, it was only an ATAPI interface chip, not an integrated CD-ROM controller. Only the follow-up chip, the Cirrus Logic CL-CR3410 (1995) integrated both the ATAPI interface and a CD-ROM decoder. Mitsumi was another early vendor of ATAPI CD-ROM drives with the FX001DE drive. So was Sony with CDU-55E and CDU-55D drives. If this document is to be believed, all those drives were available in 1994 and did not fully conform to ATAPI 1.2. Secret Society It's not a coincidence that even industry insiders called ATAPI a "secret society". According to Dal Allan, ATAPI "went public" in October 1993, and became an SFF project (before moving to X3T13 in 1996). But work on ATAPI probably started sometime in 1992 when, again according to Dal Allan, It was Compaq who told WD "You and the CD-ROM guys get together and give us a CD attached to the second ATA connector." ATAPI became sort of tied into Western Digital's EIDE, although it is unclear if Western Digital sold any ATAPI related products. It is however clear that WD lent its significant expertise when figuring out how to make ATAPI devices coexist with IDE hard disk on a single cable without upsetting the BIOS or driver software (a puzzle that was still being solved just before ATAPI 1.2 was completed). Until late 1993, ATAPI development was driven by Western Digital but done in secret, with confidential drafts of the ATAPI specification circulated amongst a handful of companies. From the Oak Technology related litigation we know that ATAPI A1.3 specification was dated June 10, 1993. We don't know any dates of the earlier versions, but it is unlikely that A1.3 would have been the initial version; it might however been the first version seen outside of Western Digital. Conclusion We still don't know who shipped the first ATAPI CD-ROM drive and when. The pre-1994 drafts of the ATAPI specification remain as secret as they were back in the day. That is a shame, because properly supporting certain old drives (such as the NEC CDR-260) would require the ATAPI 1.7B specification. We did find out that Oak Technology was an important player in the early ATAPI CD-ROM drive evolution, and that ATAPI CD-ROM drives were made in early 1994, perhaps even in late 1993, before the earliest ATAPI standard (revision 1.2) was finalized. These drives tend to be incompatible with standard ATAPI driver software, but work with the DOS drivers that these drives shipped with. This entry was posted in CD-ROM, PC history, Standards, Undocumented. Bookmark the permalink. - Looking for High Sierra 4 Responses to The Secret History of ATAPI 1. [yH5B][8bd8] David C. says: November 24, 2021 at 9:10 pm Another thing worth mentioning is that internal Zip drives shipped with three interfaces: SCSI, IDE (not ATAPI) and ATAPI. I remember that OS/2 at the time could support SCSI and IDE, but had problems with ATAPI drives. Later versions had an updated driver and could support the ATAPI drive. This is in contrast to the DOS world, where the IDE version required proprietary Iomega driver software and the ATAPI version "just worked". See also https://en.wikipedia.org/wiki/Zip_drive#Interfaces 2. [yH5B][8f7d] Michal Necasek says: November 24, 2021 at 9:55 pm It sounds like those Zip drives behaved like standard hard disks (with removable media). That was not going to really work for CD-ROMs due to the different sector size and the additional functionality CD-ROM drives needed (audio playback). I believe this was tried for CD-ROMs (making them look like IDE hard disks) but didn't really get anywhere. 3. [yH5B][7978] zeurkous says: November 24, 2021 at 10:27 pm That's an issue that has returned later to plague DVD-RAM: most drives represent them as CDs (like the other media they support), obstructing the direct access they're designed for. How did this work w/ MO drives...? Did they use ATAPI? 4. [yH5B][8f7d] Michal Necasek says: November 24, 2021 at 11:09 pm At least Fujitsu DynaMO drives used ATAPI (or SCSI). Because ATAPI is largely SCSI over ATA, it's very flexible and can support all sorts of devices. Leave a Reply Cancel reply Your email address will not be published. Required fields are marked * [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comment [ ] Name * [ ] Email * [ ] Website [ ] [Post Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. * Archives + November 2021 + October 2021 + September 2021 + August 2021 + July 2021 + June 2021 + May 2021 + April 2021 + March 2021 + February 2021 + January 2021 + December 2020 + November 2020 + October 2020 + September 2020 + August 2020 + July 2020 + June 2020 + May 2020 + April 2020 + March 2020 + February 2020 + January 2020 + December 2019 + November 2019 + October 2019 + September 2019 + August 2019 + July 2019 + June 2019 + May 2019 + April 2019 + March 2019 + February 2019 + January 2019 + December 2018 + November 2018 + October 2018 + August 2018 + July 2018 + June 2018 + May 2018 + April 2018 + March 2018 + February 2018 + January 2018 + December 2017 + November 2017 + October 2017 + August 2017 + July 2017 + June 2017 + May 2017 + April 2017 + March 2017 + February 2017 + January 2017 + December 2016 + November 2016 + October 2016 + September 2016 + August 2016 + July 2016 + June 2016 + May 2016 + April 2016 + March 2016 + February 2016 + January 2016 + December 2015 + November 2015 + October 2015 + September 2015 + August 2015 + July 2015 + June 2015 + May 2015 + April 2015 + March 2015 + February 2015 + January 2015 + December 2014 + November 2014 + October 2014 + September 2014 + August 2014 + July 2014 + June 2014 + May 2014 + April 2014 + March 2014 + February 2014 + January 2014 + December 2013 + November 2013 + October 2013 + September 2013 + August 2013 + July 2013 + June 2013 + May 2013 + April 2013 + March 2013 + February 2013 + January 2013 + December 2012 + November 2012 + October 2012 + September 2012 + August 2012 + July 2012 + June 2012 + May 2012 + April 2012 + March 2012 + February 2012 + January 2012 + December 2011 + November 2011 + October 2011 + September 2011 + August 2011 + July 2011 + June 2011 + May 2011 + April 2011 + March 2011 + January 2011 + November 2010 + October 2010 + August 2010 + July 2010 * Categories + 286 + 386 + 3Com + 3Dfx + 486 + 8086/8088 + Adaptec + AGP + AMD + AMD64 + Apple + Archiving + Assembler + ATi + BIOS + Books + Borland + BSD + Bugs + BusLogic + C + C&T + CD-ROM + Cirrus Logic + CompactFlash + Compaq + Compression + Conner + Corrections + Creative Labs + Crystal Semi + Cyrix + DDR RAM + Debugging + DEC + Development + Digital Research + Documentation + DOS + DOS Extenders + Dream + E-mu + Editors + EISA + Ensoniq + ESDI + Ethernet + Fakes + Fixes + Floppies + Graphics + Hardware Hacks + IBM + IDE + Intel + Internet + Keyboard + Kryoflux + Kurzweil + LAN Manager + Legal + Linux + MCA + Microsoft + MIDI + NetWare + Networking + NeXTSTEP + NFS + Novell + NT + OS X + OS/2 + PC architecture + PC hardware + PC history + PC press + PCI + PCMCIA + Pentium + Pentium 4 + Pentium II + Pentium III + Pentium Pro + Plug and Play + PowerPC + Pre-release + PS/2 + QNX + Random Thoughts + RDRAM + Roland + Ryzen + S3 + SCO + SCSI + Seagate + Security + Site Management + SMP + Software Hacks + Solaris + Sound + Sound Blaster + Source code + Standards + Storage + Supermicro + TCP/IP + ThinkPad + Trident + UltraSound + Uncategorized + Undocumented + UNIX + UnixWare + USB + VGA + VirtualBox + Virtualization + VLB + Watcom + Wave Blaster + Western Digital + Windows + Windows 95 + Windows XP + Wireless + WordStar + x86 + Xenix + Xeon + Yamaha OS/2 Museum Proudly powered by WordPress.