https://dl.acm.org/doi/10.1145/3589246.3595371 ACM Digital Library home ACM home * Advanced Search * Browse * About * + Sign in + Register * * Advanced Search * Journals * Magazines * Proceedings * Books * SIGs * Conferences * People * * More * Search ACM Digital Library[ ] SearchSearch Advanced Search 10.1145/3589246.3595371acmconferencesArticle/Chapter ViewAbstract Publication PagespldiConference Proceedingsconference-collections pldi * Conference * Proceedings * Upcoming Events * Authors * Affiliations * Award Winners * More * Home * Conferences * PLDI * Proceedings * ARRAY 2023 * U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning research-article Open Access Share on * * * * * U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning * Authors: * #Aaron W. Hsu Dyalog, USA Dyalog, USA View Profile , * Rodrigo Girao Serrao Dyalog, UK Dyalog, UK Search about this author Authors Info & Claims ARRAY 2023: Proceedings of the 9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array ProgrammingJune 2023 Pages 22-35https://doi.org/10.1145/3589246.3595371 Published:06 June 2023Publication HistoryCheck for updates on crossmark * 0citation * 0 * Downloads Metrics Total Citations0 Total Downloads0 Last 12 Months0 Last 6 weeks0 * Get Citation Alerts New Citation Alert added! This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. To manage your alert preferences, click on the button below. Manage my Alerts New Citation Alert! Please log in to your account * Save to Binder Save to Binder [loader-7e6] Create a New Binder Name [ ] + Cancel + Create * Export Citation * Publisher Site * * eReader * PDF ARRAY 2023: Proceedings of the 9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning Pages 22-35 PreviousChapterNextChapter ACM Digital Library ABSTRACT The APL notation would appear to be a clear match for convolutional neural networks, but traditional implementations of APL have lagged behind the performance of highly tuned, specialized frameworks designed to execute CNNs on the GPU. Moreover, most demonstrations of APL for neural networking have involved relatively small examples. We explore a more complex example in the U-net architecture and utilize a modern APL compiler with GPU support, Co-dfns, to compare the state of the art of APL against the current crop of specialized neural network frameworks in the form of PyTorch. We compare performance as well as the language design of APL for neural network programming and the clarity and transparency of the resulting code. We found that the complete "from scratch" APL source was on par with the complexity of the PyTorch reference implementation, albeit more foreign, while being more concise and complete. We also found that when compiled with Co-dfns, despite the naive implementation both of Co-dfns and our own code, performance on the GPU and the CPU were within a factor of 2.2 - 2.4 times that of the PyTorch implementation. We believe this suggests significant avenues of future exploration for machine learning language design, pedagogy, and implementation, both inside and outside of the APL community. References 1. Manuel Alfonseca. 1990. Neural Networks in APL. SIGAPL APL Quote Quad, 20, 4 (1990), may, 2-6. issn:0163-6006 https://doi.org/ 10.1145/97811.97816 Google ScholarGoogle ScholarDigital Library Digital Library 2. Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. 2017. Julia: A fresh approach to numerical computing. SIAM review, 59, 1 (2017), 65-98. Google ScholarGoogle Scholar 3. Albert Cardona, Stephan Saalfeld, Stephan Preibisch, Benjamin Schmid, Anchi Cheng, Jim Pulokas, Pavel Tomancak, and Volker Hartenstein. 2010. An Integrated Micro- and Macroarchitectural Analysis of the Drosophila Brain by Computer-Assisted Serial Section Electron Microscopy. PLOS Biology, 8, 10 (2010), 10, 1-17. https://doi.org/10.1371/journal.pbio.1000502 Google Scholar Google ScholarCross RefCross Ref 4. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. 248-255. Google ScholarGoogle ScholarCross RefCross Ref 5. Pedro Domingos. 2012. A few useful things to know about machine learning. Commun. ACM, 55, 10 (2012), 78-87. Google ScholarGoogle ScholarDigital LibraryDigital Library 6. Vincent Dumoulin and Francesco Visin. 2016. A guide to convolution arithmetic for deep learning. arXiv preprint arXiv:1603.07285. Google ScholarGoogle Scholar 7. Charles R. Harris, K. Jarrod Millman, Stefan J van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernandez del Rio, Mark Wiebe, Pearu Peterson, Pierre Gerard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature, 585 (2020), 357-362. https://doi.org/10.1038/s41586-020-2649-2 Google Scholar Google ScholarCross RefCross Ref 8. Troels Henriksen, Niels GW Serup, Martin Elsman, Fritz Henglein, and Cosmin E Oancea. 2017. Futhark: purely functional GPU-programming with nested parallelism and in-place array updates. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation. 556-571. Google ScholarGoogle ScholarDigital LibraryDigital Library 9. Aaron Wen-yao Hsu. 2019. A data parallel compiler hosted on the gpu. Ph. D. Dissertation. Indiana University. Google Scholar Google Scholar 10. Roger Hui. 2017. Stencil Lives. https://www.dyalog.com/blog/2017/ 07/stencil-lives/ Google ScholarGoogle Scholar 11. Roger Hui. 2020. Towards Improvements to Stencil. https:// www.dyalog.com/blog/2020/06/towards-improvements-to-stencil/ Google ScholarGoogle Scholar 12. Roger KW Hui and Morten J Kromberg. 2020. APL since 1978. Proceedings of the ACM on Programming Languages, 4, HOPL (2020), 1-108. Google ScholarGoogle ScholarDigital LibraryDigital Library 13. Kenneth E Iverson. 1962. A programming language. In Proceedings of the May 1-3, 1962, spring joint computer conference. 345-351. Google ScholarGoogle Scholar 14. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093. Google ScholarGoogle Scholar 15. JSoftware. 2014. Vocabulary/semidot. https://code.jsoftware.com/ wiki/Vocabulary/semidot Google ScholarGoogle Scholar 16. D Knuth. 1993. Computer literacy bookshops interview. Also available as http://yurichev. com/mirrors/C/knuth-interview1993. txt. Google ScholarGoogle Scholar 17. Donald E Knuth. 2007. Computer programming as an art. In ACM Turing award lectures. 1974. Google ScholarGoogle Scholar 18. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25 (2012). Google ScholarGoogle Scholar 19. Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE, 86, 11 (1998), 2278-2324. Google ScholarGoogle ScholarCross RefCross Ref 20. Bernard Legrand. 2009. Mastering Dyalog APL (1 ed.). Dyalog Ltd.. Google ScholarGoogle Scholar 21. Inc Math Works. 1992. MATLAB reference guide. Math Works, Incorporated. Google ScholarGoogle Scholar 22. Chigozie Nwankpa, Winifred Ijomah, Anthony Gachagan, and Stephen Marshall. 2018. Activation functions: Comparison of trends in practice and research for deep learning. arXiv preprint arXiv:1811.03378. Google ScholarGoogle Scholar 23. Keiron O'Shea and Ryan Nash. 2015. An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458. Google ScholarGoogle Scholar 24. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024-8035. http:/ /papers.neurips.cc/paper/ 9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf Google ScholarGoogle ScholarDigital LibraryDigital Library 25. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. 234-241. Google ScholarGoogle ScholarCross RefCross Ref 26. Dominik Scherer, Andreas Muller, and Sven Behnke. 2010. Evaluation of pooling operations in convolutional architectures for object recognition. In International conference on artificial neural networks. 92-101. Google ScholarGoogle ScholarCross Ref Cross Ref 27. Rodrigo Girao Serrao. 2022. Transposed convolution. https:// mathspp.com/blog/til/033##transposed-convolution Google Scholar Google Scholar 28. Artjoms Sinkarovs, Robert Bernecky, and Sven-Bodo Scholz. 2019. Convolutional neural networks in APL. In Proceedings of the 6th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming. 69-79. Google ScholarGoogle ScholarDigital LibraryDigital Library 29. Duc Minh Tran, Troels Henriksen, and Martin Elsman. 2019. Compositional Deep Learning in Futhark. In Proceedings of the 8th ACM SIGPLAN International Workshop on Functional High-Performance and Numerical Computing (FHPNC 2019). Association for Computing Machinery, New York, NY, USA. 47-59. isbn:9781450368148 https:// doi.org/10.1145/3331553.3342617 Google ScholarGoogle Scholar Digital LibraryDigital Library 30. Guido Van Rossum and Fred L. Drake. 2009. Python 3 Reference Manual. CreateSpace, Scotts Valley, CA. isbn:1441412697 Google ScholarGoogle Scholar 31. Artjoms Sinkarovs, Hans-Nikolai Viess mann, and Sven-Bodo Scholz. 2021. Array Languages Make Neural Networks Fast. In Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming (ARRAY 2021). Association for Computing Machinery, New York, NY, USA. 39-50. isbn:9781450384667 https://doi.org/10.1145/3460944.3464312 Google ScholarGoogle ScholarDigital LibraryDigital Library Cited By View all [loader] Index Terms 1. U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning 1. Software and its engineering 1. Software notations and tools 1. Compilers 1. Runtime environments 2. General programming languages 1. Language types 1. Parallel programming languages Recommendations * Convolutional neural networks in APL This paper shows how a Convolutional Neural Network (CNN) can be implemented in APL. Its first-class array support ideally fits that domain, and the operations of APL facilitate rapid and concise creation of generically reusable building blocks. For our ... Read More * Performance Evolution of Different SYCL Implementations based on the Parallel Least Squares Support Vector Machine Library In machine learning and scientific computing, some of the biggest challenges are efficient and performant portable computing. With our Parallel Least Squares Support Vector Machine (PLSSVM) library, we have not only developed an unrivaled Support ... Read More * Compiling a Subset of APL Into a Typed Intermediate Language We present a compiler and a typed intermediate language for a subset of APL. The intermediate language treats all numeric data as multi-dimensional arrays and the type system makes explicit the ranks of arrays. Primitive operators are polymorphic in ... Read More Comments Please enable JavaScript to view thecomments powered by Disqus. Login options Check if you have access through your login credentials or your institution to get full access on this article. Sign in Full Access Get this Publication * Information * Contributors * Published in cover image ACM Conferences ARRAY 2023: Proceedings of the 9th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming June 2023 74 pages ISBN:9798400701696 DOI:10.1145/3589246 + General Chairs: + Troels Henriksen University of Copenhagen, Denmark , + Artjoms Sinkarovs Heriot-Watt University, UK Copyright (c) 2023 Owner/Author Sponsors In-Cooperation Publisher Association for Computing Machinery New York, NY, United States Publication History + Published: 6 June 2023 Permissions Request permissions about this article. Request Permissions Check for updates Check for updates on crossmark Author Tags + Compilers + APL + Co-dfns + Neural Networks + Machine Learning + GPU Qualifiers + research-article Conference Acceptance Rates Overall Acceptance Rate17of25submissions,68% Upcoming Conference PLDI '23 + Sponsor: + sigplan ACM SIGPLAN Conference on Programming Language Design and Implementation June 16 - 23, 2023 Orlando , FL , USA Funding Sources * Contributor Metrics Expand All + [DSC_51] Aaron W. Hsu Indiana University Bloomington o Publication Years2011 - 2023 o Publication counts5 o Available for Download4 o Citation count8 o Downloads (cumulative)3,176 o Downloads (6 weeks)63 o Downloads (12 months)315 o Average Citation per Article2 o Average Downloads per Article794 View Full Profile + Rodrigo Girao SerraoSearch about this author Other Metrics View Article Metrics * Bibliometrics * Citations0 * Article Metrics + 0 Total Citations View Citations + 0 Total Downloads + Downloads (Last 12 months)0 + Downloads (Last 6 weeks)0 Other Metrics View Author Metrics * Cited By This publication has not been cited yet PDF Format View or Download as a PDF file. PDF eReader View online with eReader. eReader Digital Edition View this article in digital edition. View Digital Edition * Figures * Other * * Share this Publication link https://dl.acm.org/doi/10.1145/3589246.3595371 Copy Link Share on Social Media Share on * * * * * * * * * 0References * * * Close Figure Viewer Browse AllReturnChange zoom level[ ] Caption View Table of Contents Export Citations Select Citation format[BibTeX ] * Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download,a status dialog will open to start the export process. The process may takea few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download + Download citation + Copy citation Categories * Journals * Magazines * Books * Proceedings * SIGs * Conferences * Collections * People About * About ACM Digital Library * ACM Digital Library Board * Subscription Information * Author Guidelines * Using ACM Digital Library * All Holdings within the ACM Digital Library * ACM Computing Classification System * Digital Library Accessibility Join * Join ACM * Join SIGs * Subscribe to Publications * Institutions and Libraries Connect * Contact * Facebook * Twitter * Linkedin * Feedback * Bug Report The ACM Digital Library is published by the Association for Computing Machinery. Copyright (c) 2023 ACM, Inc. * Terms of Usage * Privacy Policy * Code of Ethics ACM Digital Library home ACM home About Cookies On This Site We use cookies to ensure that we give you the best experience on our website. Learn more Got it! Your Search Results Download Request We are preparing your search results for download ... We will inform you here when the file is ready. Download now! Your Search Results Download Request Your file of search results citations is now ready. Download now! Your Search Results Download Request Your search export query has expired. Please try again.