https://github.com/sbu-fsl/kernel-ml Skip to content Sign up * Why GitHub? + Features - + Mobile - + Actions - + Codespaces - + Packages - + Security - + Code review - + Issues - + Integrations - + GitHub Sponsors - + Customer stories - * Team * Enterprise * Explore + Explore GitHub - + Learn and contribute + Topics - + Collections - + Trending - + Learning Lab - + Open source guides - + Connect with others + The ReadME Project - + Events - + Community forum - + GitHub Education - + GitHub Stars program - * Marketplace * Pricing + Plans - + Compare plans - + Contact Sales - + Education - [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} sbu-fsl / kernel-ml Public * Notifications * Star 47 * Fork 2 * Machine Learning Framework for Operating Systems - Brings ML to Linux kernel Apache-2.0 License 47 stars 2 forks Star Notifications * Code * Issues 0 * Pull requests 0 * Discussions * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Wiki * Security * Insights main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show Loading {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default Loading View all tags 1 branch 1 tag Code Loading Latest commit @iumitakgun iumitakgun Update README.md ... 7ec6fa3 Nov 23, 2021 Update README.md 7ec6fa3 Git stats * 25 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .circleci Update config.yml Nov 12, 2021 .vscode updated with commit signing Nov 12, 2021 benchmark update license time Nov 22, 2021 cmake initial commit for kernel-ml framework Nov 12, 2021 dataset initial commit for kernel-ml framework Nov 12, 2021 docs/images initial commit for kernel-ml framework Nov 12, 2021 examples update license time Nov 22, 2021 include update license time Nov 22, 2021 kernel-interfaces minor compile error fixes Nov 22, 2021 kernel-ml-linux @ c0e1074 updated linux kernel submodule Nov 12, 2021 ml-models-analyses update license time Nov 22, 2021 src update license time Nov 22, 2021 test update license time Nov 22, 2021 .clang-format initial commit for kernel-ml framework Nov 12, 2021 .codecov.yml Update .codecov.yml Nov 12, 2021 .gitattributes initial commit for kernel-ml framework Nov 12, 2021 .gitignore initial commit for kernel-ml framework Nov 12, 2021 .gitmodules updated linux kernel submodule Nov 12, 2021 .travis.yml Update .travis.yml Nov 12, 2021 CMakeLists.txt initial commit for kernel-ml framework Nov 12, 2021 LICENSE Create LICENSE Nov 11, 2021 README.md Update README.md Nov 23, 2021 SECURITY.md initial commit for kernel-ml framework Nov 12, 2021 View code [ ] KML: A Machine Learning Framework for Operating Systems & Storage Systems Table of Contents Setup Clone KML Build Dependencies Install KML Linux Kernel Modifications Specify Kernel Header Location Build KML Double Check Design Example Citing KML README.md logo KML: A Machine Learning Framework for Operating Systems & Storage Systems CircleCI codecov Storage systems and their OS components are designed to accommodate a wide variety of applications and dynamic workloads. Storage components inside the OS contain various heuristic algorithms to provide high performance and adaptability for different workloads. These heuristics may be tunable via parameters, and some system calls allow users to optimize their system performance. These parameters are often predetermined based on experiments with limited applications and hardware. Thus, storage systems often run with these predetermined and possibly suboptimal values. Tuning these parameters manually is impractical: one needs an adaptive, intelligent system to handle dynamic and complex workloads. Machine learning (ML) techniques are capable of recognizing patterns, abstracting them, and making predictions on new data. ML can be a key component to optimize and adapt storage systems. We propose KML, an ML framework for operating systems & storage systems. We implemented a prototype and demonstrated its capabilities on the well-known problem of tuning optimal readahead values. Our results show that KML has a small memory footprint, introduces negligible overhead, and yet enhances throughput by as much as 2.3x. For more information on the KML project, please see our papers * A Machine Learning Framework to Improve Storage System Performance * KML: Using Machine Learning to Improve Storage Systems KML is under development by Ibrahim Umit Akgun of the File Systems and Storage Lab (FSL) at Stony Brook University under Professor Erez Zadok. Table of Contents * Setup + Clone KML + Build Dependencies + Install KML Linux Kernel Modifications + Specify Kernel Header Location + Build KML + Double Check * Example * Design * Citing KML Setup Clone KML # SSH git clone --recurse-submodules git@github.com:sbu-fsl/kernel-ml.git # HTTPS git clone --recurse-submodules https://github.com/sbu-fsl/kernel-ml.git Build Dependencies KML depends on the following third-party repositories: * google/benchmark * google/googletest # Create and enter a directory for dependencies mkdir dependencies cd dependencies # Clone repositories git clone https://github.com/google/benchmark.git git clone https://github.com/google/googletest.git # Build google/benchmark cd benchmark mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release ../ make sudo make install # Build google/googletest cd ../googletest mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release ../ make sudo make install cd ../.. Install KML Linux Kernel Modifications KML requires Linux kernel modifications to function. We recommend allocating at least 25 GiB of disk space before beginning the installation process. 1. Navigate to the kernel-ml/kernel-ml-linux directory. This repository was recursively cloned during setup cd kernel-ml-linux 2. Install the following packages git fakeroot build-essential ncurses-dev xz-utils libssl-dev bc flex libelf-dev bison 3. Install the modified kernel as normal. No changes are required for make menuconfig cp /boot/config-$(uname -r) .config make menuconfig make -j$(nproc) sudo make modules_install -j$(nproc) sudo make install -j$(nproc) 4. Restart your machine sudo reboot 5. Confirm that you now have Linux version 4.19.51+ installed uname -a Specify Kernel Header Location Edit kernel-ml/cmake/FindKernelHeaders.cmake to specify the absolute path to the aforementioned kernel-ml/kernel-ml-linux directory. For example, if kernel-ml-linux lives in /home/kernel-ml/kernel-ml-linux: ... # Find the headers find_path(KERNELHEADERS_DIR include/linux/user.h PATHS /home/kernel-ml/kernel-ml-linux ) ... Build KML # Create a build directory for KML mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-Werror" .. make Double Check In order to check everything is OK, we can run tests and benchmarks. cd build ctest --verbose Design kernel-design Example Citing KML To cite this repository: @TECHREPORT{umit21kml-tr, AUTHOR = "Ibrahim Umit Akgun and Ali Selman Aydin and Aadil Shaikh and Lukas Velikov and Andrew Burford and Michael McNeill and Michael Arkhangelskiy and Erez Zadok", TITLE = "KML: Using Machine Learning to Improve Storage Systems", INSTITUTION = "Computer Science Department, Stony Brook University", YEAR = "2021", MONTH = "Nov", NUMBER = "FSL-21-02", } @INPROCEEDINGS{hotstorage21kml, TITLE = "A Machine Learning Framework to Improve Storage System Performance", AUTHOR = "Ibrahim 'Umit' Akgun and Ali Selman Aydin and Aadil Shaikh and Lukas Velikov and Erez Zadok", NOTE = "To appear", BOOKTITLE = "HotStorage '21: Proceedings of the 13th ACM Workshop on Hot Topics in Storage", MONTH = "July", YEAR = "2021", PUBLISHER = "ACM", ADDRESS = "Virtual", KEY = "HOTSTORAGE 2021", } About Machine Learning Framework for Operating Systems - Brings ML to Linux kernel Topics machine-learning kernel-module linux-kernel operating-systems auto-tuning mlsys Resources Readme License Apache-2.0 License Releases 1 tags Packages 0 No packages published Languages * C 74.6% * Shell 15.8% * C++ 6.1% * CMake 3.2% * Makefile 0.3% * (c) 2021 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.