https://github.com/beehive-lab/TornadoVM/pull/369 Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} beehive-lab / TornadoVM Public * Notifications * Fork 94 * Star 1.1k * Code * Issues 29 * Pull requests 5 * Discussions * Actions * Projects 1 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights New issue Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community. Pick a username [ ] Email Address [ ] Password [ ] [ ] Sign up for GitHub By clicking "Sign up for GitHub", you agree to our terms of service and privacy statement. We'll occasionally send you account related emails. Already on GitHub? Sign in to your account Jump to bottom Introduce Tensor API v0.1, Tensor Utilities and compatiblity with ONNX RT #369 Open mikepapadim wants to merge 45 commits into beehive-lab:develop base: develop Choose a base branch [ ] Branches Tags Could not load branches Branch not found: {{ refName }} {{ refName }} default Could not load tags Nothing to show {{ refName }} default Are you sure you want to change the base? Some commits from the old base branch may be removed from the timeline, and old review comments may become outdated. Change base from mikepapadim:feat/tensors_api_v0.1 Open Introduce Tensor API v0.1, Tensor Utilities and compatiblity with ONNX RT #369 mikepapadim wants to merge 45 commits into beehive-lab:develop from mikepapadim:feat/tensors_api_v0.1 +2,046 -145 Conversation 0 Commits 45 Checks 0 Files changed 28 Conversation This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters mikepapadim Copy link Member @mikepapadim mikepapadim commented Mar 29, 2024 * edited Description This pull request introduces a new Tensor API v0.1, providing a comprehensive set of classes and utilities for working with tensors of various data types along with compatibility with ONNX RT Java bindings. The primary changes include: * AbstractTensor Interface: An abstract base class that defines the common contract for all tensor implementations. * Tensor Classes: A set of concrete tensor classes for different data types, including TensorFloat32, TensorFloat16, TensorInt32, TensorInt16, TensorInt64, TensorFloat64, and TensorByte. These classes extend the AbstractTensor interface and provide type-specific functionality. * DType Class: A utility class that encapsulates data type information for tensors, facilitating type-safe operations and conversions. * Shape Class: A class representing the shape (dimensions) of a tensor. Also, this PR adds test to show compatibility with ONNX RT Java bindings-> public void testOnnxCompatibility() throws OrtException { Shape shape = new Shape(1, 3, 224, 224); TensorFloat32 tornadoTensor = new TensorFloat32(shape); tornadoTensor.init(2f); OnnxTensor outputTensor = null; try (OrtEnvironment env = OrtEnvironment.getEnvironment()) { // Load the MobileNet V2 ONNX model OrtSession session = env.createSession(MODEL_PATH, new OrtSession.SessionOptions()); OnnxTensor inputTensor = OnnxTensor.createTensor(env, tornadoTensor.getFloatBuffer(), shape.dimensions()); Map inputMap = new HashMap<>(); inputMap.put(INPUT_TENSOR_NAME, inputTensor); // Run the model inference try (OrtSession.Result outputMap = session.run(inputMap)) { Optional optionalOutputTensor = outputMap.get(OUTPUT_TENSOR_NAME); if (optionalOutputTensor.isEmpty()) { throw new IllegalArgumentException("Output tensor not found in model output."); } outputTensor = (OnnxTensor) optionalOutputTensor.get(); } } finally { Assert.assertNotNull(outputTensor); } } Overview of the Tensor API Architecture: TensorInt16 Notes: It adds a dependency to ONNX runtime in the tornado-unittests packages, as an alternative we could move the compatibility tests into a seperate repo or submodule. Backend/s tested Mark the backends affected by this PR. * [*] OpenCL * [*] PTX * [ ] SPIRV OS tested Mark the OS where this PR is tested. * [*] Linux * [ ] OSx * [ ] Windows How to test the new patch? Provide instructions about how to test the new patch. make jdk21 backends=opencl # Test how to use Tornado Tensor Types with ONNX RT tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorAPIWithOnnx # Test Tensor Types for all the supported DTypes tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes --------------------------------------------------------------------- Sorry, something went wrong. All reactions mikepapadim added 30 commits February 27, 2024 18:06 @mikepapadim Add Tensor and DType classes, and tensor testing ... effe843 Implemented Tensor, DType classes in `uk.ac.manchester.tornado.api.types.tensors` package. Additionally, a tensor testing class TestTensorTypes was added in `uk.ac.manchester.tornado.unittests.tensors`. The Tensor class contains methods relating to tensor data type and shape, while DType enum lists all supported data types. @mikepapadim Refactor DataType enum into several class files ... 87e90ad This refactor turns the data types enum (DType) into several separate class files under a dtype package, aligning with good OOP practices by giving each datatype its own class. It also adds new Tensor classes with dtype and shape properties, and updates relevant tests and segments accordingly. @mikepapadim wip e704149 @mikepapadim add tensor 4950843 @mikepapadim Remove Tensor classes and update DType subclasses ... 546c15b This commit removes the Tensor classes (`FloatTensor`, `HalfFloatTensor`, `Tensor`) and adds specific data type names to the respective DType subclasses (`Bool`, `Double`, `Float`, `HalfFloat`, `Int16`, `Int32`, `Int64`, `Int8`, `QInt32`, `QInt8`, `QUInt8`). This design change simplifies the overall architecture by removing unnecessary classes and enhances code readability and maintainability. @mikepapadim Refactor TensorArray class to Tensor and update its methods ... 0b393f9 This commit renames the `TensorArray` class to `Tensor`. It also removes the necessity for an explicit cast method by taking `DType` as a parameter upon initialization. This refactor improves clarity of code, simplifies the class structure and makes it easier to work with tensors of different data types. @mikepapadim Rename HalfFloat class to HF and update references ... c2593e3 This commit changes the name of the `HalfFloat` class to `HF` for conciseness. All the references to `HalfFloat` have also been updated to reflect this change such as in the `DType` class. The renaming will make the code more succinct and manageable. @mikepapadim Update Tensor to support HalfFloat type ... 367a6dd Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity. @mikepapadim Update Tensor to support HalfFloat type ... 3f05f59 Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity. @mikepapadim Update Tensor to support HalfFloat type ... 96fc95c Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity. @mikepapadim Refactored HalfFloat handling and enabled Tensor support ... 6a6f6fa This update introduces support for the HalfFloat data type in the Tensor class and includes modifications to multiple components for this purpose. The code has also been refactored for improved clarity, including the relocation of node replacement functions in the TornadoHalfFloatReplacement section. @mikepapadim Update HalfFloat and Tensor handling across components ... 650a51e This commit introduces streamlined support for HalfFloat data type within the Tensor class and includes necessary adjustments across various components. The reshuffling of certain functions, particularly within the TornadoHalfFloatReplacement section, has also been undertaken to improve code clarity @mikepapadim WIP e73a7cd @mikepapadim Refactor and removed verbose logs from OCLTornadoDevice and OCLObject... ... 00948c9 ...Wrapper classes This commit includes several improvements in the handling of OpenCL devices and object wrapping. The unnecessary verbose logs have been removed from both classes. The code of OCLTornadoDevice has been simplified using variable capturing and switch expressions. The handling of task types and atomic variables has been streamlined for better readability and efficiency. @mikepapadim Refactor TornadoDataflowAnalysis code for clarity and efficiency ... 41a01f6 Implemented the instance of pattern in the TornadoDataflowAnalysis class to simplify repetitive if-else structures and enhance code readability. Also, removed redundant else statements to further improve the code maintainability and clarity. @mikepapadim Refactor guard elimination in TornadoHalfFloatFixedGuardElimination ... baa9acc Streamlined code in TornadoHalfFloatFixedGuardElimination by utilizing the instance of pattern for PiNode placeholderInput. This produces cleaner, easier to read code and eliminates unnecessary casting operations. We also removed unneeded "if" conditions for improved efficiency and clarity. @mikepapadim Remove array creation methods in DType ... dc3dabd The array creation methods in DType (Tornado API) have been removed. These changes make the code cleaner and easier to read, as these removed methods seemed to be extras that are not needed, hence streamlining the code. @mikepapadim Add tensor addition test and remove redundancy in TestTensorTypes ... d1efa26 An additional unit test for tensor addition with floating point numbers is added. In the same file, some non-essential print statements and commented out verifying code have been removed to avoid redundancy and make the code cleaner. @mikepapadim Add tensor addition test and remove redundancy in TestTensorTypes ... 9620c05 An additional unit test for tensor addition with floating point numbers is added. In the same file, some non-essential print statements and commented out verifying code have been removed to avoid redundancy and make the code cleaner. @mikepapadim Refactor code block in OCLHotSpotBackendFactory.java ... 91c1d87 The switch-case code block in OCLHotSpotBackendFactory.java is refactored to use the new Java Switch Expressions. This makes the code easier to read and understand while preserving the original functionality. @mikepapadim Remove calculateSize method from DType.java ... 5f18365 The function 'calculateSize' in the DType.java file has been removed. It seems like it was superfluous code as it was not being used anywhere, clearing it out for better code readability and maintainability. @mikepapadim Add Tensor creation from array and improve unit tests ... f36373d Functionality was added to allow creation of a Tensor directly from a float array in Tensor.java. This is done through the static method 'fromArray' and the helper method 'createSegment'. This necessitated updates to existing unit tests with new tests written for the added functionality. @mikepapadim Merge branch 'mikepapadim/segment_slice' into mikepapadim/ tensors_v0.5 086de25 @mikepapadim Add detailed comments and improve documentation in Tensor modules 5b6aabc @mikepapadim Refactor tensor unit tests and add random data population 6ba50f8 @mikepapadim Refactor DType.java and enhance documentation 4520e84 @mikepapadim Add tensor constructor and float buffer conversion. 9e2af7d @mikepapadim Fix merge conflicts 4655bd5 @mikepapadim Add tensor types and adjust base class permissions ... 58d8216 New tensor types (Int32, Int64, Float32, etc.) have been added to support various data types. The TornadoNativeArray class permissions were adjusted to accommodate these new tensor types. The existing Tensor class was also extended to implement the AbstractTensor interface. @mikepapadim Update and extend tensor types for data handling. f32c116 mikepapadim added 14 commits March 25, 2024 16:39 @mikepapadim Added AbstractTensor interface to Tornado API. ... a6eccba A @mikepapadim Refactored Tornado API with AbstractTensor interface and updated clas... ... 799973f ...ses accordingly. @mikepapadim Add Apache license header to tensor API classes ebe51cf @mikepapadim Update tensor classes and remove debug prints. 3ab369c @mikepapadim Expand tensor classes with functions and attributes. b3fe88b @mikepapadim Refactor TornadoNativeArray class for readability. 5f10c5a @mikepapadim Add onnxruntime dependency to unittests module ce2ffcd @mikepapadim Add TestTensorAPIWithOnnx class in tornado-unittests a2f091b @mikepapadim Modify Shape class to use long instead of int 117790d @mikepapadim Add toHeapArray and getFloatBuffer methods to TensorFloat32 7fa2853 @mikepapadim Refactored TestTensorAPIWithOnnx unit test a0d8ec4 @mikepapadim Refactored the TestTensorAPIWithOnnx unit 49102c4 @mikepapadim Remove unused code in TornadoNativeTypeElimination 1601180 @mikepapadim Update model path in TestTensorAPIWithOnnx. 81994ff @mikepapadim mikepapadim added the API label Mar 29, 2024 @mikepapadim mikepapadim requested review from jjfumero, mairooni and stratika March 29, 2024 14:22 @mikepapadim mikepapadim self-assigned this Mar 29, 2024 @mikepapadim mikepapadim changed the title [DEL:Introduce Tensor API v0.1 and Auxiliary Utilities in the Tornado API:DEL] [INS:Introduce Tensor API v0.1, Tensor Utilities and compatiblity with ONNX RT:INS] Mar 29, 2024 @mikepapadim Merge branch 'develop' of github.com:beehive-lab/TornadoVM into feat/ ... ... bd24d3e ...tensors_api_v0.1 Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment Reviewers @jjfumero jjfumero Awaiting requested review from jjfumero @mairooni mairooni Awaiting requested review from mairooni @stratika stratika Awaiting requested review from stratika At least 2 approving reviews are required to merge this pull request. Assignees @mikepapadim mikepapadim Labels API Projects None yet Milestone No milestone Development Successfully merging this pull request may close these issues. None yet 1 participant @mikepapadim Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later. Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.