https://github.com/noahtren/Freewire Skip to content Sign up Sign up * Why GitHub? Features - + Mobile - + Actions - + Codespaces - + Packages - + Security - + Code review - + Project management - + Integrations - + GitHub Sponsors - + Customer stories- * Team * Enterprise * Explore + Explore GitHub - Learn and contribute + Topics - + Collections - + Trending - + Learning Lab - + Open source guides - Connect with others + The ReadME Project - + Events - + Community forum - + GitHub Education - + GitHub Stars program - * Marketplace * Pricing Plans - + Compare plans - + Contact Sales - + Education - [ ] [search-key] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up Sign up {{ message }} noahtren / Freewire * Notifications * Star 120 * Fork 5 An experiment with "freely" wired neural networks (no layers) 120 stars 5 forks Star Notifications * Code * Issues 3 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights master Switch branches/tags [ ] Branches Tags Nothing to show {{ refName }} default View all branches Nothing to show {{ refName }} default View all tags 1 branch 0 tags Go to file Code Clone HTTPS GitHub CLI [https://github.com/n] Use Git or checkout with SVN using the web URL. [gh repo clone noahtr] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Go back Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Go back Launching Xcode If nothing happens, download Xcode and try again. Go back Launching Visual Studio If nothing happens, download the GitHub extension for Visual Studio and try again. Go back Latest commit @noahtren noahtren Update README.md ... 87d5847 Oct 16, 2019 Update README.md 87d5847 Git stats * 49 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time examples remove unused import Sep 19, 2019 freewire faster graph updates using cpu Sep 19, 2019 .gitignore first commit Aug 28, 2019 README.md Update README.md Oct 17, 2019 requirements.txt first commit Aug 28, 2019 setup.py first commit Aug 28, 2019 style_notes.md first commit Aug 28, 2019 test.py test model and graph sync Oct 3, 2019 View code README.md Freewire: Freely Wired Neural Networks Freewire is a Keras-like API for creating optimized freely wired neural networks to run on CUDA. Freely wired neural networks are defined at the level of individual nodes (or neurons) and their connections, instead of at the level of homogeneous layers. The goal of Freewire is to make it so that any arbitrary DAG of artificial neurons can be defined first and the optimized set of operations can be compiled at runtime and run on CUDA. This repository is a starting point for exploring how to design and optimize neural networks that can be wired in very novel ways at the level of individual artificial neurons, while retaining the speed and memory efficiency of traditional neural networks. Future versions will likely make use of sparse tensor operations. Compiling Parallel Operations Since freely wired neural networks may not fit the paradigm of having layers, it's necessary to consider ways to optimize them for training and inference. The most time-efficient implementation of a freely wired network would be a series of parallelized operations that extend a 1D tape of numbers, where each operation is a function of the input and the results of all previous operations. This code uses a topological sorting algorithm to find the minimum number of required operations for a given graph. This graphic shows the 1D tape on the left and the freely wired neural network that it represents on the right (biases are left out in this image for simplicity). Also note than the 1D tape is extended to 2D to allow training in batches. [6874747073][6874747073] XOR Gate Example from freewire import Node, Graph, Model # node with no arguments is an input node inputs = [Node(), Node()] # first argument of Node constructor is a list of input nodes hidden = [Node(inputs, activation='sigmoid') for _ in range(0, 5)] output = Node(hidden, activation='sigmoid') # specify which nodes are inputs, hidden, or output nodes when generating graph g = Graph(inputs, hidden, [output]) m = Model(g) # create training data data = [ [0, 0], [1, 0], [0, 1], [1, 1] ] target = [0, 1, 1, 0] # similar API to Keras m.compile(optimizer='sgd', loss='mse') m.fit(data, target, epochs=10000, batch_size=1) print("0 xor 0:", m([0, 0])) print("0 xor 1:", m([0, 1])) print("1 xor 0:", m([1, 0])) print("1 xor 1:", m([1, 1])) Visualization Example You can visualize a graph to see its architecture and weights (given that the graph is small enough). The visualization is made using the graphviz library. A graph's weights and biases start out as zero. This changes when a graph is used to construct a model. from freewire import Node, Graph, Model from freewire import visualize inputs = [Node(), Node()] hidden1 = Node(inputs) hidden2 = Node([inputs[0], hidden1]) hidden3 = Node([inputs[1], hidden1]) output = Node([hidden2, hidden3]) g = Graph(inputs, [hidden1, hidden2, hidden3], [output]) visualize(g, title="architecture") Now, create a model from this graph and view the updated weights and biases. m = Model(g, initialization="he") visualize(g, title="architecture_and_weights") [6874747073][6874747073] More Examples See the examples folder for more examples, including a network for MNIST with randomly wired layers. Installation git clone https://github.com/noahtren/Freewire cd Freewire pip install -e . This will automatically install the requirements in requirements.txt: * numpy * torch==1.2.0 * graphviz * pydot * Optional: the mnist package to run the example on MNIST dataset About An experiment with "freely" wired neural networks (no layers) Resources Readme Releases No releases published Packages 0 No packages published Languages * Python 100.0% * (c) 2021 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.