[HN Gopher] Show HN: Open source machine learning inference acce...
___________________________________________________________________
Show HN: Open source machine learning inference accelerators on
FPGA
Author : 323454
Score : 36 points
Date : 2022-03-09 15:37 UTC (7 hours ago)
(HTM) web link (www.tensil.ai)
(TXT) w3m dump (www.tensil.ai)
| sathergate wrote:
| how does this compare to apache TVM?
| 323454 wrote:
| Great question - TVM / OctoML are a great option if you have an
| off-the-shelf ML model and off-the-shelf hardware. Tensil is
| different in that you can actually customize the accelerator
| hardware itself, allowing you to get the best trade-off of
| performance / accuracy / power usage / cost given your
| particular ML workload. This is especially useful if you want
| to avoid degrading the accuracy of your models (e.g. through
| quantization) to achieve performance targets.
| sathergate wrote:
| That makes sense. So is this only for edge compute use cases,
| or can I use tensil on an FPGA I have running in my data
| centre?
| 323454 wrote:
| You absolutely can use it in a data centre. You can even
| tape out an ASIC using these designs! Currently we've done
| most of our prototyping with edge FPGA platforms but if you
| want to try other platforms we'd love to help you get
| started. You can email me at tom@tensil.ai or use the
| contact methods on the website.
| bjourne wrote:
| Wow! This looks amazingly impressive. Super-duper good. If the
| software is as impressive as the website (haven't tried it out
| yet!), you'll make tons of $$$ on this product. If I were a vc
| with millions I'd be begging you to take some. I wonder if you
| plan to support CGRAs and LSTMs? Also, what about quantized and
| compressed models? Model compression is a sore point and afaik,
| there aren't any good tools that lets you make tradeoffs between
| accuracy and compute efficiency.
| 323454 wrote:
| Just saw your edit re: model compression. One thing that Tensil
| can do is help you avoid the need to quantize or compress your
| model entirely! For example, we've found that using a 16-bit
| fixed point numeric data type preserves almost all the model
| accuracy while not sacrificing performance thanks to the huge
| amount of parallelism available on FPGA.
|
| The broader point is that Tensil is extremely flexible, so you
| can try out lots of different accelerator configurations to
| find the one that works best for your ML model. Think of it as
| optimizing the hardware first, then the software if needed.
|
| We're actually working on a tool to manage and automate this
| hardware architecture search - watch this space!
| 323454 wrote:
| Thank you for the kind words! Just to clarify, the core
| technology here is free and open source, anyone can use it
| right now for free. We do have commercialization plans in
| addition - we may explore things like additional paid features
| for enterprise use or paid tiers of extra support.
|
| Regarding LSTMs, yes. We're aiming to support all machine
| learning model architectures: do you have any particular models
| you're interested in that we should be prototyping with?
|
| For CGRAs, we don't have any immediate plans to explicitly
| support them. What kind of use case do you have in mind?
| Generally, any platform that can implement a blob of generated
| RTL should be something we can work with quite easily.
| rowanG077 wrote:
| What kind of FPGAs can this reasonably run on? Is that model
| dependent? Could a small model run on an ICE40 FPGA? I looked
| over the doc but I can't find anything concrete.
| 323454 wrote:
| It depends on the model, yes. Here are some examples in the
| benchmarks section of our docs:
| https://www.tensil.ai/docs/reference/benchmarks/
|
| We haven't specifically tested on any ICE40 FPGAs yet - if this
| is something that you'd really like to see, let me know! Taking
| a look at the lineup, the ICE40 LP8K and LP4K would be suitable
| for running a very small version of the Tensil accelerator.
| You'd want to run a small model in order to get reasonable
| performance.
|
| Generally speaking, FPGAs with some kind of DSP (digital signal
| processing) capability will work best, since they can most
| efficiently implement the multiply-accumulate operations
| needed.
| rowanG077 wrote:
| Cool! Yeah I would be interested in that. I would actually
| have some use cases for edge compute if it can fit into tiny
| FPGAs like the ICE40.
| 323454 wrote:
| That's excellent - feel free to join our Discord if you'd
| like to brainstorm ideas or get help choosing models and
| boards https://discord.gg/TSw34H3PXr
| 323454 wrote:
| Tom from Tensil here - happy to answer questions!
|
| We developed Tensil to bring custom ML accelerators to people who
| don't have the resources of companies like Google, Facebook and
| Tesla. Currently, we're focused on supporting convolutional
| neural network inference on edge FPGA (field programmable gate
| array) platforms, but we aim to support all model architectures
| on a wide variety of fabrics for both training and inference.
|
| Tensil is different from other ML accelerators in that it is open
| source and really easy to use. For example, you can generate a
| custom accelerator with one command: $ tensil rtl
| --arch <my_architecture>
|
| You can compile your ML model targeting that accelerator like so:
| $ tensil compile --arch <my_architecture> --model <my_model>`
|
| Running your model on FPGA is as simple as doing the following:
| $ tcu.load_model(<compiled_model>) $ outputs =
| tcu.run(inputs)
|
| The accelerator generator was developed in Chisel and we built
| our own parametrizable compiler to target it. The link in the
| post takes you to the documentation, and here's a link to the
| Github repository: https://github.com/tensil-ai/tensil/
| normcoreashore wrote:
| Sweet!
___________________________________________________________________
(page generated 2022-03-09 23:01 UTC)