[HN Gopher] Show HN: Put this touch sensor on a robot and learn ...
___________________________________________________________________
Show HN: Put this touch sensor on a robot and learn super precise
tasks
We just released a very excited touch sensor that finally
simplifies touch sensing for robotics. Our most exciting result:
Learned visuotactile policies for precise tasks like inserting USBs
and credit card swiping, that work out-of-the-box when you replace
skins! To the best of our knowledge, this has never been shown
before with any existing tactile sensor. Why is this important?
For the first time, you could now collect data and train models on
one sensor and expect them to generalize to new copies of the
sensor -- opening the door to the kind of large foundation models
that have revolutionized vision and language reasoning. Would love
to hear the community's questions, thoughts and comments!
Author : raunaqmb
Score : 181 points
Date : 2024-09-20 17:11 UTC (5 hours ago)
(HTM) web link (any-skin.github.io)
(TXT) w3m dump (any-skin.github.io)
| ugh123 wrote:
| For inserting USBs and similar tasks, is it sensing the angular
| change (and/or pressure differences between the two 'fingers') as
| the robot aligns into the hole? (as if the robot is 'feeling'
| it's way to aligning the usb plug).
|
| Other questions: Is the primary skin material a molded silicone
| or possibly TPU (can be 3d printed)?
| sfink wrote:
| Looks like it's a cured silicon, and you can do whatever the
| heck you want with it.
|
| https://www.smooth-on.com/products/dragon-skin-10-slow/
|
| So I don't think you could 3d print it, but you could 3d print
| a mold.
| raunaqmb wrote:
| Yes, you can 3D print a mold and we release this design tool:
| https://cad.onshape.com/documents/f3ec62110b01a3ad0fcb6d85/w.
| .. You can make whatever 2D shape you want in shape_sketch,
| as long as it is within the bounding square, and we
| automatically generate molds with the requisite inlet and
| outlet channels! It is still in prototype mode and we are
| working to make it robust, but it generally works and was
| used to make all the different shapes you see on the website
| and in the paper.
| raunaqmb wrote:
| As for what it is sensing, we learn end to end policies in
| this case, and allow the neural network can pick up on
| whatever it needs for the particular task! but we have run
| experiments with a predecessor of AnySkin, ReSkin:
| https://reskin.dev that indicate you can localize contact
| at sub-mm scale as well as sense normal and shear forces!
| fragmede wrote:
| > Learned visuotactile policies for precise tasks like inserting
| USBs and credit card swiping...
|
| > opening the door to the kind of large foundation...
|
| Sounds like this enables robots to literally open a door, using a
| door handle or door knob. Exciting!
| raunaqmb wrote:
| Yes, and importantly we find that visuotactile policies work
| even when replacing skins. This hasn't been shown before, to
| the best of our knowledge, and opens the door to a number of
| exciting large-scale applications of this sensor.
| simlevesque wrote:
| I love the "Fabrication process" graphic. You can't make it
| simpler than that.
| sfink wrote:
| I don't know anything about this space, but _damn_ , this looks
| impressive!
|
| Could it be used to sort trash and recycling? Could it
| recalibrate if gunk got on it, or as it aged? (I guess silicon is
| probably pretty resistant to aging.) Can it wash and de-stem a
| tomato?
|
| I think I want a trackpad made out of this. How much resolution
| could it get? I suppose I wouldn't want to sacrifice a lot of
| resolution for the pressure, tilt, etc. that I am assuming this
| would provide.
|
| (I said "think", because I might find out that it feels like
| running my finger over skin, and I'm wondering how creepy that
| might feel. I don't really want my laptop to have a fleshy part.)
| kibitzor wrote:
| I've worked in this space in automation with industrial grade
| robots and more bespoke end effectors that don't look like
| mainstream robots, but fulfil specific needs. Responding to
| some of your questions with how I could see the above touch
| sensor helping:
|
| Trash sort and recycling: Not many robots here, majority of
| sorting takes advantage of object material properties. Some
| companies tried to add delta robots to keep up with the high
| rates required to even approach profitability, but they weren't
| good enough. Maybe some municipalities or universities that
| have lots of funding could justify adding robots, but it's just
| hard to financially justify.
|
| Recalibration: I'm curious what the developers have for
| handling reduced magnetic fields over time along with gunk.
| Silicone is washdown rated, but anything soft at high
| throughput with parts will start to wear out and change pickup
| characteristics.
|
| Washing and destemming a tomato is more of a problem to solve
| now that will need another 10+ years of price reductions in
| robot+end effector costs and increased efficiency before it
| beats bulk washing and hand-destemming (or crude machine work).
| Maybe it'll be a grad-student's project for a theoretical
| future home-bot
|
| The Lenovo TrackPoint is likely already 95% of what you'd need
| from a trackpad, but this touch sensor is likely not even
| focused at that market.
|
| Things I see useful for this robot touch sensor:
|
| * Simpler version that detects part presence, is just a Boolean
| feedback of "part detected" which can stick on existing end
| effectors. This is often handled by load calculations of the
| robot to detect if it has a part, but could also detect if a
| part has substantially "moved" while it's been gripped, sending
| a signal to the robot to pause
|
| * Harder to suggest items for food as soft grippers (inflatable
| fingers) will grip at the precise pressure that they're
| inflated, reducing the need for sensitive feedback. The
| application for this touch sensor would be food that needs a
| combination of different pressures to properly secure
| something, can't think of a great example
|
| * Hard to also suggest places where this sensor would help with
| fine alignment, as major manufacturers have motor and arm
| feedback with WAY more sensitivity than the average person
| would realize, google Fanuc " Touch Sensing". But, this could
| help when the end effector is longer and it's harder for the
| joints to detect position
|
| * Fabric manipulation. Fabric is just a hard problem for
| robots, adding in more information about the "part" should be
| helpful. Unlocking more automations for shoe manufacturing at
| reasonable prices is a big wall
| raunaqmb wrote:
| This is a very insightful summary, thank you! A few things to
| add about AnySkin that might be relevant:
|
| - AnySkin expressly handles wear and gunk by being
| replaceable. So if it wears out, and you have a heuristic or
| learned model for the old skin, it will work pretty well on
| the new skin! We verify this through an analysis of the raw
| signal consistency across skins, as well as through
| visuotactile policies learned using behavior cloning. We
| found swapping skins to work for some pretty precise tasks
| like inserting USBs and swiping credit cards.
|
| - Could definitely be used for part motion detection
|
| - Soft, inflatable grippers are effective, but often passive.
| AnySkin is not just soft, but also offers contact information
| from the interaction to actively ensure that blueberry
| doesn't get squished!
|
| - This sensor would be key for robots that seek to use
| learned ML policies in cluttered environments. Robots are
| very likely to encounter scenarios where they see an object
| they must interact with, but the object is occluded either by
| their own end-effector(s) or by other objects. Touch, and an
| understanding of touch in relation to vision becomes critical
| to manipulate objects in these settings.
|
| - Industrial robots do have very sensitive motor and arm
| feedback. However, these systems are bulky and unsafe to
| integrate into household robotic technologies. Sensors like
| AnySkin could be used as a powerful, lightweight solution in
| these scenarios, potentially by integrating with some
| exciting recent household robotics models like Robot Utility
| Models.
|
| - ReSkin, the predecessor to AnySkin, has previously been
| used quite effectively for fabric manipulation! (see work
| from David Held's group at CMU). AnySkin is more reliable as
| well as more consistent and could potentially improve the
| performance seen in prior work.
| sfink wrote:
| > Washing and destemming a tomato is more of a problem to
| solve now that will need another 10+ years of price
| reductions in robot+end effector costs and increased
| efficiency before it beats bulk washing and hand-destemming
| (or crude machine work). Maybe it'll be a grad-student's
| project for a theoretical future home-bot
|
| Heh, fair. I wasn't thinking of this as a practical usage, it
| was just the first thing to come to mind when imagining a
| task requiring a lot of pressure sensitivity and a range of
| forces.
|
| Then again, now that I've said it, I believe the current
| approach to this is to breed really hard, tasteless tomatoes
| and then agitate them in a vat. Perhaps we can eventually get
| tastier produce if robots can handle more fragile things!
|
| Hm... or you could invert things and make a glove, then use
| it as a controller. (VR, or just a richer set of control
| dimensions for eg photo editing or something.) I guess that
| needs to generalize across hand shapes and sizes, not just
| swapping out the glove, but I'd be up for a
| calibration/training phase.
|
| > * Harder to suggest items for food as soft grippers
| (inflatable fingers) will grip at the precise pressure that
| they're inflated, reducing the need for sensitive feedback.
| The application for this touch sensor would be food that
| needs a combination of different pressures to properly secure
| something, can't think of a great example
|
| How do you know the right pressure without feedback? A lot of
| foods vary in firmness over time and ripeness. Lemons, for
| example. I guess most don't, as long as you're sticking to a
| single type of food.
| Animats wrote:
| Nice packaging.
|
| Sensors like that have been around for decades, but this is a
| nice way to package them. The replaceable cover is a big win.
| "Skin" type sensors have been built many times, but the part that
| wears out contained the sensors, so they were not suitable for
| production use.[1]
|
| You have to have a Google account to order one. Even though this
| was funded by Meta.
|
| [1] https://www.youtube.com/watch?v=HLdVkaF9ZR4
| raunaqmb wrote:
| We are just collecting emails on the Google form as contact
| information to get more details when shipping samples. I am
| sorry that the form is asking for a google account - we will
| fix that as soon as possible.
| modeless wrote:
| Very cool! Seems to me like slip detection would work better with
| fingerprint-like ridges molded into the surface. Maybe also
| combined with an accelerometer or mems microphone to sense
| vibration.
| raunaqmb wrote:
| Yes! The sleek form factor leaves a lot of room to integrate
| other sensors and modalities!
| _spduchamp wrote:
| What sensor chip are they using?
| raunaqmb wrote:
| We use the MLX90393
| theamk wrote:
| they say "AnySkin uses the same 5-magnetometer circuitry as
| ReSkin,". and ReSkin uses Melexis MLX90393.
|
| (Interesting fact: in MLX90393, you can customize 2 lower
| address bits using config pins, and there are multiple part
| numbers which only differ in higher address bits - so a single
| bus can have dozens of magnetometers attached. It's a very
| helpful feature, a lot of magnetometers are designed for
| compass use only, and thus have no support for address
| customization at all)
| aaronblohowiak wrote:
| So you embed magnetic particles in silicon rubber and magnetize
| them, then use magnometers to detect how the magnetic field is
| changing from a few different points of reference in order to
| detect the deformation of the rubber and from that you can
| analyze the "pressure points" on the surface. the innovation here
| is that you dont have a lengthy re-calibration of your "input
| signal" to the particular magnet-infused silicone interface
| because the manufacturing makes them consistent enough to be
| replaceable parts?
|
| this makes advanced touch sensors more like machine-cut screws
| than bespoke hand-forged nails.
| swamp40 wrote:
| I'll bet you could open the grippers fully and recalibrate on
| power up.
| mikewarot wrote:
| At the heart of it is a nice 3 axis magnetometer chip[1] in an
| array. The magnetic particles embedded in the replaceable skin
| get oriented in parallel at the magnetization stage of
| manufacture. This is a really interesting mix of stuff towards
| the leading edge of stuff we can all use in the home shop.
|
| [1] https://www.digikey.com/en/products/detail/melexis-
| technolog...
| alnwlsn wrote:
| Seems like the tough part of this is access to the magnetizing
| machine. I wonder if the same effect could be achieved by placing
| tiny off the shelf magnets[0] in the molded part instead?
|
| [0]https://www.kjmagnetics.com/proddetail.asp?prod=D101-N52
| raunaqmb wrote:
| While this is possible, it would create stress concentrations
| within the elastomer and could significantly affect its
| durability. We saw this effect even when using larger magnetic
| particles as with ReSkin, https://reskin.dev. If instead we
| make the elastomer more rigid, it would worsen grasp stability.
| tesch1 wrote:
| Please make a laundry folding robot. #aithatmatters
| a_t48 wrote:
| I put in a request for one, would love to get my hands on it to
| integrate with our framework.
| mkl wrote:
| This seems like it would be really useful for electronic musical
| instruments. E.g. Linnstrument
| (https://www.rogerlinndesign.com/linnstrument) which uses a grid
| of force sensing resistor strips. Do these sensors interfere with
| each other if they're sitting side by side?
| wdwvt1 wrote:
| Very cool! This kind of material is potentially very interesting
| for biotech/lab automation tasks. Any info on surviving
| sterilization techniques (autoclave, ethylene oxide, clydox,
| etc)?
___________________________________________________________________
(page generated 2024-09-20 23:00 UTC)