[HN Gopher] Shap-E: Generate 3D objects conditioned on text or i...
___________________________________________________________________
Shap-E: Generate 3D objects conditioned on text or images
Author : tim_sw
Score : 273 points
Date : 2023-05-06 01:07 UTC (21 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| qup wrote:
| I just got a 3D printer but I'm a total noob.
|
| How many more steps have to occur between these models and me
| printing them?
| moffkalast wrote:
| If you check the issues, the latest one is from Jozef Prusa
| interested in doing straight up Printables integration... so
| not that many.
| dceddia wrote:
| I'm not sure what format it's generating these models in, but
| if they're SDL, you should be able to drop them into a slicer
| program (like PrusaSlicer, Cura, etc) and generate the gcode
| there (the 3D printer movement instructions). Then load that
| gcode into your printer (via SD card, wifi, whatever it
| supports) and go!
|
| If these aren't in SDL, but are some format Blender can open,
| then Blender can turn them into SDL. It looks from the
| description like it's compatible with Blender.
| kuida0r3 wrote:
| From the jupyter notebook in the repo [0], the output is in
| .ply or polygon file format which is used by some 3D scanners
| and can be converted to STL supported by most 3D printers.
|
| 0-https://github.com/openai/shap-e/blob/main/shap_e/examples/
| s...
| krisoft wrote:
| > How many more steps have to occur between these models and me
| printing them?
|
| Shouldn't be too hard. Based on the code of the examples it
| seems it can output polygonal meshes in ply format. Either your
| slicer natively supports that, or you can use something (for
| example blender) to convert the ply to a format your slicer
| supports.
|
| What can go wrong if the model "looks okay" but have some
| subtle geometric issue (like if it has a missing face, which
| makes it non-manifold or non-watertight) Nowadays many slicers
| have tools to heal such problems, but there can be dragons that
| way.
|
| I looked for an example object in their repo and found this
| one:
| https://github.com/openai/shap-e/blob/main/shap_e/examples/e...
|
| If that is representative it is a very nice clean geometry. I
| have sliced it with prusa slicer no problem and I could be
| printing it within a minute of downloading it if I would need a
| 3d printed cactus in my life.
|
| > I just got a 3D printer but I'm a total noob.
|
| How noob you mean? If this would be literally your first print
| I would recommend printing first a sample file which came with
| your printer. (just to make sure everything is okay with the
| printer itself) But as a second print you can totally try to
| print one of these of course. Just don't give up the hobby if
| something goes wrong :) Something always goes wrong.
| qup wrote:
| > How noob you mean?
|
| I successfully printed a hollow cube yesterday, my first ever
| print, and failed on about 6 attempts at a "mario ring" file.
| It's PETG and I'm just fiddling with settings, getting a lot
| of stringing etc.
|
| Anyway, very new, but I'm prepared for the learning curve.
| I'm a good debugger, and I don't give up easily, so hopefully
| that'll carry through to the printing.
| filoleg wrote:
| I got into 3D printing from zero knowledge a couple of
| months ago, and i found PETG an absolute nightmare to deal
| with compared to PLA.
|
| Mind you, i only bought filament from Prusa and
| AtomicFilament, two very respectable manufacturers. Had
| zero issues with their PLA, and it looked beautiful. With
| PETG, no matter how much i tried adjusting live-z levels or
| playing in PrusaSlicer with print speed levels,
| temperature, layer height, etc., it all would mess up at
| some point. Out of 5 prints of the same object, I finally
| managed to get one great print out of PETG. But when I
| tried printing another one, I would start having various
| issues again and would have to adjust things again in hopes
| of eventually getting it right. I just went back to PLA and
| had zero issues at all since then.
|
| Even though I am aware of PLA disadvantages compared to
| PETG, I simply realized that for my purposes, they didn't
| matter nearly as much as consistency and ease of good
| prints.
| mitthrowaway2 wrote:
| I don't want to deny your experience but I want to
| provide a counterpoint. I actually find PETG easier and
| more reliable to print with than PLA, aside from a little
| bit more stringing. It's especially good for large
| objects because the warping is very minimal, and it has
| more strength and flexibility than PLA, and better inter-
| layer bonding strength, so I don't have to worry as much
| about brittle failures or print orientation. I use a
| Prusa MK3S+, with Prusa's textured sheet. After setting
| the first-layer Z height properly and cleaning the bed
| with an alcohol pad, it works flawlessly basically every
| time. I've used both Prusa's PETG and 3rd party filaments
| with equal success.
|
| After many years of falling out of love with 3D printing,
| PETG helped me fall back in love with it.
| krisoft wrote:
| > I don't want to deny your experience but I want to
| provide a counterpoint.
|
| I heard that PETG is more sensitive to humidity from the
| air than PLA. So maybe that is the difference? Maybe you
| live in a dryer climate and the person you are responding
| to has more moisture in the air?
| mitthrowaway2 wrote:
| It's a reasonable thought, but I live in the pacific
| northwest, which is one of the wettest climates in North
| America... it's possible, though.
| qup wrote:
| I'll check out PLA if I can't get PETG going properly. I
| also have some TPU to play with, but haven't yet.
|
| Thanks!
|
| I'm also going to re-run the cube to see if the same
| settings work a second time, prompted by your comment.
| I've really only had failures with the one model.
| wkat4242 wrote:
| Like the others said, forget about petg until you know what
| you're doing. Just use PLA :)
| ehsankia wrote:
| I'm also pretty new, and in my experience, when it gets
| tricky is support. Quality models for 3d printers are made
| to print without supports, but a lot of these normal models
| have a lot of weird details and geometries that require
| supports. And then you gotta fiddle with a bunch of
| settings to find the right parameters for it.
| chaxor wrote:
| Seems extremely similar to the Point-e work done a few months
| ago. I would imagine that just doing Veitoris Rips on the point-e
| would get close to this solution as well, although that would be
| a bit too logic based for the ML world we live in now.
| jahewson wrote:
| That would get you a mesh but not a NeRF.
| [deleted]
| personjerry wrote:
| Every time I click Samples.md and then click back, GitHub tells
| me "Access to this site has been restricted." and times me out
| for a while. Anybody have the same thing or know why?
| knobhandle wrote:
| Had the same issue. On other browser I was able to browse the
| site. Just clear recent history and cookies.
| jamilton wrote:
| The models seem charmingly low-poly from the gifs.
| krisoft wrote:
| Tangentially related: A hobby I found immensely rewarding as of
| late is to ask a generative model to draw something pretty and
| then try to make the object in real life.
|
| I have previously asked dall-e to draw me steampunk watch-like
| gizmos, selected the prettiest and then I went and carved it out
| of wax. And right now I'm 3d modelling a copper owl jewellery
| design dreamed up by Midjourney.
|
| It is a really exciting exercise because it makes me think about
| what do I like about the image, what is the "essence of it". It
| is also stretches my maker muscles because when I design things
| for myself from scratch I always keep in mind the constraints of
| my tools and techniques. But of course the AI has no such
| compunctions. It just draws something pretty, manufacturability
| be damned.
| vertis wrote:
| I've found it very rewarding as a source of fantastical ideas.
| I like that generating items in mid-journey can break me out of
| one line of thinking even if that's because it's result is
| absurd.
| matlo wrote:
| [flagged]
| cinntaile wrote:
| 190 GBP for a spoon. It reminds me of a certain expression,
| something about fools and money...
| dkural wrote:
| Eating is important. Life, is all about eating, and life
| gave rise to intelligence. /que dramatic music.
| cinntaile wrote:
| You should mention that this is your project by the way.
| You're pretending like it's not by writing they instead of
| we, while on your own website you state that you're co-
| founder of oio.
| samstave wrote:
| please share!
| [deleted]
| cl42 wrote:
| I've been playing with this a bit. Sharing some examples that
| aren't from their repo, in case people are curious.
|
| Simple objects are surprisingly good. Examples of a viking helmet
| below:
|
| -- https://phasellm.com/static/img/helmet-01.png
|
| -- https://phasellm.com/static/img/helmet-02.png
|
| More complex objects, e.g., "a viking" are harder though you can
| clearly see what the model is trying to do (cape, sword, person,
| etc.)...
|
| -- https://phasellm.com/static/img/viking-01.png
|
| Maybe more iterations of the model could work. I'm sure it's only
| a matter of time before this gets much better, more details, etc.
| amelius wrote:
| Wouldn't a more promising approach be to go through NeRFs
| first? So first figure out what you want the model to look like
| from various angles, and then try to make a textured 3d model
| out of it.
| georyb wrote:
| I'm curios how much more difficult is generative 3D?
|
| We are barely able to generate 2D 1080p images, without artefacts
| using diffusion methods. How much more complex is the jump to
| production grade 3D?
|
| As someone creating re-usable 3D product models on Polymock @
| https://polymock.com the bigest issue for me, is that you want
| your models to be editable. 3D modellers want to post-process and
| add tweaks to existing 3D models. Current AI generators lack this
| layering structures.
|
| What I actually want, is a 3D generator for textures / materials.
| It should be a much simpler task for current AI.
|
| At the moment, the state of 3D generators is slowly ripening for
| indie game developers. I'm sure that in the near future, the
| models will be high-enough quality to generate a basic character
| set.
|
| Others have mentioned it, but I think NVIDIA has an edge in 3D
| synthesis & rendering. OpenAI is arguable the leader in consumer
| text-transformers - I wonder the goal for releasing this POC (I
| don't think they will allocate any serious resource here)
| placeholder_0 wrote:
| Have you tried texturelab.xyz?
| prox wrote:
| Someone made a SD/blender hybrid add-on where texturing will be
| based on a prompt.
| bemmu wrote:
| HuggingFace space: https://huggingface.co/spaces/hysts/Shap-E
| newswasboring wrote:
| Did it get hugged to death? I only see a runtime error.
| babuloseo wrote:
| TIL what a Ube ice cream cone is
| astrange wrote:
| it's popular in the Philippines and it's an impressive purple
| color. Taro and pandan are good in desserts too.
|
| And then there's durian.
| uhhyeahdude wrote:
| Ube has been an increasingly trendy/popular flavor for all
| sorts of dessert food in America; this is perhaps most
| evident if you visit Hawaii, but it can also be found in
| pretty much any bubble tea shop as a flavor option.
| ubaehawaii makes an ube crinkle cookie that is as delicious
| as it is purple. I'm not affiliated or anything, just a fan
| of good cookies (and ube!)
| alexmolas wrote:
| Finally some open AI from OpenAI
| juunpp wrote:
| Is it? The code in the repository is MIT, but the models are
| downloaded from Azure:
|
| https://github.com/openai/shap-e/blob/main/shap_e/models/dow...
|
| The code license in such cases would not extend to binary blobs
| downloaded from the Internet. And the code is basically useless
| without the blob.
| newswasboring wrote:
| I can't figure it out, what's the license on the blob?
| juunpp wrote:
| Not a lawyer, but at least for code, no license means you
| have no rights to that code, even if it's just one download
| way:
|
| https://opensource.stackexchange.com/questions/1720/what-
| can...
|
| I don't know if the same is true of blobs, and I also
| haven't checked whether the blobs are licensed anywhere on
| their site.
| pabs3 wrote:
| It definitely wouldn't pass Debian's machine learning policy:
|
| https://salsa.debian.org/deeplearning-team/ml-policy/
| https://deepdive.opensource.org/podcast/why-debian-wont-
| dist...
| speedgoose wrote:
| OpenAi also released Whisper in September. They do open stuff
| once in a while.
| nwoli wrote:
| Seems like they're doing the bare minimum for their name not
| to be embarrassing
| startupsfail wrote:
| There was something like that, but seemingly (comparing demo
| videos) much higher quality released a few months ago from NVIDIA
| - https://research.nvidia.com/labs/dir/magic3d/
| astrange wrote:
| The colors on those are strange and unpleasant. I wonder if
| they forgot something basic like gamma correction.
| Racing0461 wrote:
| maybe thats why they released at on github instead of adding it
| as a feature on openai.com, it sucks.
| orliesaurus wrote:
| I actually thought this initially, but I also believe that
| given the nature of where we are going (in the 3d space at
| least) it makes sense to open source something that isn't
| commercially resellable (thinking all the people who just
| repackage openai APIs) and let the community iterate over it
| and make it better perhaps?
| ehsankia wrote:
| I don't understand, if you have a 3d model, why do these gifs
| need to be so low res and aliased? The AI isn't generated gifs,
| why can't they properly render the previews in higher quality?
| The magic3d models don't look that much more detailed, they're
| just rendered properly...
| mysterydip wrote:
| perhaps to hide imperfections/rough edges and make it look
| nicer than it is?
| marviel wrote:
| https://github.com/openai/shap-e/blob/main/shap_e/examples/s...
|
| ^ Single file Jupyter Notebook for Text Into:
|
| (1) Gif (derived from NERFs I suppose)
|
| (2) .PLY
| joeld42 wrote:
| Tip: If you're loading these .glbs in blender, add an "Attribute"
| node in the shader editor, put "Col" in the name field, and
| connect "Color" to "Base Color" in the material to see the
| colors.
| EamonnMR wrote:
| The next super Scribblenauts game is going to be wild.
| [deleted]
___________________________________________________________________
(page generated 2023-05-06 23:02 UTC)