[HN Gopher] Brickit - scans your Lego bricks and helps you build...
___________________________________________________________________
Brickit - scans your Lego bricks and helps you build new creations
Author : ChrisArchitect
Score : 181 points
Date : 2021-06-30 21:23 UTC (2 days ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| tacLog wrote:
| This is a really cool concept. The scene from the video where it
| is identifying bricks from a pile seems like a crazy awesome
| computer vision application. Does anyone have any references to
| how they managed that brick identifying algorithm?
| elephantum wrote:
| Several neural networks, lots of hard work in data labelling,
| more hard work in data synthesis, even more hard work in
| dataset curation.. et voila!
|
| Source: it was me with the team who did all of that.
| lgrebe wrote:
| Super neat! With all this hard work, how is it free? Am I /
| my data being sold or is it just a "student tech demo" with
| an Apple dev account ?
| c9fc42ad wrote:
| This is really cool! Do you have any in depth posts
| describing the ML behind this?
| elephantum wrote:
| Not yet, but it seems that we will talk about it
| elephantum wrote:
| To dig deeper, there are couple of interesting nuances there:
|
| - extreme number of objects on single photo, typical number
| of visible pieces in large pile is 1500-2000
|
| - extreme number of classes in multi-class classification,
| there are ~1000 most common Lego bricks and up to 30000
| classes if you include rare bricks and different patterns
|
| - really hard data labelling: one photo can take up to a 5
| work days to label
| ChrisArchitect wrote:
| example video also found on their instagram (which is where they
| posted their 'how it works'? weird)
|
| https://twitter.com/AlexanderNL/status/1410253599502962692
| dang wrote:
| Since that page has something publicly viewable while
| https://brickit.app/ seems not to, we've switched the URL to
| it. Thanks!
| tiernano wrote:
| WTF! Instagram wont let you view the page without logging in,
| which, I DONT WANT TO! F*K off Zuck! so, to anyone who has a
| product like this or wants to promot it, put a video on Youtube
| (or anywhere for that matter that doesnt require a login to
| view) so I dont have to give Facebook (or anyone else for that
| matter) my info... /end rant...
| Timothee wrote:
| I've tried it this week and it's really cool to see it process a
| big batch of bricks.
|
| I was a bit disappointed by the suggested builds, because they're
| small, but I also understand it's early on and a difficult thing
| to do.
|
| Not vital, but I was surprised that it doesn't look for colors at
| all.
|
| One thing I'd love is the same technology, but used to look for a
| particular part. It does that once you pick a build, but imagine
| building your own thing rather than suggestions and having the
| app tell you where you can find the piece you're looking for. If
| you're adventurous, you would even have a custom tray with a
| spotlight directed at your pile :)
|
| I've been trying to take thousands of mixed parts back into their
| original sets and I was hoping this might be helpful. The
| technology itself probably would, but not packaged as it is for
| the moment.
|
| All that being said: really well done!
| guepe wrote:
| I just tried it on my kids' Legos. I made two scans, one per box:
| it found ~2500 bricks, which I think is a very high hit rate.
| However, the build suggestions are very, very lacking. For
| reference, my kids are 8 and 4. The builds proposed are very
| simple, small, and still mention missing pieces (probably related
| to colors, which honestly most kids don't care about). I think my
| 8yo would like it, she tends to build "worlds", assembling many
| related contraption that each are fairly simple and match the
| themes found on suggestions.
|
| As for my 4yo, forget it. He is a builder, making very elaborate
| and complex constructions, the bigger the better. He would find
| suggestions absolutely uninteresting. I didn't try to scan
| Technic legos (which he is fond of and make astonishingly complex
| contraption given his age). But it's an even more complex problem
| to solve.
|
| I suspect work on the app was focused on recognition, which
| honestly is impressive. The second part, finding models, maybe
| need a bit of work ? I would recommend at least having an option
| to "ignore color": it's nice to have a build that looks good, but
| overall most kids - including myself - like to build first.
| elephantum wrote:
| We ignore color by default now.
|
| MOCs library is not very big at the moment, but we did not
| intend to go viral either :)
|
| The library of different builds will grow for sure!
| weejewel wrote:
| Kids don't need an app like this, they still have imagination
| we've long left behind. :')
| jacquesm wrote:
| This is very impressive given that reliably recognizing a
| particular brick with custom built optics and near ideal lighting
| is hard enough. Props to whoever built this, I very much
| appreciate how much hard work this must have been, especially
| labeling the data.
|
| Do you have any figures on the error rates?
| unilynx wrote:
| They should integrate this with bricklink - I feel for all those
| people who have to catalog all their little bricks to sell them
| or just 1 or 2 cents a piece
|
| (but a great way to spend the holidays, reconstructing your old
| sets and having to order the missing pieces from all over the
| world)
| dharmon wrote:
| We downloaded this last night and my 4-year old has been having a
| lot of fun working through his 50 or so build options.
|
| We routinely get various lego build books from the library, but
| it's frustrating how almost every build requires a few critical
| pieces that we don't have. It'd be cool if this app could tell
| you if there's some set or kit or odd-lot pieces we could buy
| that would suddenly make a large number of builds available.
| elephantum wrote:
| That's what Lego does.
|
| It seems almost intentional that each set has several rare
| pieces that can not be found in other sets.
| jacquesm wrote:
| You can forget about the 'amost', it is intentional and has
| been for many years. The time that you could build a model
| perfectly from the bricks you already had is long gone.
| guepe wrote:
| I think that's what make a model such a nice experience. It's
| different than building yourself: you end-up with something
| that looks definitely better than an amateur build. You
| didn't work on imagination, but end-up with an amazing build.
| Then you can use those unique pieces to put some highlight
| into your own build !
| castlecrasher2 wrote:
| >but it's frustrating how almost every build requires a few
| critical pieces that we don't have
|
| This is to add exclusive value to each set. Look at
| bricklink.com and find an expensive set, and you'll always find
| one or two pieces that are a unique color or completely unique
| to the set that are astronomically high in price. The original
| Millennium Falcon UCS set for example has two grey (I forget
| which specific grey) ladder pieces to highlight the engine, and
| those are the pieces that were most expensive last I looked.
| peteradio wrote:
| Thats NP-hard.
| yccs27 wrote:
| But how big is n?
| alanbernstein wrote:
| I don't think it's NP-hard to identify the individual pieces
| you need, then show you how to buy them on bricklink.
| [deleted]
| JoshGlazebrook wrote:
| Now someone make one for K'nex!
| annoyingnoob wrote:
| Not signing up for Instagram to 'see how it works'. I'm kind of
| turned off by the whole thing now.
|
| Edit: Downvote all you want, throwing details behind a paywall
| isn't helpful. I can only assume that the app requires me to
| login with Facebook or some other nonsense. If you can't share
| the basics without a login to another service then maybe you are
| not legit, or maybe you just don't look like you know what you
| are doing.
| lgrebe wrote:
| Tried it just now and it's amazing. A very palpable experience of
| I assume ML classifiers that I hadn't experienced before. Showing
| me where pieces are located is the best part IMO as I'd usually
| spend most time looking for specific parts when building
| something. That said I'd love to just have an index function
| where I scan the bricks and can then search for eg a 2x4 red
| brick or similar.
|
| Lastly I won't show this to my 5yo just yet because I love the
| things build by them from imagination only and I also think
| having to search for parts, maybe finding something else that's a
| better fit or gives a new idea is an essential part of play
| elephantum wrote:
| > Showing me where pieces are located is the best part IMO as
| I'd usually spend most time looking for specific parts when
| building something.
|
| Exactly that pain brought Brickit to life!
| lgrebe wrote:
| Any chance of getting a search function? Or simply a sorted
| list of all bricks found with "tap to show in scanned pile"
| elephantum wrote:
| Technically it can be done and we had prototypes that did
| something similar.
|
| Hopefully mobile team will manage to implement all the
| improvements that come from users :)
| hoppyhoppy2 wrote:
| To see how it works I need to log in with an Instagram account?
| No thanks.
|
| (thanks to the person who posted an alternate link, but it's
| weird that I can't view the video linked on their homepage)
| elephantum wrote:
| You do not need any login to scan.
|
| Just download the app and core functionality is ready to be
| used.
| packetslave wrote:
| The issue is that the "How it Works" link points to a saved
| Instagram STORY, which requires viewers to be logged in.
|
| If it was a regular Instagram post of a video, people could
| view it without logging in (just tested in an Incognito
| window).
|
| But yeah, probably better to self-host, or at least put it on
| YouTube.
| nightcracker wrote:
| The problem is that your "See how it works" link prompts
| people to log in. I don't want to log in, so I guess I'll
| never find out how it works, or download the app.
| elephantum wrote:
| I see. It would make sense to self-host this video.
|
| Will forward it to product guys
| e12e wrote:
| Self-host might be a good option, or youtube or vimeo. Or
| even Twitter. But Instagram is a wierd choice.
| toast0 wrote:
| Probably not twitter, any remotely popular link shows
| 'sorry we can't show this, maybe try reloading' for users
| without an account, unless they're very persistent.
|
| Meta: this had gotten much better in late January, but is
| back to where it was again.
| alanbernstein wrote:
| Yeah, I can't see the instagram thing despite being
| logged in.
| MillenialMan wrote:
| Just host it on YouTube?
| zuhayeer wrote:
| This is awesome, can people submit creations too? would be cool
| if folks can submit randomly created combinations (as well as the
| pieces required) and it gets added to the directory of possible
| creations.
| lordnacho wrote:
| Are you gonna do android? I have an old iPhone 6 otherwise, will
| it work on that?
| elephantum wrote:
| Yes, Android app is being actively developed right now.
|
| iPhone 6 would be very slow for a pleasant experience. If I'm
| not mistaken.
|
| We use iPhone 8 as a "low end device" for testing.
|
| Latest iPhones are performing extremely well on this task
| ElijahLynn wrote:
| Thanks, I explicitly came to this thread and `ctrl + f` then
| searched for Android to find this comment!
|
| I'd pay money for this on the Android store for my kids to
| use it!
___________________________________________________________________
(page generated 2021-07-02 23:00 UTC)