[HN Gopher] Brickit - scans your Lego bricks and helps you build...
___________________________________________________________________
Brickit - scans your Lego bricks and helps you build new creations
Author : ChrisArchitect
Score : 591 points
Date : 2021-06-30 21:23 UTC (3 days ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| tacLog wrote:
| This is a really cool concept. The scene from the video where it
| is identifying bricks from a pile seems like a crazy awesome
| computer vision application. Does anyone have any references to
| how they managed that brick identifying algorithm?
| elephantum wrote:
| Several neural networks, lots of hard work in data labelling,
| more hard work in data synthesis, even more hard work in
| dataset curation.. et voila!
|
| Source: it was me with the team who did all of that.
| lgrebe wrote:
| Super neat! With all this hard work, how is it free? Am I /
| my data being sold or is it just a "student tech demo" with
| an Apple dev account ?
| caf wrote:
| It seems like "go viral, get acquired by Lego" would be a
| reasonable game plan.
| wjnc wrote:
| Yea, this. An app like this is a total and legal
| potentially huge competitor to Lego. I'd wager Lego will
| happily pay 100 MEUR+ if this would never come to
| fruition in order to keep the new box with book flow not
| slowing due to this idea. The cat is out of the box
| however. This is disruption in optima forma.
| zby wrote:
| Congrats! I have been working (or rather I tried to find
| someone competent to work on an app like this) but it is
| still in proof of concept phase.
| c9fc42ad wrote:
| This is really cool! Do you have any in depth posts
| describing the ML behind this?
| elephantum wrote:
| Not yet, but it seems that we will talk about it
| elephantum wrote:
| To dig deeper, there are couple of interesting nuances there:
|
| - extreme number of objects on single photo, typical number
| of visible pieces in large pile is 1500-2000
|
| - extreme number of classes in multi-class classification,
| there are ~1000 most common Lego bricks and up to 30000
| classes if you include rare bricks and different patterns
|
| - really hard data labelling: one photo can take up to a 5
| work days to label
| brutus1213 wrote:
| Very cool! How do you deal with occlusion? What kind of NN
| are you using? Would an off-the-shelf model like
| Yolo/SSD/FasterRCNN work for this or did you have to devise
| some new tricks?
| tacLog wrote:
| I am really curious about the data labelling. Do you mean
| that you took photos of piles of Legos and labeled them by
| hand. Or individual Legos from different angles?
|
| Also you mentioned data synthesis. How would this be
| possible? Unless your suggesting that you rendered photo
| realistic piles of Legos and used trained on them because
| if that is the case, please do a write up of the project. I
| can't imagine more interesting way to generate training
| data.
| ChrisArchitect wrote:
| example video also found on their instagram (which is where they
| posted their 'how it works'? weird)
|
| https://twitter.com/AlexanderNL/status/1410253599502962692
| dang wrote:
| Since that page has something publicly viewable while
| https://brickit.app/ seems not to, we've switched the URL to
| it. Thanks!
| tiernano wrote:
| WTF! Instagram wont let you view the page without logging in,
| which, I DONT WANT TO! F*K off Zuck! so, to anyone who has a
| product like this or wants to promot it, put a video on Youtube
| (or anywhere for that matter that doesnt require a login to
| view) so I dont have to give Facebook (or anyone else for that
| matter) my info... /end rant...
| mrw wrote:
| Try viewing it on Nitter [1]. It's one of those (no|min)-JS
| frontends, along with Teddit [2] (Reddit), and Bibliogram [3]
| (Instagram).
|
| But still, does anyone know why video quality on Twitter is
| so bad?
|
| [1]: https://nitter.eu/AlexanderNL/status/1410253599502962692
|
| [2]: https://teddit.net
|
| [3]: https://bibliogram.art
| sen wrote:
| The amount of businesses who use Instagram and Facebook as
| their primary method of serving customers is outright
| depressing these days, as both sites now don't let you view
| anything unless you have an account.
|
| I can't see the menu and opening times for half our local cafes
| and restaurants, can't see what's on for local events in my
| town, etc etc.
|
| It's infuriating.
| croes wrote:
| Governments too. Live streaming press conferences on FB, so
| much about data protection and privacy.
| Timothee wrote:
| I've tried it this week and it's really cool to see it process a
| big batch of bricks.
|
| I was a bit disappointed by the suggested builds, because they're
| small, but I also understand it's early on and a difficult thing
| to do.
|
| Not vital, but I was surprised that it doesn't look for colors at
| all.
|
| One thing I'd love is the same technology, but used to look for a
| particular part. It does that once you pick a build, but imagine
| building your own thing rather than suggestions and having the
| app tell you where you can find the piece you're looking for. If
| you're adventurous, you would even have a custom tray with a
| spotlight directed at your pile :)
|
| I've been trying to take thousands of mixed parts back into their
| original sets and I was hoping this might be helpful. The
| technology itself probably would, but not packaged as it is for
| the moment.
|
| All that being said: really well done!
| alisonkisk wrote:
| Finding builds is the easiest part of the whole app, it's just
| a simple subset check on each design.
| guepe wrote:
| I just tried it on my kids' Legos. I made two scans, one per box:
| it found ~2500 bricks, which I think is a very high hit rate.
| However, the build suggestions are very, very lacking. For
| reference, my kids are 8 and 4. The builds proposed are very
| simple, small, and still mention missing pieces (probably related
| to colors, which honestly most kids don't care about). I think my
| 8yo would like it, she tends to build "worlds", assembling many
| related contraption that each are fairly simple and match the
| themes found on suggestions.
|
| As for my 4yo, forget it. He is a builder, making very elaborate
| and complex constructions, the bigger the better. He would find
| suggestions absolutely uninteresting. I didn't try to scan
| Technic legos (which he is fond of and make astonishingly complex
| contraption given his age). But it's an even more complex problem
| to solve.
|
| I suspect work on the app was focused on recognition, which
| honestly is impressive. The second part, finding models, maybe
| need a bit of work ? I would recommend at least having an option
| to "ignore color": it's nice to have a build that looks good, but
| overall most kids - including myself - like to build first.
| elephantum wrote:
| We ignore color by default now.
|
| MOCs library is not very big at the moment, but we did not
| intend to go viral either :)
|
| The library of different builds will grow for sure!
| artursapek wrote:
| you should let people photograph their builds step by step
| and upload them. user generated content would create a nice
| network effect.
| breck wrote:
| This is so cool! UX is so simple and smooth. I was impressed
| by just it counting how many bricks I put out. This will be
| fun with my daughters as they get older and we collect more.
|
| I'm a big fan of all Lego communities like
| https://www.mecabricks.com/ and
| https://www.reddit.com/r/lego/ and your Brickit just made my
| day. Thank you!
| klintcho wrote:
| Oh I was looking for something like this but with a bit of a
| different intent: building from instructions, but of course
| you have to look for pieces in a random pile of legos. Use CV
| to identify the piece(s) you are looking for.
|
| Would be awesome! Actually thought that was what this was
| doing at first as well.
| iNate2000 wrote:
| Just downloaded it and tried it out. For the instructions
| it has, you can tap a part and it lights up the bounding
| box for that piece in the original image.
| dumbfounder wrote:
| Curious about who you are and why you built this. What's the
| story?
| elephantum wrote:
| One of the founders got sick of looking for Lego pieces
| while playing with his son. And he thought that there
| should be an app for that :)
| slg wrote:
| > but we did not intend to go viral either :)
|
| This is hilarious and makes me wonder about the relationship
| of the value of going viral with the repercussions in going
| viral before you feel you are ready for it.
|
| Great, lots more people know about the app.
|
| Not so great, some people's first impressions are negative
| because of flaws that you are probably already working to
| fix.
|
| Overall it is an impressive app with lots of potential to be
| even better.
| elephantum wrote:
| So true!
|
| We were hoping to start marketing alongside with the
| release of Android app )
| newswasboring wrote:
| I always wanted to ask this to someone who made this
| choice, why build the iphone app before Android app? Also
| are there no good (in your view) frameworks where it will
| just work for both the platforms?
| guepe wrote:
| I hope the library evolves, this has potential to give new
| build ideas in a very simple to use package !
| weejewel wrote:
| Kids don't need an app like this, they still have imagination
| we've long left behind. :')
| exporectomy wrote:
| Not at all. This idea that kids just spontaneously form ideas
| from a vacuum is obviously false. Inspiration comes from all
| sorts of guidance. I would have loved to know more techniques
| for building things when I was a kid and was stuck in the rut
| of similar ideas for a long time. I craved books that showed
| how to build things. I also created my own stuff but
| instruction material was gold. It took you to more exciting
| things than you could think of yourself.
| guepe wrote:
| My kids are way more creative than I at same age... because
| they watch speed builds on Youtube compared to me in a
| Vacuum in the 90's. I absolutely agree that inspiration
| comes for all sort of things, not only those videos. I
| applaud this app, I hope it evolves to offer more value to
| older kids.
| dragonwriter wrote:
| > Kids don't need an app like this, they still have
| imagination we've long left behind. :')
|
| My 5-year-old likes playing with built models a lot, and also
| likes building with instructions, but not free building. More
| instructions for the huge pile of Lego we have, he would
| like.
|
| My 3-year-old is pretty much exclusively into (often quite
| elaborate) freebuilding. Even when kids are super creative,
| it doesn't manifest in the same way.
| ezconnect wrote:
| I agree, I observed my 3 year old often and he is more
| creative by applying his imagination. He tends to lose
| interest in building it himself he see something better than
| what he created. When I am helping him color and I color it
| precisely he just lose interest in coloring and say color it
| please you are better. If I don't help him at all he will
| finish the whole piece and even be proud of it. Now I color
| just like him so he will not lose interest during the
| coloring.
| mustacheemperor wrote:
| Well I'm happy someone posted this video to Twitter to be shared
| because the "See how it works" link on brickit.app seems to take
| me to an Instagram account creation page.
|
| Very cool, I expect I'm not alone among HN's audience in having
| grown up building legos and continuing that hobby as an adult.
| Sending this to my parents to see what happens if they point it
| at one of the boxes of random bits left over at home. Fortunately
| I can send this tweet because they do not use instagram.
|
| Edit: I see that in the time it took me to post this comment I
| became the third or fourth person to have registered the same
| complaint. Apologies for the clutter.
| ceilingcorner wrote:
| Isn't the point of legos to use your creativity and come up with
| new ideas?
|
| I really dislike this future of choosing between different
| options, rather than creating new ones.
| [deleted]
| scrollaway wrote:
| Such an arrogant and pretentious attitude, honestly. Come on.
| It's like a carbon copy of "kids these days, reading books
| instead of just using their imagination, what a shame".
|
| You don't have to use the app, you don't have to use Lego, you
| don't have to do anything. You can even bunker yourself up and
| never communicate with the outside world if you want. Just stop
| being so negative with people's cool apps. Please.
| ceilingcorner wrote:
| Your reply here is about 100x more negative than my comment.
| mdavis6890 wrote:
| Please charge for this app. Don't do in-app purchases.
| jacquesm wrote:
| This is very impressive given that reliably recognizing a
| particular brick with custom built optics and near ideal lighting
| is hard enough. Props to whoever built this, I very much
| appreciate how much hard work this must have been, especially
| labeling the data.
|
| Do you have any figures on the error rates?
| dekhn wrote:
| I don't have numbers but they're pretty high. At least, similar
| bricks are often mistaken for each other (1x2 and 2x2, similar
| but slightly differently sloped angles).
| cloudking wrote:
| Now just add a feature to show me where the relevant pieces are
| in the pile :)
| riotnrrd wrote:
| If you watch the video, it shows exactly that at the end
| (albeit quickly).
| postscapes1 wrote:
| Looking forward to trying this with my 6 year old. Not sure if
| already baked in but suggestion would be for it to enter ID of a
| set you know you have in the pile and help you put it back
| together again...
| quickthrower2 wrote:
| Please do this for my fridge/pantry and show the dishes I can
| make!
| rishikeshs wrote:
| Wow!
| amelius wrote:
| I'd like to see a pick-and-place robot to build LEGO models, but
| probably too much to ask for.
| travoc wrote:
| Very cool, but I don't want to sign in with Google, Apple or
| Facebook. How about a stand alone login? For some reason I trust
| you more than big tech.
| unilynx wrote:
| They should integrate this with bricklink - I feel for all those
| people who have to catalog all their little bricks to sell them
| or just 1 or 2 cents a piece
|
| (but a great way to spend the holidays, reconstructing your old
| sets and having to order the missing pieces from all over the
| world)
| dekhn wrote:
| My son (12) and I pulled out the old box of unassembled legos. It
| works fairly well but misidentifies some pieces. Also it's not
| totally clear how to use the "find brick" feature- does it update
| after you remove a brick? Can you make the "find brick" feature
| work in realtime, as you move the camera?
|
| Also, can you disable "show builds with missing pieces"?
| dharmon wrote:
| We downloaded this last night and my 4-year old has been having a
| lot of fun working through his 50 or so build options.
|
| We routinely get various lego build books from the library, but
| it's frustrating how almost every build requires a few critical
| pieces that we don't have. It'd be cool if this app could tell
| you if there's some set or kit or odd-lot pieces we could buy
| that would suddenly make a large number of builds available.
| elephantum wrote:
| That's what Lego does.
|
| It seems almost intentional that each set has several rare
| pieces that can not be found in other sets.
| jacquesm wrote:
| You can forget about the 'amost', it is intentional and has
| been for many years. The time that you could build a model
| perfectly from the bricks you already had is long gone.
| guepe wrote:
| I think that's what make a model such a nice experience. It's
| different than building yourself: you end-up with something
| that looks definitely better than an amateur build. You
| didn't work on imagination, but end-up with an amazing build.
| Then you can use those unique pieces to put some highlight
| into your own build !
| castlecrasher2 wrote:
| >but it's frustrating how almost every build requires a few
| critical pieces that we don't have
|
| This is to add exclusive value to each set. Look at
| bricklink.com and find an expensive set, and you'll always find
| one or two pieces that are a unique color or completely unique
| to the set that are astronomically high in price. The original
| Millennium Falcon UCS set for example has two grey (I forget
| which specific grey) ladder pieces to highlight the engine, and
| those are the pieces that were most expensive last I looked.
| peteradio wrote:
| Thats NP-hard.
| yccs27 wrote:
| But how big is n?
| alanbernstein wrote:
| I don't think it's NP-hard to identify the individual pieces
| you need, then show you how to buy them on bricklink.
| [deleted]
| JoshGlazebrook wrote:
| Now someone make one for K'nex!
| the_arun wrote:
| Cool idea. We should have something to scan our code base & build
| something without writing single line of code.. but just pulling
| code from existing repository.
| annoyingnoob wrote:
| Not signing up for Instagram to 'see how it works'. I'm kind of
| turned off by the whole thing now.
|
| Edit: Downvote all you want, throwing details behind a paywall
| isn't helpful. I can only assume that the app requires me to
| login with Facebook or some other nonsense. If you can't share
| the basics without a login to another service then maybe you are
| not legit, or maybe you just don't look like you know what you
| are doing.
| zactato wrote:
| Maybe I used legos different than everyone else as a child, but
| this would have ruined the experience for me. I don't a device to
| tell me what to build I want to imagine the things myself.
|
| From a technology perspective its very cool, but it doesn't
| appeal to my childhood love of legos
| lifeformed wrote:
| The cool part isn't telling you what to build, it's finding the
| pieces you're looking for.
| wiredfool wrote:
| I gave it a shot with a small sample of my kid's legos and it
| seems like it did pretty well, even with a bunch of technix mixed
| in.
|
| Tho, I'm not sure how well it would really work for approx 10k
| pieces, as there's not really enough flat space in the house for
| them to be spread out.
|
| Anyway, he's off to play with the app for a bit.
| lgrebe wrote:
| Tried it just now and it's amazing. A very palpable experience of
| I assume ML classifiers that I hadn't experienced before. Showing
| me where pieces are located is the best part IMO as I'd usually
| spend most time looking for specific parts when building
| something. That said I'd love to just have an index function
| where I scan the bricks and can then search for eg a 2x4 red
| brick or similar.
|
| Lastly I won't show this to my 5yo just yet because I love the
| things build by them from imagination only and I also think
| having to search for parts, maybe finding something else that's a
| better fit or gives a new idea is an essential part of play
| elephantum wrote:
| > Showing me where pieces are located is the best part IMO as
| I'd usually spend most time looking for specific parts when
| building something.
|
| Exactly that pain brought Brickit to life!
| lgrebe wrote:
| Any chance of getting a search function? Or simply a sorted
| list of all bricks found with "tap to show in scanned pile"
| elephantum wrote:
| Technically it can be done and we had prototypes that did
| something similar.
|
| Hopefully mobile team will manage to implement all the
| improvements that come from users :)
| Cort3z wrote:
| Making an inventor of all my lego and adding a search function
| would be so amazing. This is in my opinion much more appealing
| than getting suggestions for what to make. I'm also not in what
| I assume to be the target audience, but I would likely pay for
| that feature.
| srswtf123 wrote:
| Have you tried taking a complete set and laying it all out for
| view, then seeing if it matches the set correctly?
|
| I don't have a single complete set anymore ... but I do have
| happy nieces, which I vastly prefer :)
| jostmey wrote:
| I can't wait to get home and try this out. Does it work with
| duplos?
| elephantum wrote:
| Not yet, but Duplos and Technic support is widely requested, we
| will add support for them
| hoppyhoppy2 wrote:
| To see how it works I need to log in with an Instagram account?
| No thanks.
|
| (thanks to the person who posted an alternate link, but it's
| weird that I can't view the video linked on their homepage)
| elephantum wrote:
| You do not need any login to scan.
|
| Just download the app and core functionality is ready to be
| used.
| packetslave wrote:
| The issue is that the "How it Works" link points to a saved
| Instagram STORY, which requires viewers to be logged in.
|
| If it was a regular Instagram post of a video, people could
| view it without logging in (just tested in an Incognito
| window).
|
| But yeah, probably better to self-host, or at least put it on
| YouTube.
| nightcracker wrote:
| The problem is that your "See how it works" link prompts
| people to log in. I don't want to log in, so I guess I'll
| never find out how it works, or download the app.
| elephantum wrote:
| I see. It would make sense to self-host this video.
|
| Will forward it to product guys
| e12e wrote:
| Self-host might be a good option, or youtube or vimeo. Or
| even Twitter. But Instagram is a wierd choice.
| toast0 wrote:
| Probably not twitter, any remotely popular link shows
| 'sorry we can't show this, maybe try reloading' for users
| without an account, unless they're very persistent.
|
| Meta: this had gotten much better in late January, but is
| back to where it was again.
| alanbernstein wrote:
| Yeah, I can't see the instagram thing despite being
| logged in.
| MillenialMan wrote:
| Just host it on YouTube?
| bpcpdx wrote:
| Should incorporate this technology in some kind of wearable
| device that scans the floor as you walk and warns you before
| stepping on a lego.
| steveharman wrote:
| Having trouble finding the Android version.... :-(
| amelius wrote:
| Now kids can learn what it means if your job is taken over by AI.
| arketyp wrote:
| Ideally kids can now be creative at new levels, but the cynic
| in me also sees children simply not coming up with their own
| design anymore. Anyway, while I can certainly romanticize the
| tedious search for pieces and perhaps getting ideas in the
| process, quick detection is probably a net positive.
| guynamedloren wrote:
| Super impressive. With a young LEGO builder who has cranked
| through every set and every instruction booklet he gets his hands
| on, I recently delved into the world of MOCs thinking that'd keep
| him busy, but even with sites like Rebrickable (which is
| awesome!), the instruction options are surprisingly limited.
|
| How are you sourcing instructions for Brickit?
| bitcoinmoney wrote:
| What's MOC?
| gentaro2 wrote:
| It stands for My Own Creation, what the AFOL (Adult Fan of
| LEGO) community calls a creation of any kind that they build
| without following official instructions.
| virtualritz wrote:
| Whenever I got a new Lego set, I built it once, using the manual
| and then would take it apart, a day or two later. For the bricks
| to be absorbed into my pile and let me build something new,
| better. Something I had imagined myself.
|
| For me there are definitely two kinds of Lego users. Those who
| struggle with the building instructions. For them following these
| is almost meditative. Then there are those who don't and who have
| their own ideas. They will skim through the instructions and
| build the set once (if at all). And in general just welcome the
| addition of (new) bricks.
|
| For me the resp. set served mostly as fuel for my imagination. If
| it was a spaceship, the next thing I would build was one. Just
| not the one in the set.
|
| My dad built me a chest of drawers with lots of boxes for sorting
| Legos. And sorted them for me when he gave it to me as a present,
| for my 5th b-day. It was almost as tall as me at the time. I.e. I
| could just open and peek into the top drawer.
|
| He also told me why it was important to sort the bricks again,
| after I deconstructed my opi:
|
| I could let my imagination run free and find the bricks I needed
| quickly when building my next piece.
|
| Before, when they were all a pile in a big box, very often the
| creative flow was killed by the annoyance of searching a pile of
| bricks for the one I needed. You start building something and
| eventually you just stop.
|
| This thing looks like a tool for people who lack both. Sorted
| Legos and imagination.
| laurent92 wrote:
| I'm surprised there isn't any one-the-shelf solution for
| sorting Lego. As you said, it's the first step.
| hypertele-Xii wrote:
| The chaos of mixed bricks can be creative in its self. I used
| to play this game where I had a small baseplate as an isolated
| island; Every morning, I'd grab a fistfull of bricks from the
| grand pile to "wash up" on the island shore, then challenge
| myself to build from them a home for the stranded minifig on
| the island. It was exciting _not knowing_ what pieces you 'd
| get!
| rendall wrote:
| Peripheral question. The app appears to be iPhone only.
|
| I'm honestly curious why people do this, and I've never gotten a
| satisfactory answer. iPhone market share is 15% at best while
| Android is 75%. It seems like releasing an iPhone-only first
| iteration kneecaps the enterprise out of the gate. What am I not
| understanding?
| antonium wrote:
| You are overthinking it.
|
| This looks like something that evolved from a side/hobby
| project, so it must've been made for whatever devices that devs
| themselves use. That's it.
| sudo-i wrote:
| Like many things it's a combination of many answers, but from
| my own experience it's a simpler pipeline with Apple phones. On
| Android it's supporting tons of screen sizes, out of date
| versions, different vendor modifications to the OS.
|
| Looking at it as iOS vs Android isn't correct because Android
| is a fragmented mess.
| elephantum wrote:
| Android app is under development right now.
| lancepioch wrote:
| iOS has more market share in the US than Android. Regarding
| paid apps, iOS has higher app store revenue than Android
| overall.
| herval wrote:
| Technical aspects aside (no OS variance, etc), iPhone apps make
| more money than Android (2x or more, last I checked, despite
| being a smaller share of total phones). So targeting iPhone
| first is a common strategy
| wst_ wrote:
| This seems really cool. At least I think it is - I have Android.
| But how does it solve the problem of more complex bricks lying on
| the side and actually hiding their complexity? They may look like
| a standard simple bricks from certain angle.
| zuhayeer wrote:
| This is awesome, can people submit creations too? would be cool
| if folks can submit randomly created combinations (as well as the
| pieces required) and it gets added to the directory of possible
| creations.
| lordnacho wrote:
| Are you gonna do android? I have an old iPhone 6 otherwise, will
| it work on that?
| elephantum wrote:
| Yes, Android app is being actively developed right now.
|
| iPhone 6 would be very slow for a pleasant experience. If I'm
| not mistaken.
|
| We use iPhone 8 as a "low end device" for testing.
|
| Latest iPhones are performing extremely well on this task
| ElijahLynn wrote:
| Thanks, I explicitly came to this thread and `ctrl + f` then
| searched for Android to find this comment!
|
| I'd pay money for this on the Android store for my kids to
| use it!
___________________________________________________________________
(page generated 2021-07-03 23:01 UTC)