[HN Gopher] Backend of Meta Threads is built with Python 3.10
___________________________________________________________________
Backend of Meta Threads is built with Python 3.10
Author : BerislavLopac
Score : 339 points
Date : 2023-07-06 08:17 UTC (14 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| lopkeny12ko wrote:
| Python is just the web server. All the _truly_ performance-
| sensitive components--edge proxy, load balancers, backend
| services, databases, caches, and storage services--are mostly
| C++.
| globular-toast wrote:
| Which is exactly the way Python is meant to be used and has
| always been used.
| citrin_ru wrote:
| I would argue a web server which processes all (or most)
| requests is a performance sensitive part of the system.
| Straightforward rewriting from Python to Go in many cases gives
| at least 2x-3x performance improvement which on FAANG scale can
| mean huge difference in hardware cost.
| brrrrrm wrote:
| Developer velocity is extremely important at FAANG scale.
| that_guy_iain wrote:
| Is it really? Don't they work really slowly compared to
| most of the industry? I really get the impression that it's
| slow progress for things at FAANG.
| super_flanker wrote:
| Not really, especially Facebook (Meta) moves really fast.
| They have a slogan "Move fast, break things", okay not
| "Break things" anymore. But you get the gist, in my
| experience working at almost all of FAANG, Meta and
| Netflix move the fastest.
| PartiallyTyped wrote:
| I think AWS must be the slowest of the bunch tbh. Too
| many meetings.
| super_flanker wrote:
| I agree, but Google is the slowest.
| ItsMeYourUncle wrote:
| Slow compared to startups, very fast compared to
| similarly sized companies, even other tech companies.
| Tade0 wrote:
| It's probably cached to seven hells, which makes that 2-3x
| improvement much less relevant.
|
| Wordpress wouldn't be this successful if this approach didn't
| work. After all, without caching it will happily take over 20
| seconds to render the main page.
| mrcwinn wrote:
| I'm building a tiny little product for trail runners and cyclists
| in Python 3.10 (Flask) at https://www.outandback.com. Python has
| been a lot of fun to pick up (I rely heavily on pandas and some
| ML libraries so it felt like the right stack).
| justsocrateasin wrote:
| Could you describe your tech stack more?
|
| I'm a data engineer trying to teach myself React so I can build
| a full stack app using some APIs and it's certainly a good
| exercise, but I could also certainly build that in Python. Your
| app looks great and I'd like to explore that more!
|
| P.S - fellow runner and cyclist and out and I'm checking out
| the app more now.
| kleton wrote:
| Really, all the performance intensive parts are in various c++
| aggregator and recommendation type services. But the webserver is
| Django, yes.
| fleetfox wrote:
| Django is WSGI/ASGI framework not a webserver. What do they
| actually use to terminate HTTP?
| freeplay wrote:
| Most likely a reverse proxy of some sort.
| [deleted]
| taopai wrote:
| Thanks for your comment. I'm really interested in this topic.
| How do you know the web server is Django? I searched but
| couldn't find this.
|
| Why would they use Django? I did some small projects in it but
| I assumed it wasn't very fast and wouldn't be suitable for a
| big app like this. I would like to know the pros and cons. Why
| didn't they build up something in C++ or Rust? Won't python
| limit the speed of responses despite being build the hard stuff
| in compiled languages? Sorry for being so naive, I am an
| amateur.
| dporter wrote:
| At least a few years ago, most of Instagram's server side
| code was in Python. Python may be slower than C++, but it's
| not about raw speed it's about being "fast enough."
| tinco wrote:
| They deemed getting to the market fast was more important
| than hardware costs. If their architecture is good, then they
| were probably really quick and flexible using Python to glue
| their architecture together which would have been the
| development bottle neck to getting the service up and
| running. All the components of the service can be done by
| separate teams in whatever language they deem most effective.
| reissbaker wrote:
| I worked at IG on a previous iteration of Threads, with some
| of the engineers who wrote the current Threads app. It's
| Django!
|
| (Heavily modded, run on a custom Python JIT, and using an
| extremely custom FB-developed database, also used for IG and
| FB.)
|
| It's Django because IG was originally written in Django back
| in the day. FB's general approach to scaling is keep the
| interface generally similar, and slowly replace things behind
| the scenes as necessary -- rather than doing big rewrites.
| It's seemed to work pretty well.
|
| Ultimately the language used for the web server isn't a huge
| deal compared to the performance of the database, for these
| kinds of massive social apps. Plus, well, they do have the
| custom JIT -- but that's _very_ new, and when I first joined
| IG in 2019, we were running vanilla Python in production.
| taopai wrote:
| Thank you, awesome answer! HN comunity is awesome.
|
| Thanks also to every other answer I received for my
| question, I appreciated all of them.
| DarkNova6 wrote:
| Most of Facebook is built on PHP. I'm surprised they didn't
| choose Laravel.
| yurishimo wrote:
| Laravel wasn't created until 2010/11~
| cptcobalt wrote:
| Laravel is not a requirement to use PHP effectively.
| DarkNova6 wrote:
| Oh, I know. But as a joke I chose the most well known
| framework.
|
| Personally evren I prefer php over python.
| sangnoir wrote:
| It turns out calling the app "Threads _by Instagram "_ is
| not just a branding gimmick. The Instagram backend was
| always Python/Django since before the acquisition.
| mrweasel wrote:
| That's pretty interesting, I wonder if that means that
| it's build by a team at Instagram, and not Facebook. I'd
| assume so.
|
| I get that Facebook is still a huge success, but I do
| find it telling that they opted to put Threads under
| Instagram, rather than Facebook.
| jadtz wrote:
| I think its pretty obvious they went with Instagram
| branding, as Facebook branding has been tarnished for a
| while.
| Solvency wrote:
| I'm confused.
|
| It's Facebook by Meta. Instagram by Meta. Why isn't it
| Threads by Meta?
| epolanski wrote:
| Tied to ig account.
| goykasi wrote:
| The web codebase was once written in PHP, but they moved
| away from that a decade ago.
| rafark wrote:
| It's still very similar to php. Same syntax, same
| execution model. If you look at some hack snippets you
| would think it's php (because it mostly is).
|
| Switching from hack to php and viceversa should be
| extremely easy.
|
| That being said, they chose python because Instagram is
| written in python, they are probably using a lot of
| existing libraries.
| goykasi wrote:
| That wouldn't even make sense. They created hack/hhvm to
| scale and optimize the platform and codebase. Taking a
| step backwards and refactoring (hack != php now) would be
| a horrible idea.
| [deleted]
| leodriesch wrote:
| It is fairly normal to build web servers in a way so they can
| be scaled horizontally (more instances rather than larger
| instances). So they can just have more containers run their
| Django servers and distribute the load between them.
| coding123 wrote:
| Whether it's in C++ or Rust or Python, almost all of any
| slowness would be from database waiting anyway.
| patrec wrote:
| It wouldn't. Databases are fast, python is slow.
| patrec wrote:
| https://techspot.zzzeek.org/files/2015/pymysql_runsnake.p
| ng
| DeathArrow wrote:
| Not really. Look at Techempower web benchmarks.
| sgt wrote:
| Interesting and lovely to hear they decided to use Django. What
| is your source, though?
| strict9 wrote:
| Instagram devs have given a lot talks about python and django
| at conferences. As far as I know IG is still running on
| django so it makes sense that the dev team would stick with
| it for something new.
| appplication wrote:
| How is Django now? I'm a long time python dev (data eng
| space) but new-ish to web dev. I started a Flask project 2
| years ago and found it to be pretty full of footguns, mixed
| messaging on best practices for scalable apps, and the
| ecosystem feels overbloated with vapor ware extensions.
|
| Is this just a Flask problem, or does Django have the same
| issues?
| bjourne wrote:
| Django is the opposite. It holds your hand and tells you
| how to structure your "app", templates, database, and media
| assets. And django-admin is godsend. It's awesome when you
| are on the beaten path cause everything just works. But if
| you step of the beaten path you will have to tweak various
| obscure settings and create weird hooks to override
| Django's defaults. Flask is the opposite. Just a very well-
| written web server and it's up to you to decide things like
| project structure, template engine, orm, asset manager,
| etc. Personally I prefer Django since most of my web
| applications are quite mundane and don't require the
| flexibility Flask can provide.
| kstrauser wrote:
| I've described that as Django is fantastic for writing
| Django apps. If the thing you want to do can be
| completely modeled in the defaults, it's impossible to
| beat.
|
| If that's only 99% true, you might want to investigate
| other options.
| winrid wrote:
| You can always build 99% of the App (crud, ui stuff) in
| Django and the 1% be its own service.
| ak217 wrote:
| Having used Node/express, Rails, Flask, and Django
| extensively, I have been very impressed with Django. The
| framework itself is good, but the superpowers come from the
| ORM and DRF, and a few other great plugins. The stuff you
| can do with the ORM and DRF is nothing short of incredible.
| Now, you could argue that the framework shouldn't be
| tightly coupled to the ORM, and sqlalchemy does provide
| some stiff competition as an example of that, but in my
| experience Django ORM edges out sqlalchemy in terms of
| usability.
|
| There are some footguns around N+1 queries, but they are
| general to all database interfaces and pretty easy to avoid
| IMO.
| sgt wrote:
| Same opinion. Django is marvelous for getting stuff done.
| winrid wrote:
| I can't give Django enough praise. I never thought I'd be
| writing Python.
|
| I've mostly used Node, Java, and Rust, with about 12yrs
| experience now. Only about one year with Python and Django
| recently. I am so much more productive than anything else
| I've tried. Using anything else feels like a mistake for
| web apps or CRUD apis. I also use type hints.
|
| I use Django + django-ninja + django-unicorn for dynamic
| UIs.
|
| Building sidewaysdata.com with django right now! Govscent
| is in Django too, the ORM is nice to combine with AI stuff
| already in python.
|
| I'm also following the Ash framework which looks promising.
| doteka wrote:
| You are also running sidewaysdata.com with DEBUG turned
| on in production, apparently ;)
| winrid wrote:
| fixed
| winrid wrote:
| lol thanks. it's not really ready for use yet anyway :)
| coder_san wrote:
| Django is fantastic. It's much bigger than flask, and has
| one of the best ORMs I've ever used. Also comes with a
| built in admin panel which is always nice. You will want to
| use Django REST Framework to make REST APIs, but if getting
| a site up and running is all you want, it's perfect.
|
| If you want to purely make APIs though, give FastAPI a try.
| pphysch wrote:
| Django is excellent. There are some rough edges and signs
| of age, but the core ORM is fantastic to develop with.
| thewataccount wrote:
| You simply can't beat Django's ORM for general stuff. It's
| too awesome. This alone makes it so hard to choose anything
| else.
|
| I know django doesn't have that "shiny factor" to it these
| days - but it's very reliable.
|
| > mixed messaging on best practices for scalable apps
|
| The WSGI stuff can be kinda confusing and is used across a
| lot of python frameworks including django and I think
| flask?
|
| My advice for "simple scaling" is to start with a separate
| Postgres instance, and then use gunicorn. Use celery
| immediately if you have any "long lived" tasks such as
| email. If you containerize your web layer, you'll be able
| to easily scale with that.
|
| Finally - use redis caching, and most importantly - put
| Nginx in front! DO NOT serve static content with django!
|
| > the ecosystem feels overbloated with vapor ware
| extensions.
|
| This still exists to some degree for some more niche stuff,
| largely because of it's age. Although impressively they'll
| generally still work or work with minimal modifications.
| It's popular enough and old enough that most normal things
| you'd want to do have decent extensions or built in support
| already.
| murkt wrote:
| Ehh, Django ORM always was one of the worst parts of
| Django.
| mrweasel wrote:
| True, but it's the best ORM currently available for any
| language. It might not be the fastest, but it is the one
| that's has the highest level of developer comfort.
|
| Using Django is probably the reason why I can stand using
| SQLAlchemy, it's way to complicated for everything I do
| and it's just not a nice an experience.
| cosmic_quanta wrote:
| > True, but it's the best ORM currently available for any
| language.
|
| Such a strong assertion!
|
| I've used multiple ORMs (not Django's), including some of
| Haskell's type-safe ORMs (e.g. Persistent and Beam). I
| could not imagine going back. What makes Django's ORM so
| great?
| DeathArrow wrote:
| >True, but it's the best ORM currently available for any
| language.
|
| Better than Entity Framework? Consider me impressed.
| mrweasel wrote:
| Depending on what you consider important, and I haven't
| used Entity since... 2010 I think. For me, those two
| doesn't compare, the simplicity of the Django ORM makes
| Entity look dated, complex, like something out of the
| mainframe era.
|
| Entity is way more flexible, in the sense that you can
| use in any .Net project really. The Django ORM have no
| value outside Django, I have yet to anyone use just the
| ORM, without the rest of the framework.
| _fdsfs12123 wrote:
| > Use celery immediately if you have any "long lived"
| tasks such as email
|
| Hey, quick question from a relative newbie who is
| currently trying to solve this exact problem.
|
| Besides Celery, what are good options for handling long-
| running requests with Django?
|
| I see 3 options:
|
| - Use Celery or django Q to offload processing to worker
| nodes (how do you deliver results from the worker node
| back to the FE client?)
|
| - Use a library called django channels that I think
| supports all sorts of non-trivial use cases (jobs,
| websockets, long polling).
|
| - Convert sync Django to use ASGI and async views and run
| it using uvicorn. This option is super convoluted based
| on this talk [0], because you have to ensure all
| middleware supports ASGI, and because the ORM is sync-
| only, so seems like very easy to shoot yourself in the
| foot.
|
| The added complication, like I mentioned, is that my
| long-running requests need to return data back to the
| client in the browser. Not sure how to make it happen yet
| -- using a websocket connection, or long polling?
|
| Sorry I am ambushing you randomly in the comments like
| this, but it sounds like you know Django well so maybe
| you have some insights.
|
| ---
|
| [0] Async Django by Ivaylo Donchev
| https://www.youtube.com/watch?v=UJzjdJGS1BM
| roflyear wrote:
| Use anything except Celery, is my vote. Even if that
| "anything" is something you roll yourself.
|
| Celery is mature, but has bitten me more than anything
| else.
|
| For scheduling, there are many libraries, but it's good
| to keep this separate from Celery IMO.
|
| For background tasks, I think rolling your own solution
| (using a communication channel and method tailored to
| your needs) is the way to go. I really do at this point.
|
| It definitive is not using async, I think that will bite
| you and not be worth the effort.
|
| Huey is worth a look.
| Zetice wrote:
| Not only can you not beat Django's modeling in Python,
| but I simply cannot find an in-kind solution in JS
| either. Nothing covers all of the bases the way Django
| does, or more specifically DRF/graphene-django.
|
| The current state of the art is apparently Prisma, but it
| covers only a small part of the full picture.
| FredPret wrote:
| I'm a self-taught noob as far as web engineering goes, but
| between this course (cs50.harvard.edu/web/2020/) and
| ChatGPT, I got several commercial Django sites live in a
| very short amount of time and effort.
| SCUSKU wrote:
| I use django professionally and for personal projects. I
| started with Flask and then did FastAPI for a while, but I
| like django the most since it's the most mature.
|
| The ORM and django admin are killer features out of the box
| that the other frameworks don't have. I will say though
| that FastAPI is really nice, especially if you need async
| support. However, I have found that using django ninja [1]
| adds a lot of nice to haves that FastAPI has to django that
| makes it much more fun to use again.
|
| [1] https://django-ninja.rest-framework.com/
| SalmoShalazar wrote:
| +1 for django ninja. I built a little side project with
| it a few months back and was super impressed by how easy
| it was to get up and running. I didn't want to leave the
| Django ORM behind so was very pleased to find this
| project.
| gen220 wrote:
| I've used Flask/SQLAlchemy and Django a bunch in my day,
| professionally and on the side.
|
| It's a different strokes for different folks sort of deal.
|
| Django is very opinionated, in a way that is "eventually"
| correct (i.e. they might have had some bad opinions many
| years ago, but they have generally drifted in a better
| direction). If the opinions don't align with what you're
| building, the escape hatches are not generally well-
| documented or without penalty.
|
| Flask is minimalist and flexible. Once you find a "groove"
| to building with it that fits your sensibilities, it's
| quite, quite nice. That being said, the most recent
| versions of flask have excellent documentation, imo, and
| the tutorial is a bit more opinionated in a highly
| productive way. The "patterns" section of the docs is also
| super useful for productionizing your app.
|
| Personally, I prefer the combo of Flask+SQLAlchemy, and
| eventually Alembic once you decide that's good for your
| app's maturity level. I respect Django a lot, I just enjoy
| the explicitly "less magical" aspects of a Flask stack,
| which is an opinion-based trade-off imo.
| amethyst wrote:
| It's a heavily modified internal fork of Django. Source: I
| work on our Python Language Foundation team.
| KRAKRISMOTT wrote:
| Are you using the ORM/auth/admin features or is Django just
| a lightweight router? I can't imagine the ORM being too
| useful at Facebook's scale.
| amethyst wrote:
| I'm not familiar enough with Django to say for sure, but
| I assume at this point it's almost entirely custom ORM
| and libraries on top of async django routing/response.
| [deleted]
| PartiallyTyped wrote:
| What exactly do you do? This sounds like a
| JIT/Interpreter/Language dev job. Is this correct?
|
| If so, do you have any recommendations or suggestions for
| someone getting their feet wet on this?
| amethyst wrote:
| It's mostly foundational work around developer tooling
| and infrastructure that isn't already covered by other
| dedicated teams (eg, there is a dedicated Cinder team).
| My latest work has focused around formatting and linting,
| and includes open source work on usort, our import sorter
| [1], and fixit, our custom linter [2] that makes it easy
| for teams to build and deploy their own custom lint rules
| with autofixes.
|
| Some of my team mates work on driving internal adoption
| of Cinder, rolling out new versions of Python everywhere,
| improvements to our build and packaging systems,
| supporting other Python tooling teams, etc. There's a lot
| of cross-functional project work, and our primary goal is
| to improve the Python developer experience, both
| internally and externally wherever we can.
|
| 1: https://usort.rtfd.io
|
| 2: https://fixit.rtfd.io
| Myrmornis wrote:
| How heavily are type annotations and type checking used in the
| codebase?
| phyrex wrote:
| It's fully typed and nothing can be merged that doesn't pass
| the type checker
| wing-_-nuts wrote:
| What are the benefits of python with types vs a statically
| typed language like java, golang, etc?
| jajajajajajajaj wrote:
| The main "benefit" is that Instagram is written in Python
| and always has been.
|
| It's millions of lines of code, you can't just change it
| to be Java one day.
| echelon wrote:
| This would partially explain why it feels so slow. Some page
| refreshes are taking up to a couple of seconds.
|
| It's probably hurried along python code and fairly simple
| horizontal scaling. Some population of individual instances are
| probably pegged, and random requests are frequently landing on
| their backlogged request queues.
|
| This seems to have been rushed out the door to capture a unique
| market opportunity, and they'll clean up the engineering once
| they get engagement.
| seanp2k2 wrote:
| For the "Python isn't fast enough for production backend" crowd
| from the same company that brought you the largest social network
| built on PHP + MySQL.
| that_guy_iain wrote:
| For me, it does seem really strange since PHP (currently) is
| faster than Python and PHP wasn't fast enough for Facebook. But
| it's interesting to see how it'll play out.
| rafark wrote:
| It's obvious why they went with python. Since Instagram was
| written in python and Threads is based on instagram, they're
| probably using a lot of the libraries they already have for
| Instagram.
| that_guy_iain wrote:
| That makes some sense.
| DeathArrow wrote:
| Probably Python is deemed as more appealing to hipsters than
| good old PHP.
| wiseowise wrote:
| Python is older than PHP.
| paultopia wrote:
| OTOH threads is kinda slow as shit right now
| anayak wrote:
| While that is technically correct, they haven't been running
| vanilla PHP for quite some time - it's jitted into x86 and run
| natively on the machine. It's also pretty extensively
| optimized. And the VM that does the jitting is all C++. Their
| MySQL deployment is also an internal fork that's been heavily
| modified for scalability (storage/compute separation, sharding,
| RocksDB based storage engine instead of InnoDB, Raft for
| synchronous replication, etc.). Lots of really fantastic
| engineering work has gone into making their backend infra
| scale.
|
| Some worthwhile references:
|
| [1] https://engineering.fb.com/2016/08/31/core-data/myrocks-a-
| sp...
|
| [2] https://research.facebook.com/file/529018501538081/hhvm-
| jit-...
|
| [3]
| https://research.facebook.com/file/700800348487709/HHVM_ICPE...
| ericbarrett wrote:
| Facebook scaled to ~1b daily actives on MySQL + InnoDB. There
| was _lots_ of engineering work, like schema sharding
| (denormalization), automation, plenty of bug fixes and
| patches for MySQL (most or all contributed back to upstream,
| from what I remember), and of course a massive caching layer;
| plus throwing crazy hardware at the problem. Nonetheless the
| underlying engine was something any MySQL user or admin would
| have recognized. And we backed it all up, every day, in < 24
| hours, using an unmodified mysqldump. (FB MySQL team
| 2009-2012)
| 9dev wrote:
| > And we backed it all up, every day, in < 24 hours, using
| an unmodified mysqldump.
|
| But... how? Considering all the transactions in flight, and
| everything? And did you ever Test disaster recovery with
| that setup?
|
| I've worked on relatively big projects, but FAANG
| engineering is like some entirely different field of
| software engineering. Fascinating.
| grishka wrote:
| Curiously, VKontakte also started from PHP+MySQL but went
| another way. PHP is compiled ahead of time with something
| called KittenPHP. It's open-source. For databases they
| switched some parts to their own bespoke solutions in C that
| talk with PHP over the memcached protocol. These are called
| KittenDB (or "engines") for the simple single-purpose ones,
| and there was also a more generic MySQL replacement in
| development when I left in 2016, MeowDB.
| jonhohle wrote:
| I'm not sure why someone of facebook's scale would want JIT
| instead of AOT. That's a lot of servers all JITing the same
| thing that could be done once at build time.
| jeffffff wrote:
| JIT compilation has the opportunity to do profile-guided
| optimization at runtime. JIT compilation is also simpler
| when distributing an application to non-identical
| servers, as it can optimize for the exact hardware it is
| running on.
| yuliyp wrote:
| Facebook actually does some compile-time optimizations,
| but also ships some of the profiling data for JITing
| between early deploy phases and the rest of the fleet:
| https://engineering.fb.com/2021/03/03/developer-
| tools/hhvm-j...
|
| Also, Facebook used to do ahead of time compilation
| (HPHPc) but eventually HHVM managed to outperform it.
| woadwarrior01 wrote:
| HHVM's predecessor at FB was an AOT PHP to C++ transpiler
| called HPHPc. There are a lot more optimization
| opportunities with JIT compilation when dealing with
| dynamically typed languages like PHP.
| xuancanh wrote:
| It may not be because of Python, but Thread is definitely not
| ready for the current load. Few hours ago no one could access
| to Zuck's profile page, and he had to take it to private to
| hide the issue.
| ryantgtg wrote:
| I think there have been issues where profile pages say "this
| account is private and has 0 followers." I saw that yesterday
| on every account I clicked on (on web). And then a few
| minutes later it was back to normal. It might not be that he
| manually took it private to hide an issue.
| rafark wrote:
| Tweets load very slowly on my end. So slow I thought there
| was a js error on my end, you know, like when you get a never
| ending spinner animation because there was an unhandled
| exception thrown.
| Solvency wrote:
| Surely you mean threads.
| hello_moto wrote:
| Yep, Vanilla Python isn't fast enough so they have to do more:
|
| "It's running on Instagram's #Cinder fork that includes a JIT,
| lazy-loaded modules, pre-compiled static modules, and a bunch
| of other interesting tweaks against vanilla Python 3.10"
|
| From the Tweet.
|
| Personally, I don't care what language people use. We all know
| that if you have money, you can just throw in more servers.
| Until your CFO and CTO decided that the next big thing is to do
| "Cost Saving" (in the eye of recession).
|
| I'm surprised in 2023 people are still debating mainstream
| programming language "PROD" readiness.
|
| It's like a "cool kid" competition.
| ubercore wrote:
| > We all know that if you have money, you can just throw in
| more servers.
|
| Definitely not true at Facebook scale. You need to be smart,
| fast, _and_ throw more servers at it.
| chippiewill wrote:
| PHP is actually shockingly fast for a non-JIT'd language these
| days.
| icelancer wrote:
| PHP8 has a JIT now but I get what you mean.
| albertopv wrote:
| They didn't use vanilla Python, though.
| shortrounddev2 wrote:
| Quite often, the choice of language is failing to see the
| forest for the trees. Saving nanoseconds or cycles because you
| chose C++ over Python pales in comparison to milliseconds spent
| at network barriers reading from a cache service or database
| throwawaymaths wrote:
| At Facebook scale those nanoseconds are worth millions in
| compyte, energy, etc.
| yeck wrote:
| Perceivable latency isn't the only consideration. Depending
| on your business, compute for your proprietary workloads
| might be one of your biggest expenses. You could see an order
| of magnitude improvement to resource consumption depending on
| the type of workload and the language it is written in.
|
| Ultimately these these choices are all about trade-offs.
| Maybe python is fine for them, maybe they've built themselves
| into a corner. Time will tell.
| MengerSponge wrote:
| When they need a full re-write the specification will be
| easy: they've already got the pseudocode. (I kid because I
| love Python)
| IshKebab wrote:
| The difference between C++ and Python is not nanoseconds.
|
| And I have definitely seen projects fail due - in part - to
| language choice. Of course projects can succeed in almost any
| language but that doesn't mean the language choice is
| irrelevant.
| DeathArrow wrote:
| Python is just fine for all the 100 users Threads will manage
| to acquire. :)
| latexr wrote:
| > all the 100 users
|
| That's off by multiple orders of magnitude, if you believe
| their reports of 30 million sign ups in under a day.
|
| https://www.theverge.com/2023/7/6/23786108/threads-
| internal-...
| Twitterboi wrote:
| You mean sign-in's?
| latexr wrote:
| I'm using their terminology. The distinction seems
| irrelevant for the conversation.
| Klonoar wrote:
| In addition to the Facebook/PHP comments here, an often
| overlooked one is that Instagram was a big Python/Django shop
| for years (and probably still is in many parts).
| arcticbull wrote:
| Most of Instagram is Django.
| nikhizzle wrote:
| Earlyish facebook engineer here. Early days FB php was nothing
| like the php used to template websites. All kinds of
| specialized libraries to enable a much more sophisticated
| programming style. Think functional helpers, and asynchronous
| execution on thousands of cores, spanning trees across data
| centers using ssh etc. As a tangent a lot of the good stuff I
| used was written by Evan Priestley, who also did Phabricator
| and lots of other strong systems.
| iknowSFR wrote:
| Can you speak to the onboarding and learning curve in that
| environment? What was the training like? What kinds of
| timelines did you experience regarding expectations as a new
| hire?
|
| Asking because I'm curious about these early stage mega-tech
| companies that are working at a scale I can't fathom.
| nikhizzle wrote:
| Six week boot camp, you are pretty fluent by week 4.
| Notably though this was before mega tech days, we had under
| 100 engineers.
| supernikio2 wrote:
| I'm of course speaking from an outsider's point of view, but
| do you think PHP was a good option, or was it a "fitting a
| square peg into a round hole" situation?
| troupo wrote:
| > but do you think PHP was a good option, or was it a
| "fitting a square peg into a round hole" situation?
|
| Facebook was founded in _2004_. Whatever fancy tech you 're
| thinking that would fit the round hole, it didn't exist
| then. They also never thought it would serve a couple
| billion of users one day, so they built it in whatever they
| had and knew at the time.
|
| Same for Instagram and Python (they started with Django).
| zitterbewegung wrote:
| How they have used Instagram with Django is that they
| have replaced pieces of Django over time with custom
| parts which is a benefit of the design of Django.
| nikhizzle wrote:
| It was not a deliberate choice, just it was what the site
| was built in, so people like Evan and Marcel (Laverdet)
| made php beautiful for us.
| grumpyprole wrote:
| PHP didn't scale for the number of users or the number of
| developers. Presumably that's why they built their own PHP-like
| language called Hack with static types and a new runtime.
| gedy wrote:
| Would this pass their System Design interview?
| berkle4455 wrote:
| Except PHP actually is fast while Python isn't.
| ldng wrote:
| Except it's not. Bindings to C libraries are.
| pjmlp wrote:
| You mean the company that had to create their two PHP
| implementations and a translator to C++, to actually scale?
| DoesntMatter22 wrote:
| Lol. Do you think you are just going to use go and everything
| is going to scale to FB level size?
|
| Facebook was already at a scale larger than 99.99999 percent
| of sites before they had HHVM
| throwawaymaths wrote:
| Of course not. One uses Erlang and scales to Whatsapp
| scale.
| pjmlp wrote:
| Remember when Twitter was written in Ruby?
| ramesh31 wrote:
| >Remember when Twitter was written in Ruby?
|
| Ruby is the exception to the rule.
|
| Friends don't let friends start new Ruby projects in
| 2023.
| epolanski wrote:
| Ruby in 2023 is miles ahead it was 10 years ago.
| jarjoura wrote:
| LOL, at least Ruby inspired a new generation of languages
| that we have today.
| c7DJTLrn wrote:
| Guess I'll just write my next project in Bash then, after
| all, if it's a good idea they will come right? Execution can
| wait until you've got a few hundred mil.
| ramesh31 wrote:
| >Guess I'll just write my next project in Bash then, after
| all, if it's a good idea they will come right? Execution
| can wait until you've got a few hundred mil.
|
| Yea actually. Bash/CGI could handle plenty. If thats your
| most comfortable language then go for it.
| disgruntledphd2 wrote:
| They ended up compiling PHP to assembly! Facebook is a
| company that will go to ridiculous lengths to avoid rewrites.
| snapetom wrote:
| Was there a separate initiative for assembly, or are you
| thinking about HipHop? HipHop was for C++ and got a lot of
| attention back when it was released.
| pjmlp wrote:
| Yes, the 2nd rewrite for a VM, where the JIT compiler
| turned out to be quite good versus HipHop, and
| development focus switched to HHVM.
| disgruntledphd2 wrote:
| I was thinking about HHVM, which I believed was
| essentially assembly for a VM. Am I wrong on that?
| snapetom wrote:
| No, you're right. HHVM was the 2nd iteration. I was
| thinking of the first project, which was for C++. Looking
| at wikipedia, that was called HPHPc.
| nikanj wrote:
| Rewriting in hip-tech-du-jour is for startups with lots of
| funding and little of traction.
|
| Why would a massively successful company toss a stick in
| their own spokes by tearing the product to ground?
| nobleach wrote:
| But it's actually a _really bad_ strategy for a startup!
| It'll be realized the moment you try to hire for hip-
| tech-du-jour. Most likely, you don't even have the
| problem that hip-tech-du-jour claims to solve.
| jarjoura wrote:
| I know you're being hyperbolic, but during my time there,
| your impact was measured only on measurable things like
| performance gains but never on lines of code. So engineers
| would always go in the direction of whatever gets you the
| biggest win.
|
| Sometimes that actually meant rewriting the thing from
| scratch, and other times that meant adding something else
| as a middleware to prevent touching the original thing.
|
| For the most public success story, React was a complete
| rewrite of the thing before it, for example.
| disgruntledphd2 wrote:
| I spent five years there, and made that joke constantly.
|
| It's mostly hyperbole but there's definitely a kernel of
| truth to it. I generally dislike rewrites and I wonder if
| FBs approach influenced me more than I realized at the
| time.
| that_guy_iain wrote:
| I thought they tried it multiple times. The problem is they
| had so many developers writing new code any rewrite attempt
| would fall behind super quickly?
|
| So they created their own language Hack so allow a slow
| transfer from PHP to Hack. Which is basically the idea
| behind Kotlin.
| rafark wrote:
| Facebook had hundreds of millions daily active users before
| writing a compiler.
| pjmlp wrote:
| And lots of nodes not able to keep up, rewriting backend
| services in C++ to make up for it.
| coding123 wrote:
| 98% of the people that initially thought (Python for
| Threads - OMG performance!) work at a company that will
| tell you straight up they expect 100M RPS in a year or two,
| will actually only get to 50k RPS in 5 years before the
| "Our incredible journey" letter is issued.
| rewmie wrote:
| > Facebook had hundreds of millions daily active users
| before writing a compiler.
|
| In other words, Facebook felt compelled to write a compiler
| when they had hundreds of millions daily active users.
| depereo wrote:
| If you have hundreds of millions of daily active users
| then needing to write a compiler isn't the end of the
| world.
| steveBK123 wrote:
| They wrote a new app in the language 99.9% of 0-5 year
| experienced devs know.
|
| Absolutely shocking.
| ineedasername wrote:
| Pfft. It would be 100x more performant if they'd written it in
| asm.
| baq wrote:
| They also wouldn't release this year, perhaps neither next
| babuloseo wrote:
| Never forget https://github.com/twitter/the-algorithm/pull/447
| aio2 wrote:
| Finally, a difference between the two.
| nunez wrote:
| Can we block Twitter links on HN?
|
| No point in supporting that user-hostile tire fire.
|
| This could have been on Mastodon.
| ambivalence wrote:
| It is on Mastodon as well, actually:
| https://mastodon.social/@ambv/110665808046065754
| philipov wrote:
| Could we get the mastodon link as the main link, please? Can
| actually view that one.
| aeyes wrote:
| Is the Twitter link really not working for you at the
| moment? I can open it without having an account.
| rsynnott wrote:
| You can't see replies on Twitter links anymore, though
| that doesn't really matter in this particular case.
|
| Of course, who knows what it'll be doing tomorrow. I'd
| definitely think it's probably time for HN to prefer
| Mastodon (or Threads, I suppose) links, where a post is
| available on multiple microblogging things; Twitter has
| had any one of about five behaviours for non-logged-in
| linked tweets (work properly, work but without replies,
| spin forever, redirect to login, generic error page) over
| the last week, depending on when you try.
| reledi wrote:
| The tweet is currently viewable while logged out. Have you
| tried?
| timeon wrote:
| Only thing I see is: "Something went wrong. Try
| reloading."
|
| (Mastodon link works.)
| xk_id wrote:
| The links are being submitted and voted on by other members of
| the community. I'm not sure what gives you the impression
| you're in a position of authority.
| _Algernon_ wrote:
| Asking a question is having the impression that one is in a
| position of authority now?
| the-dude wrote:
| No, but appealing to 'we' is. Not that 'we' in this case
| has much meaning, it actually could read 'you'.
| johnfn wrote:
| What other pronoun do you want OP to use? "Can I ban..."
| - well, obviously not. "Can you ban..." who is "you" in
| that case?
| the-dude wrote:
| > "Can I ban..." - well, obviously not
|
| Of course he can. And he already does I assume,
| considering his other remarks in the comment. So
| including him in the 'we' does not really make sense ...
| if you consider it a question. Which it is not.
|
| > "Can you ban..." who is "you" in that case?
|
| That would be us dude, but not him as there is no reason
| to ask him, his position is clear.
|
| So he is telling us what to do, dressed up as a question.
| [deleted]
| mr90210 wrote:
| Exactly my thoughts. Seems like asking genuine questions is
| not acceptable anymore because some people can't help but
| to resort to personal attacks. Unbelievable.
| the-dude wrote:
| This is not a genuine question. This reads like 'Darling,
| could you please put the garbage out?' is a question.
|
| It is not a question.
| mr90210 wrote:
| Oh lord, the person asked a question, A QUESTION!!!!
|
| Geez, it is you who is the authoritarian for coming so hard
| at the person who asked a pertinent question.
|
| Geez.
| redleader55 wrote:
| I think the cautionary tale here is not to do what Twitter did:
| prevent links to competing products, blocking open access, etc.
|
| Don't you find it ironic that you are proposing we do on HN
| exactly what Twitter did, which is probably why you suggest not
| linking to Twitter?
| czardoz wrote:
| I worked on Cinder, and on the web server. Happy to answer
| technical questions if any :)
| vmsp wrote:
| Can you expand on the web server? Does it implement WSGI?
| czardoz wrote:
| It has some elements of WSGI, but has expanded into its own
| thing which supports asyncio (this happened before ASGI was a
| thing).
| ntonozzi wrote:
| What are the most impactful and likely features that you'd like
| to see merged into CPython?
| czardoz wrote:
| I'd love to see lazy imports become part of the upstream at
| some point.
| feydaykyn wrote:
| Hi,
|
| Would you recommend using Cinder for stack made of - Django -
| Cython - Numpy
|
| Thanks!
| czardoz wrote:
| If this stack is heavily skewed towards numerical
| computations, Cinder might require a lot of tuning to be
| effective.
| whalesalad wrote:
| Who would benefit from Cinder? What kinds of workloads? The
| README is kinda lacking on this.
| czardoz wrote:
| Cinder's feature set is highly optimized for IO bound web
| services that run under a forked-worker model.
|
| For example: you start a main process, warm it up with a few
| requests, run the JIT compiler and then fork off worker
| processes to handle the main chunk of traffic.
|
| As of now, it requires hand-tuning to get the best possible
| performance.
|
| In terms of use cases, Cinder does the best when faced with
| "business logic" code (lots of inheritance, attribute
| lookups, method calls, etc). It can speed up numerical
| computations too, but you're probably better off using a
| library if that's the majority of the workload.
| zengid wrote:
| Ok but is the front-end in React Native?
| akrymski wrote:
| I'd suspect so, but would love to see that confirmed.
| blkhp19 wrote:
| Native
| https://twitter.com/vinaygaba/status/1676968580712366080?s=6...
| [deleted]
| costco wrote:
| There's definitely some overlap in the backends because
| https://www.threads.net/t/C is the first Instagram post but on
| Threads with the exact same amount of likes it has on Instagram
| yet https://www.threads.net/@kevin shows no threads.
| ddorian43 wrote:
| No tweaks mentioned at all inside. The title is wrong. Kinda
| weird to start a new project that needs to scale with Python
| though (I use Python most of the time).
| kleinsch wrote:
| IG is built in Python (which is why this is also) and it seems
| to scale fine.
| [deleted]
| threeseed wrote:
| "JIT, lazy-loaded modules, pre-compiled static modules, and a
| bunch of other interesting tweaks against vanilla Python 3.10"
| ddorian43 wrote:
| That is just CPython fork from instagram
| https://github.com/facebookincubator/cinder/
| ambivalence wrote:
| The fork is where the tweaks live.
| jackmott wrote:
| [dead]
| troupo wrote:
| It seems they are re-purposing Instagram's backend for this.
| And Instagram's backend is Python
| super_flanker wrote:
| As far as I can tell from my prior experience of working at
| Insta is that all the work is done by various services written
| in C++/hack. Python/Cinder would be just interface layer
| routing the incoming request/response from/to client.
| threeseed wrote:
| Would love to know more details about the rollout.
|
| Because it's adding well over a million new users each hour which
| is pretty staggering.
| phyrex wrote:
| It's just running on the shared meta infrastructure which is
| built to handle that scale
| zokier wrote:
| I would assume its massively overprovisioned at this point to
| cover first few weeks if not months of projected growth.
|
| But still there probably is all sorts of juicy details that
| would be neat to hear about
| sanderjd wrote:
| One of the "fun" things when I worked at a big company was
| thinking through how to make the very first launched version
| of something work at millions-of-users scale. There is no
| organic growth in traffic from zero when a non-negligible
| subset of billions of active users will try it out right
| away.
|
| But I think on net it was still an easier problem than
| scaling a startup product from zero, because the tooling and
| processes to support huge usage is already so mature at these
| big companies.
| pavlov wrote:
| Meta has 3.8 billion monthly active users across its products.
|
| A few million new people is less than 0.1% of the total. It's
| entirely within expected variance.
| nomel wrote:
| I think they're looking at it from a an individual product
| perspective. % users of an entire company can be extremely
| different than % growth of a new product, within that
| company.
|
| My naive assumption is that they have some neat simulated
| user/load system to help identify any scaling issues that
| might arise when launching a service, and the extreme %
| growth. That 3.8 billion was spread over 15 years.
| mattfrommars wrote:
| Makes me wonder why did they not opt for Java or C# as their
| backend choice. I better stop learning C# and move on.
| manca wrote:
| I've been using Threads since late yesterday and it's been very
| slow... sometimes it wouldn't even load content when you browse
| specific profiles (especially the active ones with tens of
| thousands of followers). So I am not surprised the backend is in
| Python :)
| mlrpfx wrote:
| Always the same people that get their names in the news.
|
| I would say that the amount of tweaks that prop up Python is no
| cause for celebration.
| blitzar wrote:
| Yet here it is ... powering a global scale app.
|
| Too weak for someones hobby project, strong enough for global
| scale.
| mlrtph100 wrote:
| So Google and Dropbox, who switched to Golang, are hobby
| projects.
| cmrdporcupine wrote:
| Choice of language is not what makes things succeed or fail
| in production.
|
| Team best practices, enforcement of coding style and
| technique, good project management, infrastructure, and
| team cohesion etc.
|
| And weeding out bad engineers who think that "switching to
| a new [language|framework|religion]" will solve all
| problems.
|
| 10+ years ago Facebook itself did a remarkable job of
| scaling up, with hardly any outages, a PHP LAMP stack
| thingy to one of the hugest traffic websites in the world.
| Meanwhile back then Twitter mucked around with every novel
| technology they could find or invent, and had constant
| outages.
|
| Also Google didn't "switch to Golang." There's plenty of
| C++, Java, Python there. I worked there for 10 years and
| encountered only a handful of Go projects. Lots of cloud
| services in C++.
|
| Would I _personally_ choose Python for a project? No, I don
| 't like it, and I work in Rust full time because I prefer
| it. But if I worked at Meta / Instagram and had a team of
| Python engineers and existing libraries & infrastructure,
| this would be the right approach.
| aeyes wrote:
| Also, what's really different from Instagram? You change
| the picture for a short text and many other features
| already existed - comments, likes, subscriptions,
| recommendations, personal messages...
|
| I think this is just an Instagram skin with some feature
| flags.
| n_ary wrote:
| Off-topic: it is not yet [global scale] because it is only
| available in US. Here in EU, I still get "not available in
| your country".
| blitzar wrote:
| It is only unavailable in the EU.
|
| Something, something privacy regulations.
| reaperducer wrote:
| _it is not yet [global scale] because it is only available
| in US. Here in EU, I still get "not available in your
| country"._
|
| It's not only available in the U.S. It's available in 100
| countries. There's more to "global" than the E.U.
| superq wrote:
| Throw a trillion cores at anything and it can be global
| scale.
| Yoric wrote:
| My few years of personal experience of using Python with even
| small-ish scale professional projects have been...
| underwhelming so far. I wonder how Meta (and Google, and all
| the others who write serious Python code) manage to keep
| things running - and I'm not talking (only) of performance,
| but largely of bugs.
|
| My personal suspicion is that the secret sauce is simply to
| have a sufficient budget to hire an army of code quality
| engineers. I could be wrong, though.
| hospitalJail wrote:
| >My few years of personal experience of using Python with
| even small-ish scale professional projects have been...
| underwhelming so far
|
| You have years of experience but couldn't point to anything
| in particular?
|
| Heck, I like python and I can complain about dynamic typing
| issues in for loops, or that they are adding features like
| generators/decorators that make code more difficult to
| understand, which goes against the zen of python.
|
| (But I still think python is great)
| Yoric wrote:
| Why would I need to point to anything in particular? I
| think that all Python developers know them already.
| o1y32 wrote:
| I cannot find any substance in your comment. At minimum it
| should provide some rough examples instead of simply
| "underwhelming".
| Yoric wrote:
| Well, I wasn't planning to spend time criticizing Python.
| I think that we all know roughly its pros and cons, no
| need to add fire.
|
| My questions remain, though.
| marcinzm wrote:
| You can write crap code in any language. To avoid it you
| need to enforce some decent set of best practices for that
| specific language. That also means not using best practices
| from language X in language Y and then complaining that
| language Y is crap.
| Yoric wrote:
| Yes. And one way to rephrase my question is: which best
| practices work for these Python mega-projects?
| ambivalence wrote:
| In short: static analysis, unit tests, staging
| environments, effective log aggregation.
|
| The first one is a big productivity booster as it shows you
| bugs before you commit them.
|
| Here's a post about this, not exactly new but still
| describing the general principles very well:
| https://instagram-engineering.com/static-analysis-at-
| scale-a...
| VWWHFSfQ wrote:
| I would say that those things don't really contribute to
| the actual runtime scalability of the application. Those
| things ensure that the code is at least correct for what
| it's supposed to do, not necessarily whether it's
| performant. Even the most perfectly-written Python code
| will still be dog-slow and require an enormous budget to
| actually run at scale. It's obviously fine to do that and
| there are many success stories, but all of them required
| massive infrastructure investment.
| ambivalence wrote:
| > Even the most perfectly-written Python code will still
| be dog-slow and require an enormous budget to actually
| run at scale.
|
| Nah, asyncio on uvloop is plenty fast.
| Yoric wrote:
| For context, I was asking about scaling code complexity,
| not scaling performance.
|
| That being said, my personal experience suggests that
| both are really hard in Python, and not independent.
| Yoric wrote:
| Thanks. For context, I happen to have a PhD in static
| analysis, so I'm not entirely stranger to the general
| idea :)
|
| Sadly, all the attempts I've had with static analysis in
| Python screamed that while the language and tools make a
| valiant effort at supporting a reasonable set of
| annotations, 8-9 years after PEP 484, the libraries are
| simply not yet ready for it (not even, in many cases, the
| standard library).
|
| Unit tests, staging environment, effective log
| aggregation are all important tools, of course.
| [deleted]
| code_runner wrote:
| Tests is likely a large part of it. There are some great
| tools for code formatting, quality, and type checking. It's
| a very flexible language...
|
| When people are hyper critical of Python I feel it's
| typically they haven't seen real professional Python before
| and just throw scripts around... if you work with some
| hardened Python pros you pick up the tricks really quickly
| and it's very enlightening.
|
| I've always struggled to find all of those tricks in one
| online resource personally.
| epgui wrote:
| Every language will have its gurus and its own peculiar
| with-the-grain approach, but the python language has a
| ton of non-essential complexity baked into its very
| design/core (especially for a language that people seem
| to think is "simple"-- it's not, not at all; it's a
| language full of "pragmatic compromises", apparently by
| intentional philosophy), which places fundamental limits
| on how safe your application can be.
| smrtinsert wrote:
| The secret sauce is Intellij and some community knowledge.
| There's a library for everything and a great IDE makes it
| pretty painless and fun. Not sure I'd always jump for it
| thanks to the GIL but I wouldn't mind a job where it's my
| primary instead of my secondary or third language.
|
| The biggest pain with it is of course refactoring. It's
| tedious, but I think still comes out ahead in terms of
| productivity for high level stuff.
| nobleach wrote:
| But this is what FB/Meta are known for. They managed to scale
| PHP for years. They did fork it into Hack. They do have their
| own extensions. But they've put a ton of time and engineering
| effort into it. They didn't do it because they thought PHP
| was a great language. It was because they already had a
| massive infrastructure and tons of engineers that were
| already productive. Doing the math they realized it'd be
| cheaper to keep using PHP.
|
| The same is true here. They spent so much money getting
| Instagram to scale. They created Cinder to try and fold their
| hacks/tweaks back into Python. Spinning up some
| Rust/Java/Nim/Zig/Pony/Whatever stack sounds like it'd be
| fun... but spinning up a stack with which you're already VERY
| familiar, sounds like a money-maker.
|
| I'm totally agreeing btw, I realize this came off like it
| might be a counterpoint.
| Solvency wrote:
| Why has everyone on here been comparing Threads to Twitter when
| threads.com _literally_ says its a Slack alternative for
| _makers_? The product has absolutely nothing to do with a Twitter
| style social network.
| PartiallyTyped wrote:
| Threads.com is not Threads.net. The latter owned by
| Instagram/Meta.
| Solvency wrote:
| Jesus Christ this is confusing.
| nologic01 wrote:
| No matter what the true combination of reasons, this is a massive
| endorsement of the python ecosystem.
|
| Its an interesting dynamic: popularity draws attention, attention
| helps adress pain points, this makes the platform more attractive
| and the popularity increases...
| shaftoe444 wrote:
| > It's running on Instagram's #Cinder fork that includes a JIT,
| lazy-loaded modules, pre-compiled static modules, and a bunch of
| other interesting tweaks against vanilla Python 3.10.
|
| So not entirely just python 3.1.
| roflyear wrote:
| Python is a spec not an implementation, for what it is worth.
| zitterbewegung wrote:
| I have ported code from Cinder to CPython . The fork has some
| optimizations that can be easily put in CPython and Facebook is
| open to port features. I'm not sure if Facebook wants to
| continually have a fork but CPython is open to have those
| features merged in if they make sense.
| amethyst wrote:
| The Cinder team's longterm goal is to upstream as much as we
| can, and make the rest available as pip-installable
| extensions that anyone could theoretically install and use on
| CPython. The fewer internal changes and patches we need to
| maintain, the faster we can adopt upstream Python releases
| and all of the associated performance and tooling wins.
| andrewstuart wrote:
| If you know you've got gigantic scale why would you not use a
| compiled language?
|
| Surely it makes massive difference to hardware utilization?
|
| I like Python and javascript but there's perfectly fine tools
| around now for building systems that allow one server to do a lot
| more.
|
| They say you should not prematurely optimize. Equally you should
| not be prematurely unoptimised.
| paulddraper wrote:
| Using a compiled languages for web stuff is like using a
| Ferrari for a commute.
|
| It's better, but not the limiting factor.
| andrewstuart wrote:
| Threads is intended to be massive scale.
|
| I'd use the analogy of a Japanese bullet train as the
| compiled language and a diesel train as Python.
|
| Python is my primary language along with JS but if I was
| building a new service for Meta I wouldn't use either at the
| back end.
| callamdelaney wrote:
| Because Zuck probably told his team to make this a few weeks
| ago to annoy Musk. Weird ego thing going on.
|
| Python is likely faster to push out an app like this than say
| rust or even go.
|
| Also why spend time optimising something that will be dead in a
| few months?
| [deleted]
| smrtinsert wrote:
| !remindme 2 months. After 30 million signups in a day I'm
| pretty sure we just saw the birth of a new Meta social
| network.
| jajajajajajajaj wrote:
| If you want the real answer, it's because Instagram was
| acquired and the startup used Django. That's it.
| oaiey wrote:
| Exactly. This is all over engineered thinking. They probably
| pulled the core Instagram engineering team and gave them a
| new task. They used what they had.
| TheAdamist wrote:
| Ironic that a post about Threads backend is hosted on twitter.
| [deleted]
| HarHarVeryFunny wrote:
| Is there a web-based version of threads at the moment, or just
| the app ?
| [deleted]
| ambivalence wrote:
| If you want real irony here, it's this: powering an app called
| Threads with Python that's got a global interpreter lock which
| makes threading hard.
|
| Get it? Threads, threading...? OK, I'll see myself out.
| hynek wrote:
| I guess PEP 703 has to be auto-accepted now. I don't make the
| rules.
| nerdponx wrote:
| Meta does have their own higher-performance JIT-compiled
| implementation of Python, but I don't know if it still uses a
| global interpreter lock.
| sho_hn wrote:
| ... wait, what? Did this morph out of HHVM? Any links on
| this?
| rat9988 wrote:
| Click on the link, it's linked inside.
| timeon wrote:
| Seems to be behind login.
| nerdponx wrote:
| https://github.com/facebookincubator/cinder
| schnable wrote:
| Now that is Meta!
| [deleted]
| pookha wrote:
| Gotten burned by that. Have a python utility that calls
| something that's very CPU intensive (GDAL) and I literally
| have to wrap this python process in java for real
| concurrency. Not a python expert by any means but I'd love to
| know what other people do when this happens. I'm assuming
| it's something revolving around containers and messaging
| (blech)...
| antod wrote:
| I thought Python calling a native lib was the one time you
| could multithread your way around the GIL if you were cpu
| limited? Are the GDAL bindings not releasing the lock or
| something?
| ska wrote:
| Is your problem that the c part of GDAL isn't concurrent
| and you are calling it multiple times, or are you blocking
| on something you shouldn't?
|
| Python calling a concurrent c thing is super common. For
| more complex things there is multiprocessing, but that
| essentially is simplification of IPC.
| pookha wrote:
| Had a massive amount of images that I was trying to
| process for a specific purpose. I wound up spinning up
| multiple python processes within a java wrapper using
| concurrent linked queues. Got the job done but it's not
| the proudest moment of my life.
| seunosewa wrote:
| Why did you need the Java wrapper? You can orchestrate
| stuff like that within Python. The multiprocessing
| module.
| giantrobot wrote:
| You could have done this with the multiprocessing module.
| It interfaces like the threading library but it's
| launching new processes for each "thread". I've done a
| shitload of image processing in Python with it. You'll
| end up with a 100% Python version of what you did.
| mihaic wrote:
| Why downvote this? It's a fair point.
| RamblingCTO wrote:
| Because it's not ironic. It's obvious. Twitter is just a
| medium to post stuff. Threads is as well. I can post this on
| mastodon even. Also: Europe still can't join in on the fun.
| Dah00n wrote:
| EU != Europe. Biggest country in Europe is Russia. Biggest
| population is Russians.
| ffhhttt wrote:
| Just like America is not just the US. But nobody cares...
| user_named wrote:
| I'm in Europe and just downloaded the apk. All my Chinese
| friends have joined too.
| ambivalence wrote:
| A fair point about what? Threads is out for less than a day,
| I have 8 followers there so far. I originally posted this
| information on Mastodon, actually:
| https://mastodon.social/@ambv/110665808046065754
|
| However, I have a much larger following on Twitter, so it was
| the tweet that got picked up by someone here on HN.
|
| We'll see how this looks like in twelve months.
| sanderjd wrote:
| A fair point that it's stupid to implement a mostly plain
| text publishing site that doesn't support linking to any of
| the text published there.
|
| Edit: Someone kindly pointed out that you can see the
| content in the browser without an account from following a
| direct link. (You just can't have the app installed or it
| will open that and drop you on the login page.) This makes
| me a lot happier with the product!
| gochi wrote:
| You can link to it, what are you talking about? An
| example fetched from a verge article ->
| https://www.threads.net/t/CuVaOg1PgU4/
| sanderjd wrote:
| Oh nice! Thanks!
|
| I hadn't seen one of these deep links before I installed
| the app. After I installed the app they just opened the
| app but did nothing because I don't have an account. But
| I just uninstalled the app and now I see. This makes me a
| lot more positive on this product!
| Descon wrote:
| That link doesn't work for me - just get a splash page
| amadeuspagel wrote:
| They don't even have the dotcom?
| mihaic wrote:
| This is the first time I'm seeing this. I'm surprised
| they have a .net domain and that the design is so
| terrible.
|
| They just extracted Instagram comments and threw them in
| a sea of whitespace. No thought is given to anchoring
| content by importance or context, nor is there any
| esthetic apeal to it.
| rsynnott wrote:
| Linking to it works better than linking to Twitter, in
| fact, as of this week (Twitter no longer shows replies
| unless you're logged in...)
| sanderjd wrote:
| Twitter no longer shows _anything_ unless you 're logged
| in. Everything just redirects to a login page.
| RoyGBivCap wrote:
| [dead]
| sillysaurusx wrote:
| You can't link to threads? By design?
|
| I was seriously considering trying it out, but if there's
| no search engine visibility, there's no point. Most of
| the searches for my real name pull up tweets.
|
| Hopefully it's just a missing feature that will be added
| later. Even TikToks are linkable, and those have very
| little google visibility.
| cma wrote:
| I think Facebook/Instagram pioneered login walls for
| public post content.
| ipaddr wrote:
| Pay walls have been around longer than facebook
| sanderjd wrote:
| It seems like you can generate links, but they don't do
| anything for people who don't have the app (except direct
| you to install the app). I haven't fiddled with it to see
| if they've implemented something to make the content
| visible to search engines. (Which presumably would mean I
| could see it too with some effort.) But I haven't seen
| such a thing yet.
|
| Edit: Someone kindly pointed out that you can see the
| content in the browser without an account from following
| a direct link. (You just can't have the app installed or
| it will open that and drop you on the login page.) This
| makes me a lot happier with the product!
| sillysaurusx wrote:
| At least there are links. That's a good sign. TikTok
| survived on mostly that model, though it's still
| important to make the content visible without installing
| an app.
| sanderjd wrote:
| Check my edit: It turns out this does work for deep links
| to threads and user profiles (the homepage is still
| entirely useless) as long as you _don 't_ have the app
| installed. Definitely much better than I thought.
| cbsks wrote:
| I am able to view links to Threads from a web browser
| without being logged in:
| https://www.threads.net/t/CuVaOg1PgU4/
| Asooka wrote:
| Threads is a mobile-only app, like Instagram. You can't
| actually use Threads on a desktop browser, so that's probably
| why this is shared as a Twitter post. Even if it were a Threads
| post, it would not be submittable to HN.
| sanderjd wrote:
| Right. But this demonstrates why that's dumb. People can't
| even share press about their app without using a different
| product.
|
| Edit: Someone kindly pointed out that you can see the content
| in the browser without an account from following a direct
| link. (You just can't have the app installed or it will open
| that and drop you on the login page.) This makes me a lot
| happier with the product!
| eatonphil wrote:
| Not exactly true. They have a read-only browser view.
|
| In fact a Thread thread has already been on the front of HN.
|
| https://news.ycombinator.com/item?id=36609632
| superq wrote:
| > Not exactly true.
|
| Still, _mostly_ true.. read-only is just dumb. Nice to be
| back in 2008, when mobile-first turned into mobile-only.
|
| The entire point of this thread was that this is a
| Python/Django web app, but it's not actually a web _app_ at
| all, so what 's the point of pointing out any of this?
| [deleted]
| HarHarVeryFunny wrote:
| I see these https://www.threads.net/t/xxx links to specific
| threads, but is there any top level page where you can
| search for stuff or browse ? https://www.threads.net just
| has a QR code (to install the app presumably).
| eatonphil wrote:
| You can get to users profiles too and view their posts:
| https://www.threads.net/@zuck.
|
| But there's no broader browser yet it seems.
| SECProto wrote:
| > Threads is a mobile-only app, like Instagram
|
| Instagram can be used in a desktop browser just fine.
| hotpotamus wrote:
| Let's imagine that I don't have an Instagram account
| because I don't. (I never really understood it because it
| just felt like a slightly differently flavored Facebook and
| I already had a Facebook account.) Up until this weekend, I
| could simply view a Twitter post regardless of whether I
| had an account. And while I've never been much of a Twitter
| user, I did find that useful as a lot of local news and
| weather type stuff would get posted and I could easily view
| them. Is there any analogy to that with Threads?
| Grazester wrote:
| You can view single IG posts without an account.
| uxp100 wrote:
| You could view a tweet if linked to it, but it would
| limit your ability to view a feed or navigate elsewhere.
| Which is similar to instagram, but insta is more
| aggressive with stopping you, whereas twitter gave a
| slightly larger grace period.
|
| Twitter removed that behavior under Elon, then developed
| something even more restrictive. Great.
|
| Neither are good, and unfortunate that tumblr has
| recently ish copied this (for default css at least). Real
| shithead behavior.
|
| Unfortunately, Facebook looks the best out of all of
| them. If a restaurant posts their specials on Facebook, I
| can see them. Basically not true for other social sites.
| lloeki wrote:
| It also launched "in more than 100 countries" but none of the
| the E.U. ones.
|
| So until it either solves its privacy issues of becomes part
| of the fediverse, audience is crippled.
| snarkyturtle wrote:
| Chill it's been in development for like 6 months and they
| wanted it to scale to 10M users. I'm sure a web interface is
| on the roadmap.
| superq wrote:
| Six months and it still doesn't have a basic and open web
| interface. Not impressive. Many apps, including HN, operate
| for years with only a web interface.
| tredre3 wrote:
| Instagram launched in 2010 (but really gained traction only
| in 2011). It has been available via the web since 2012 but it
| took until 2015 for the web version to reach feature parity
| with the app version (namely posting and discovery).
|
| Based on that, I think it's safe to assume that a full
| featured web version of Threads should be up within a year.
| Threads already has some web presence (you can view threads
| and profiles).
| mrits wrote:
| Ironic people on MySpace talked about Facebook
| smrtinsert wrote:
| Or, "Threads" and the GIL maybe? Love Python though.
| melling wrote:
| Is the person from Meta? Doesn't appear so.
|
| It's still perfectly acceptable for Twitter users to discus
| other platforms on Twitter.
| ctur wrote:
| He was and worked on, among other things, Python while there.
| He's a trustworthy source.
| melling wrote:
| So, no he doesn't work at Meta. We could have stopped this
| thread an hour ago.
|
| We need a social network that removes the noise from the
| conversation
| no_wizard wrote:
| Their web app is not there. I think you can directly link to a
| Thread but it only shows what you linked to, not even the
| discussion tree or anything like that.
|
| Its not open to the web yet. Supposedly, it will be
| KolmogorovComp wrote:
| I will probably be downvoted for this joke but I cannot resist. I
| hope they use Threads over async/await.
| beardedman wrote:
| So glad we're condensing entire tech stacks down to a language
| now.
___________________________________________________________________
(page generated 2023-07-06 23:01 UTC)