[HN Gopher] Backend of Meta Threads is built with Python 3.10
       ___________________________________________________________________
        
       Backend of Meta Threads is built with Python 3.10
        
       Author : BerislavLopac
       Score  : 339 points
       Date   : 2023-07-06 08:17 UTC (14 hours ago)
        
 (HTM) web link (twitter.com)
 (TXT) w3m dump (twitter.com)
        
       | lopkeny12ko wrote:
       | Python is just the web server. All the _truly_ performance-
       | sensitive components--edge proxy, load balancers, backend
       | services, databases, caches, and storage services--are mostly
       | C++.
        
         | globular-toast wrote:
         | Which is exactly the way Python is meant to be used and has
         | always been used.
        
         | citrin_ru wrote:
         | I would argue a web server which processes all (or most)
         | requests is a performance sensitive part of the system.
         | Straightforward rewriting from Python to Go in many cases gives
         | at least 2x-3x performance improvement which on FAANG scale can
         | mean huge difference in hardware cost.
        
           | brrrrrm wrote:
           | Developer velocity is extremely important at FAANG scale.
        
             | that_guy_iain wrote:
             | Is it really? Don't they work really slowly compared to
             | most of the industry? I really get the impression that it's
             | slow progress for things at FAANG.
        
               | super_flanker wrote:
               | Not really, especially Facebook (Meta) moves really fast.
               | They have a slogan "Move fast, break things", okay not
               | "Break things" anymore. But you get the gist, in my
               | experience working at almost all of FAANG, Meta and
               | Netflix move the fastest.
        
               | PartiallyTyped wrote:
               | I think AWS must be the slowest of the bunch tbh. Too
               | many meetings.
        
               | super_flanker wrote:
               | I agree, but Google is the slowest.
        
               | ItsMeYourUncle wrote:
               | Slow compared to startups, very fast compared to
               | similarly sized companies, even other tech companies.
        
           | Tade0 wrote:
           | It's probably cached to seven hells, which makes that 2-3x
           | improvement much less relevant.
           | 
           | Wordpress wouldn't be this successful if this approach didn't
           | work. After all, without caching it will happily take over 20
           | seconds to render the main page.
        
       | mrcwinn wrote:
       | I'm building a tiny little product for trail runners and cyclists
       | in Python 3.10 (Flask) at https://www.outandback.com. Python has
       | been a lot of fun to pick up (I rely heavily on pandas and some
       | ML libraries so it felt like the right stack).
        
         | justsocrateasin wrote:
         | Could you describe your tech stack more?
         | 
         | I'm a data engineer trying to teach myself React so I can build
         | a full stack app using some APIs and it's certainly a good
         | exercise, but I could also certainly build that in Python. Your
         | app looks great and I'd like to explore that more!
         | 
         | P.S - fellow runner and cyclist and out and I'm checking out
         | the app more now.
        
       | kleton wrote:
       | Really, all the performance intensive parts are in various c++
       | aggregator and recommendation type services. But the webserver is
       | Django, yes.
        
         | fleetfox wrote:
         | Django is WSGI/ASGI framework not a webserver. What do they
         | actually use to terminate HTTP?
        
           | freeplay wrote:
           | Most likely a reverse proxy of some sort.
        
         | [deleted]
        
         | taopai wrote:
         | Thanks for your comment. I'm really interested in this topic.
         | How do you know the web server is Django? I searched but
         | couldn't find this.
         | 
         | Why would they use Django? I did some small projects in it but
         | I assumed it wasn't very fast and wouldn't be suitable for a
         | big app like this. I would like to know the pros and cons. Why
         | didn't they build up something in C++ or Rust? Won't python
         | limit the speed of responses despite being build the hard stuff
         | in compiled languages? Sorry for being so naive, I am an
         | amateur.
        
           | dporter wrote:
           | At least a few years ago, most of Instagram's server side
           | code was in Python. Python may be slower than C++, but it's
           | not about raw speed it's about being "fast enough."
        
           | tinco wrote:
           | They deemed getting to the market fast was more important
           | than hardware costs. If their architecture is good, then they
           | were probably really quick and flexible using Python to glue
           | their architecture together which would have been the
           | development bottle neck to getting the service up and
           | running. All the components of the service can be done by
           | separate teams in whatever language they deem most effective.
        
           | reissbaker wrote:
           | I worked at IG on a previous iteration of Threads, with some
           | of the engineers who wrote the current Threads app. It's
           | Django!
           | 
           | (Heavily modded, run on a custom Python JIT, and using an
           | extremely custom FB-developed database, also used for IG and
           | FB.)
           | 
           | It's Django because IG was originally written in Django back
           | in the day. FB's general approach to scaling is keep the
           | interface generally similar, and slowly replace things behind
           | the scenes as necessary -- rather than doing big rewrites.
           | It's seemed to work pretty well.
           | 
           | Ultimately the language used for the web server isn't a huge
           | deal compared to the performance of the database, for these
           | kinds of massive social apps. Plus, well, they do have the
           | custom JIT -- but that's _very_ new, and when I first joined
           | IG in 2019, we were running vanilla Python in production.
        
             | taopai wrote:
             | Thank you, awesome answer! HN comunity is awesome.
             | 
             | Thanks also to every other answer I received for my
             | question, I appreciated all of them.
        
           | DarkNova6 wrote:
           | Most of Facebook is built on PHP. I'm surprised they didn't
           | choose Laravel.
        
             | yurishimo wrote:
             | Laravel wasn't created until 2010/11~
        
             | cptcobalt wrote:
             | Laravel is not a requirement to use PHP effectively.
        
               | DarkNova6 wrote:
               | Oh, I know. But as a joke I chose the most well known
               | framework.
               | 
               | Personally evren I prefer php over python.
        
             | sangnoir wrote:
             | It turns out calling the app "Threads _by Instagram "_ is
             | not just a branding gimmick. The Instagram backend was
             | always Python/Django since before the acquisition.
        
               | mrweasel wrote:
               | That's pretty interesting, I wonder if that means that
               | it's build by a team at Instagram, and not Facebook. I'd
               | assume so.
               | 
               | I get that Facebook is still a huge success, but I do
               | find it telling that they opted to put Threads under
               | Instagram, rather than Facebook.
        
               | jadtz wrote:
               | I think its pretty obvious they went with Instagram
               | branding, as Facebook branding has been tarnished for a
               | while.
        
               | Solvency wrote:
               | I'm confused.
               | 
               | It's Facebook by Meta. Instagram by Meta. Why isn't it
               | Threads by Meta?
        
               | epolanski wrote:
               | Tied to ig account.
        
             | goykasi wrote:
             | The web codebase was once written in PHP, but they moved
             | away from that a decade ago.
        
               | rafark wrote:
               | It's still very similar to php. Same syntax, same
               | execution model. If you look at some hack snippets you
               | would think it's php (because it mostly is).
               | 
               | Switching from hack to php and viceversa should be
               | extremely easy.
               | 
               | That being said, they chose python because Instagram is
               | written in python, they are probably using a lot of
               | existing libraries.
        
               | goykasi wrote:
               | That wouldn't even make sense. They created hack/hhvm to
               | scale and optimize the platform and codebase. Taking a
               | step backwards and refactoring (hack != php now) would be
               | a horrible idea.
        
           | [deleted]
        
           | leodriesch wrote:
           | It is fairly normal to build web servers in a way so they can
           | be scaled horizontally (more instances rather than larger
           | instances). So they can just have more containers run their
           | Django servers and distribute the load between them.
        
           | coding123 wrote:
           | Whether it's in C++ or Rust or Python, almost all of any
           | slowness would be from database waiting anyway.
        
             | patrec wrote:
             | It wouldn't. Databases are fast, python is slow.
        
               | patrec wrote:
               | https://techspot.zzzeek.org/files/2015/pymysql_runsnake.p
               | ng
        
             | DeathArrow wrote:
             | Not really. Look at Techempower web benchmarks.
        
         | sgt wrote:
         | Interesting and lovely to hear they decided to use Django. What
         | is your source, though?
        
           | strict9 wrote:
           | Instagram devs have given a lot talks about python and django
           | at conferences. As far as I know IG is still running on
           | django so it makes sense that the dev team would stick with
           | it for something new.
        
           | appplication wrote:
           | How is Django now? I'm a long time python dev (data eng
           | space) but new-ish to web dev. I started a Flask project 2
           | years ago and found it to be pretty full of footguns, mixed
           | messaging on best practices for scalable apps, and the
           | ecosystem feels overbloated with vapor ware extensions.
           | 
           | Is this just a Flask problem, or does Django have the same
           | issues?
        
             | bjourne wrote:
             | Django is the opposite. It holds your hand and tells you
             | how to structure your "app", templates, database, and media
             | assets. And django-admin is godsend. It's awesome when you
             | are on the beaten path cause everything just works. But if
             | you step of the beaten path you will have to tweak various
             | obscure settings and create weird hooks to override
             | Django's defaults. Flask is the opposite. Just a very well-
             | written web server and it's up to you to decide things like
             | project structure, template engine, orm, asset manager,
             | etc. Personally I prefer Django since most of my web
             | applications are quite mundane and don't require the
             | flexibility Flask can provide.
        
               | kstrauser wrote:
               | I've described that as Django is fantastic for writing
               | Django apps. If the thing you want to do can be
               | completely modeled in the defaults, it's impossible to
               | beat.
               | 
               | If that's only 99% true, you might want to investigate
               | other options.
        
               | winrid wrote:
               | You can always build 99% of the App (crud, ui stuff) in
               | Django and the 1% be its own service.
        
             | ak217 wrote:
             | Having used Node/express, Rails, Flask, and Django
             | extensively, I have been very impressed with Django. The
             | framework itself is good, but the superpowers come from the
             | ORM and DRF, and a few other great plugins. The stuff you
             | can do with the ORM and DRF is nothing short of incredible.
             | Now, you could argue that the framework shouldn't be
             | tightly coupled to the ORM, and sqlalchemy does provide
             | some stiff competition as an example of that, but in my
             | experience Django ORM edges out sqlalchemy in terms of
             | usability.
             | 
             | There are some footguns around N+1 queries, but they are
             | general to all database interfaces and pretty easy to avoid
             | IMO.
        
             | sgt wrote:
             | Same opinion. Django is marvelous for getting stuff done.
        
             | winrid wrote:
             | I can't give Django enough praise. I never thought I'd be
             | writing Python.
             | 
             | I've mostly used Node, Java, and Rust, with about 12yrs
             | experience now. Only about one year with Python and Django
             | recently. I am so much more productive than anything else
             | I've tried. Using anything else feels like a mistake for
             | web apps or CRUD apis. I also use type hints.
             | 
             | I use Django + django-ninja + django-unicorn for dynamic
             | UIs.
             | 
             | Building sidewaysdata.com with django right now! Govscent
             | is in Django too, the ORM is nice to combine with AI stuff
             | already in python.
             | 
             | I'm also following the Ash framework which looks promising.
        
               | doteka wrote:
               | You are also running sidewaysdata.com with DEBUG turned
               | on in production, apparently ;)
        
               | winrid wrote:
               | fixed
        
               | winrid wrote:
               | lol thanks. it's not really ready for use yet anyway :)
        
             | coder_san wrote:
             | Django is fantastic. It's much bigger than flask, and has
             | one of the best ORMs I've ever used. Also comes with a
             | built in admin panel which is always nice. You will want to
             | use Django REST Framework to make REST APIs, but if getting
             | a site up and running is all you want, it's perfect.
             | 
             | If you want to purely make APIs though, give FastAPI a try.
        
             | pphysch wrote:
             | Django is excellent. There are some rough edges and signs
             | of age, but the core ORM is fantastic to develop with.
        
             | thewataccount wrote:
             | You simply can't beat Django's ORM for general stuff. It's
             | too awesome. This alone makes it so hard to choose anything
             | else.
             | 
             | I know django doesn't have that "shiny factor" to it these
             | days - but it's very reliable.
             | 
             | > mixed messaging on best practices for scalable apps
             | 
             | The WSGI stuff can be kinda confusing and is used across a
             | lot of python frameworks including django and I think
             | flask?
             | 
             | My advice for "simple scaling" is to start with a separate
             | Postgres instance, and then use gunicorn. Use celery
             | immediately if you have any "long lived" tasks such as
             | email. If you containerize your web layer, you'll be able
             | to easily scale with that.
             | 
             | Finally - use redis caching, and most importantly - put
             | Nginx in front! DO NOT serve static content with django!
             | 
             | > the ecosystem feels overbloated with vapor ware
             | extensions.
             | 
             | This still exists to some degree for some more niche stuff,
             | largely because of it's age. Although impressively they'll
             | generally still work or work with minimal modifications.
             | It's popular enough and old enough that most normal things
             | you'd want to do have decent extensions or built in support
             | already.
        
               | murkt wrote:
               | Ehh, Django ORM always was one of the worst parts of
               | Django.
        
               | mrweasel wrote:
               | True, but it's the best ORM currently available for any
               | language. It might not be the fastest, but it is the one
               | that's has the highest level of developer comfort.
               | 
               | Using Django is probably the reason why I can stand using
               | SQLAlchemy, it's way to complicated for everything I do
               | and it's just not a nice an experience.
        
               | cosmic_quanta wrote:
               | > True, but it's the best ORM currently available for any
               | language.
               | 
               | Such a strong assertion!
               | 
               | I've used multiple ORMs (not Django's), including some of
               | Haskell's type-safe ORMs (e.g. Persistent and Beam). I
               | could not imagine going back. What makes Django's ORM so
               | great?
        
               | DeathArrow wrote:
               | >True, but it's the best ORM currently available for any
               | language.
               | 
               | Better than Entity Framework? Consider me impressed.
        
               | mrweasel wrote:
               | Depending on what you consider important, and I haven't
               | used Entity since... 2010 I think. For me, those two
               | doesn't compare, the simplicity of the Django ORM makes
               | Entity look dated, complex, like something out of the
               | mainframe era.
               | 
               | Entity is way more flexible, in the sense that you can
               | use in any .Net project really. The Django ORM have no
               | value outside Django, I have yet to anyone use just the
               | ORM, without the rest of the framework.
        
               | _fdsfs12123 wrote:
               | > Use celery immediately if you have any "long lived"
               | tasks such as email
               | 
               | Hey, quick question from a relative newbie who is
               | currently trying to solve this exact problem.
               | 
               | Besides Celery, what are good options for handling long-
               | running requests with Django?
               | 
               | I see 3 options:
               | 
               | - Use Celery or django Q to offload processing to worker
               | nodes (how do you deliver results from the worker node
               | back to the FE client?)
               | 
               | - Use a library called django channels that I think
               | supports all sorts of non-trivial use cases (jobs,
               | websockets, long polling).
               | 
               | - Convert sync Django to use ASGI and async views and run
               | it using uvicorn. This option is super convoluted based
               | on this talk [0], because you have to ensure all
               | middleware supports ASGI, and because the ORM is sync-
               | only, so seems like very easy to shoot yourself in the
               | foot.
               | 
               | The added complication, like I mentioned, is that my
               | long-running requests need to return data back to the
               | client in the browser. Not sure how to make it happen yet
               | -- using a websocket connection, or long polling?
               | 
               | Sorry I am ambushing you randomly in the comments like
               | this, but it sounds like you know Django well so maybe
               | you have some insights.
               | 
               | ---
               | 
               | [0] Async Django by Ivaylo Donchev
               | https://www.youtube.com/watch?v=UJzjdJGS1BM
        
               | roflyear wrote:
               | Use anything except Celery, is my vote. Even if that
               | "anything" is something you roll yourself.
               | 
               | Celery is mature, but has bitten me more than anything
               | else.
               | 
               | For scheduling, there are many libraries, but it's good
               | to keep this separate from Celery IMO.
               | 
               | For background tasks, I think rolling your own solution
               | (using a communication channel and method tailored to
               | your needs) is the way to go. I really do at this point.
               | 
               | It definitive is not using async, I think that will bite
               | you and not be worth the effort.
               | 
               | Huey is worth a look.
        
               | Zetice wrote:
               | Not only can you not beat Django's modeling in Python,
               | but I simply cannot find an in-kind solution in JS
               | either. Nothing covers all of the bases the way Django
               | does, or more specifically DRF/graphene-django.
               | 
               | The current state of the art is apparently Prisma, but it
               | covers only a small part of the full picture.
        
             | FredPret wrote:
             | I'm a self-taught noob as far as web engineering goes, but
             | between this course (cs50.harvard.edu/web/2020/) and
             | ChatGPT, I got several commercial Django sites live in a
             | very short amount of time and effort.
        
             | SCUSKU wrote:
             | I use django professionally and for personal projects. I
             | started with Flask and then did FastAPI for a while, but I
             | like django the most since it's the most mature.
             | 
             | The ORM and django admin are killer features out of the box
             | that the other frameworks don't have. I will say though
             | that FastAPI is really nice, especially if you need async
             | support. However, I have found that using django ninja [1]
             | adds a lot of nice to haves that FastAPI has to django that
             | makes it much more fun to use again.
             | 
             | [1] https://django-ninja.rest-framework.com/
        
               | SalmoShalazar wrote:
               | +1 for django ninja. I built a little side project with
               | it a few months back and was super impressed by how easy
               | it was to get up and running. I didn't want to leave the
               | Django ORM behind so was very pleased to find this
               | project.
        
             | gen220 wrote:
             | I've used Flask/SQLAlchemy and Django a bunch in my day,
             | professionally and on the side.
             | 
             | It's a different strokes for different folks sort of deal.
             | 
             | Django is very opinionated, in a way that is "eventually"
             | correct (i.e. they might have had some bad opinions many
             | years ago, but they have generally drifted in a better
             | direction). If the opinions don't align with what you're
             | building, the escape hatches are not generally well-
             | documented or without penalty.
             | 
             | Flask is minimalist and flexible. Once you find a "groove"
             | to building with it that fits your sensibilities, it's
             | quite, quite nice. That being said, the most recent
             | versions of flask have excellent documentation, imo, and
             | the tutorial is a bit more opinionated in a highly
             | productive way. The "patterns" section of the docs is also
             | super useful for productionizing your app.
             | 
             | Personally, I prefer the combo of Flask+SQLAlchemy, and
             | eventually Alembic once you decide that's good for your
             | app's maturity level. I respect Django a lot, I just enjoy
             | the explicitly "less magical" aspects of a Flask stack,
             | which is an opinion-based trade-off imo.
        
           | amethyst wrote:
           | It's a heavily modified internal fork of Django. Source: I
           | work on our Python Language Foundation team.
        
             | KRAKRISMOTT wrote:
             | Are you using the ORM/auth/admin features or is Django just
             | a lightweight router? I can't imagine the ORM being too
             | useful at Facebook's scale.
        
               | amethyst wrote:
               | I'm not familiar enough with Django to say for sure, but
               | I assume at this point it's almost entirely custom ORM
               | and libraries on top of async django routing/response.
        
               | [deleted]
        
             | PartiallyTyped wrote:
             | What exactly do you do? This sounds like a
             | JIT/Interpreter/Language dev job. Is this correct?
             | 
             | If so, do you have any recommendations or suggestions for
             | someone getting their feet wet on this?
        
               | amethyst wrote:
               | It's mostly foundational work around developer tooling
               | and infrastructure that isn't already covered by other
               | dedicated teams (eg, there is a dedicated Cinder team).
               | My latest work has focused around formatting and linting,
               | and includes open source work on usort, our import sorter
               | [1], and fixit, our custom linter [2] that makes it easy
               | for teams to build and deploy their own custom lint rules
               | with autofixes.
               | 
               | Some of my team mates work on driving internal adoption
               | of Cinder, rolling out new versions of Python everywhere,
               | improvements to our build and packaging systems,
               | supporting other Python tooling teams, etc. There's a lot
               | of cross-functional project work, and our primary goal is
               | to improve the Python developer experience, both
               | internally and externally wherever we can.
               | 
               | 1: https://usort.rtfd.io
               | 
               | 2: https://fixit.rtfd.io
        
         | Myrmornis wrote:
         | How heavily are type annotations and type checking used in the
         | codebase?
        
           | phyrex wrote:
           | It's fully typed and nothing can be merged that doesn't pass
           | the type checker
        
             | wing-_-nuts wrote:
             | What are the benefits of python with types vs a statically
             | typed language like java, golang, etc?
        
               | jajajajajajajaj wrote:
               | The main "benefit" is that Instagram is written in Python
               | and always has been.
               | 
               | It's millions of lines of code, you can't just change it
               | to be Java one day.
        
         | echelon wrote:
         | This would partially explain why it feels so slow. Some page
         | refreshes are taking up to a couple of seconds.
         | 
         | It's probably hurried along python code and fairly simple
         | horizontal scaling. Some population of individual instances are
         | probably pegged, and random requests are frequently landing on
         | their backlogged request queues.
         | 
         | This seems to have been rushed out the door to capture a unique
         | market opportunity, and they'll clean up the engineering once
         | they get engagement.
        
       | seanp2k2 wrote:
       | For the "Python isn't fast enough for production backend" crowd
       | from the same company that brought you the largest social network
       | built on PHP + MySQL.
        
         | that_guy_iain wrote:
         | For me, it does seem really strange since PHP (currently) is
         | faster than Python and PHP wasn't fast enough for Facebook. But
         | it's interesting to see how it'll play out.
        
           | rafark wrote:
           | It's obvious why they went with python. Since Instagram was
           | written in python and Threads is based on instagram, they're
           | probably using a lot of the libraries they already have for
           | Instagram.
        
             | that_guy_iain wrote:
             | That makes some sense.
        
           | DeathArrow wrote:
           | Probably Python is deemed as more appealing to hipsters than
           | good old PHP.
        
             | wiseowise wrote:
             | Python is older than PHP.
        
         | paultopia wrote:
         | OTOH threads is kinda slow as shit right now
        
         | anayak wrote:
         | While that is technically correct, they haven't been running
         | vanilla PHP for quite some time - it's jitted into x86 and run
         | natively on the machine. It's also pretty extensively
         | optimized. And the VM that does the jitting is all C++. Their
         | MySQL deployment is also an internal fork that's been heavily
         | modified for scalability (storage/compute separation, sharding,
         | RocksDB based storage engine instead of InnoDB, Raft for
         | synchronous replication, etc.). Lots of really fantastic
         | engineering work has gone into making their backend infra
         | scale.
         | 
         | Some worthwhile references:
         | 
         | [1] https://engineering.fb.com/2016/08/31/core-data/myrocks-a-
         | sp...
         | 
         | [2] https://research.facebook.com/file/529018501538081/hhvm-
         | jit-...
         | 
         | [3]
         | https://research.facebook.com/file/700800348487709/HHVM_ICPE...
        
           | ericbarrett wrote:
           | Facebook scaled to ~1b daily actives on MySQL + InnoDB. There
           | was _lots_ of engineering work, like schema sharding
           | (denormalization), automation, plenty of bug fixes and
           | patches for MySQL (most or all contributed back to upstream,
           | from what I remember), and of course a massive caching layer;
           | plus throwing crazy hardware at the problem. Nonetheless the
           | underlying engine was something any MySQL user or admin would
           | have recognized. And we backed it all up, every day, in  < 24
           | hours, using an unmodified mysqldump. (FB MySQL team
           | 2009-2012)
        
             | 9dev wrote:
             | > And we backed it all up, every day, in < 24 hours, using
             | an unmodified mysqldump.
             | 
             | But... how? Considering all the transactions in flight, and
             | everything? And did you ever Test disaster recovery with
             | that setup?
             | 
             | I've worked on relatively big projects, but FAANG
             | engineering is like some entirely different field of
             | software engineering. Fascinating.
        
           | grishka wrote:
           | Curiously, VKontakte also started from PHP+MySQL but went
           | another way. PHP is compiled ahead of time with something
           | called KittenPHP. It's open-source. For databases they
           | switched some parts to their own bespoke solutions in C that
           | talk with PHP over the memcached protocol. These are called
           | KittenDB (or "engines") for the simple single-purpose ones,
           | and there was also a more generic MySQL replacement in
           | development when I left in 2016, MeowDB.
        
             | jonhohle wrote:
             | I'm not sure why someone of facebook's scale would want JIT
             | instead of AOT. That's a lot of servers all JITing the same
             | thing that could be done once at build time.
        
               | jeffffff wrote:
               | JIT compilation has the opportunity to do profile-guided
               | optimization at runtime. JIT compilation is also simpler
               | when distributing an application to non-identical
               | servers, as it can optimize for the exact hardware it is
               | running on.
        
               | yuliyp wrote:
               | Facebook actually does some compile-time optimizations,
               | but also ships some of the profiling data for JITing
               | between early deploy phases and the rest of the fleet:
               | https://engineering.fb.com/2021/03/03/developer-
               | tools/hhvm-j...
               | 
               | Also, Facebook used to do ahead of time compilation
               | (HPHPc) but eventually HHVM managed to outperform it.
        
               | woadwarrior01 wrote:
               | HHVM's predecessor at FB was an AOT PHP to C++ transpiler
               | called HPHPc. There are a lot more optimization
               | opportunities with JIT compilation when dealing with
               | dynamically typed languages like PHP.
        
         | xuancanh wrote:
         | It may not be because of Python, but Thread is definitely not
         | ready for the current load. Few hours ago no one could access
         | to Zuck's profile page, and he had to take it to private to
         | hide the issue.
        
           | ryantgtg wrote:
           | I think there have been issues where profile pages say "this
           | account is private and has 0 followers." I saw that yesterday
           | on every account I clicked on (on web). And then a few
           | minutes later it was back to normal. It might not be that he
           | manually took it private to hide an issue.
        
           | rafark wrote:
           | Tweets load very slowly on my end. So slow I thought there
           | was a js error on my end, you know, like when you get a never
           | ending spinner animation because there was an unhandled
           | exception thrown.
        
             | Solvency wrote:
             | Surely you mean threads.
        
         | hello_moto wrote:
         | Yep, Vanilla Python isn't fast enough so they have to do more:
         | 
         | "It's running on Instagram's #Cinder fork that includes a JIT,
         | lazy-loaded modules, pre-compiled static modules, and a bunch
         | of other interesting tweaks against vanilla Python 3.10"
         | 
         | From the Tweet.
         | 
         | Personally, I don't care what language people use. We all know
         | that if you have money, you can just throw in more servers.
         | Until your CFO and CTO decided that the next big thing is to do
         | "Cost Saving" (in the eye of recession).
         | 
         | I'm surprised in 2023 people are still debating mainstream
         | programming language "PROD" readiness.
         | 
         | It's like a "cool kid" competition.
        
           | ubercore wrote:
           | > We all know that if you have money, you can just throw in
           | more servers.
           | 
           | Definitely not true at Facebook scale. You need to be smart,
           | fast, _and_ throw more servers at it.
        
         | chippiewill wrote:
         | PHP is actually shockingly fast for a non-JIT'd language these
         | days.
        
           | icelancer wrote:
           | PHP8 has a JIT now but I get what you mean.
        
         | albertopv wrote:
         | They didn't use vanilla Python, though.
        
         | shortrounddev2 wrote:
         | Quite often, the choice of language is failing to see the
         | forest for the trees. Saving nanoseconds or cycles because you
         | chose C++ over Python pales in comparison to milliseconds spent
         | at network barriers reading from a cache service or database
        
           | throwawaymaths wrote:
           | At Facebook scale those nanoseconds are worth millions in
           | compyte, energy, etc.
        
           | yeck wrote:
           | Perceivable latency isn't the only consideration. Depending
           | on your business, compute for your proprietary workloads
           | might be one of your biggest expenses. You could see an order
           | of magnitude improvement to resource consumption depending on
           | the type of workload and the language it is written in.
           | 
           | Ultimately these these choices are all about trade-offs.
           | Maybe python is fine for them, maybe they've built themselves
           | into a corner. Time will tell.
        
             | MengerSponge wrote:
             | When they need a full re-write the specification will be
             | easy: they've already got the pseudocode. (I kid because I
             | love Python)
        
           | IshKebab wrote:
           | The difference between C++ and Python is not nanoseconds.
           | 
           | And I have definitely seen projects fail due - in part - to
           | language choice. Of course projects can succeed in almost any
           | language but that doesn't mean the language choice is
           | irrelevant.
        
         | DeathArrow wrote:
         | Python is just fine for all the 100 users Threads will manage
         | to acquire. :)
        
           | latexr wrote:
           | > all the 100 users
           | 
           | That's off by multiple orders of magnitude, if you believe
           | their reports of 30 million sign ups in under a day.
           | 
           | https://www.theverge.com/2023/7/6/23786108/threads-
           | internal-...
        
             | Twitterboi wrote:
             | You mean sign-in's?
        
               | latexr wrote:
               | I'm using their terminology. The distinction seems
               | irrelevant for the conversation.
        
         | Klonoar wrote:
         | In addition to the Facebook/PHP comments here, an often
         | overlooked one is that Instagram was a big Python/Django shop
         | for years (and probably still is in many parts).
        
         | arcticbull wrote:
         | Most of Instagram is Django.
        
         | nikhizzle wrote:
         | Earlyish facebook engineer here. Early days FB php was nothing
         | like the php used to template websites. All kinds of
         | specialized libraries to enable a much more sophisticated
         | programming style. Think functional helpers, and asynchronous
         | execution on thousands of cores, spanning trees across data
         | centers using ssh etc. As a tangent a lot of the good stuff I
         | used was written by Evan Priestley, who also did Phabricator
         | and lots of other strong systems.
        
           | iknowSFR wrote:
           | Can you speak to the onboarding and learning curve in that
           | environment? What was the training like? What kinds of
           | timelines did you experience regarding expectations as a new
           | hire?
           | 
           | Asking because I'm curious about these early stage mega-tech
           | companies that are working at a scale I can't fathom.
        
             | nikhizzle wrote:
             | Six week boot camp, you are pretty fluent by week 4.
             | Notably though this was before mega tech days, we had under
             | 100 engineers.
        
           | supernikio2 wrote:
           | I'm of course speaking from an outsider's point of view, but
           | do you think PHP was a good option, or was it a "fitting a
           | square peg into a round hole" situation?
        
             | troupo wrote:
             | > but do you think PHP was a good option, or was it a
             | "fitting a square peg into a round hole" situation?
             | 
             | Facebook was founded in _2004_. Whatever fancy tech you 're
             | thinking that would fit the round hole, it didn't exist
             | then. They also never thought it would serve a couple
             | billion of users one day, so they built it in whatever they
             | had and knew at the time.
             | 
             | Same for Instagram and Python (they started with Django).
        
               | zitterbewegung wrote:
               | How they have used Instagram with Django is that they
               | have replaced pieces of Django over time with custom
               | parts which is a benefit of the design of Django.
        
             | nikhizzle wrote:
             | It was not a deliberate choice, just it was what the site
             | was built in, so people like Evan and Marcel (Laverdet)
             | made php beautiful for us.
        
         | grumpyprole wrote:
         | PHP didn't scale for the number of users or the number of
         | developers. Presumably that's why they built their own PHP-like
         | language called Hack with static types and a new runtime.
        
         | gedy wrote:
         | Would this pass their System Design interview?
        
         | berkle4455 wrote:
         | Except PHP actually is fast while Python isn't.
        
           | ldng wrote:
           | Except it's not. Bindings to C libraries are.
        
         | pjmlp wrote:
         | You mean the company that had to create their two PHP
         | implementations and a translator to C++, to actually scale?
        
           | DoesntMatter22 wrote:
           | Lol. Do you think you are just going to use go and everything
           | is going to scale to FB level size?
           | 
           | Facebook was already at a scale larger than 99.99999 percent
           | of sites before they had HHVM
        
             | throwawaymaths wrote:
             | Of course not. One uses Erlang and scales to Whatsapp
             | scale.
        
             | pjmlp wrote:
             | Remember when Twitter was written in Ruby?
        
               | ramesh31 wrote:
               | >Remember when Twitter was written in Ruby?
               | 
               | Ruby is the exception to the rule.
               | 
               | Friends don't let friends start new Ruby projects in
               | 2023.
        
               | epolanski wrote:
               | Ruby in 2023 is miles ahead it was 10 years ago.
        
               | jarjoura wrote:
               | LOL, at least Ruby inspired a new generation of languages
               | that we have today.
        
           | c7DJTLrn wrote:
           | Guess I'll just write my next project in Bash then, after
           | all, if it's a good idea they will come right? Execution can
           | wait until you've got a few hundred mil.
        
             | ramesh31 wrote:
             | >Guess I'll just write my next project in Bash then, after
             | all, if it's a good idea they will come right? Execution
             | can wait until you've got a few hundred mil.
             | 
             | Yea actually. Bash/CGI could handle plenty. If thats your
             | most comfortable language then go for it.
        
           | disgruntledphd2 wrote:
           | They ended up compiling PHP to assembly! Facebook is a
           | company that will go to ridiculous lengths to avoid rewrites.
        
             | snapetom wrote:
             | Was there a separate initiative for assembly, or are you
             | thinking about HipHop? HipHop was for C++ and got a lot of
             | attention back when it was released.
        
               | pjmlp wrote:
               | Yes, the 2nd rewrite for a VM, where the JIT compiler
               | turned out to be quite good versus HipHop, and
               | development focus switched to HHVM.
        
               | disgruntledphd2 wrote:
               | I was thinking about HHVM, which I believed was
               | essentially assembly for a VM. Am I wrong on that?
        
               | snapetom wrote:
               | No, you're right. HHVM was the 2nd iteration. I was
               | thinking of the first project, which was for C++. Looking
               | at wikipedia, that was called HPHPc.
        
             | nikanj wrote:
             | Rewriting in hip-tech-du-jour is for startups with lots of
             | funding and little of traction.
             | 
             | Why would a massively successful company toss a stick in
             | their own spokes by tearing the product to ground?
        
               | nobleach wrote:
               | But it's actually a _really bad_ strategy for a startup!
               | It'll be realized the moment you try to hire for hip-
               | tech-du-jour. Most likely, you don't even have the
               | problem that hip-tech-du-jour claims to solve.
        
             | jarjoura wrote:
             | I know you're being hyperbolic, but during my time there,
             | your impact was measured only on measurable things like
             | performance gains but never on lines of code. So engineers
             | would always go in the direction of whatever gets you the
             | biggest win.
             | 
             | Sometimes that actually meant rewriting the thing from
             | scratch, and other times that meant adding something else
             | as a middleware to prevent touching the original thing.
             | 
             | For the most public success story, React was a complete
             | rewrite of the thing before it, for example.
        
               | disgruntledphd2 wrote:
               | I spent five years there, and made that joke constantly.
               | 
               | It's mostly hyperbole but there's definitely a kernel of
               | truth to it. I generally dislike rewrites and I wonder if
               | FBs approach influenced me more than I realized at the
               | time.
        
             | that_guy_iain wrote:
             | I thought they tried it multiple times. The problem is they
             | had so many developers writing new code any rewrite attempt
             | would fall behind super quickly?
             | 
             | So they created their own language Hack so allow a slow
             | transfer from PHP to Hack. Which is basically the idea
             | behind Kotlin.
        
           | rafark wrote:
           | Facebook had hundreds of millions daily active users before
           | writing a compiler.
        
             | pjmlp wrote:
             | And lots of nodes not able to keep up, rewriting backend
             | services in C++ to make up for it.
        
             | coding123 wrote:
             | 98% of the people that initially thought (Python for
             | Threads - OMG performance!) work at a company that will
             | tell you straight up they expect 100M RPS in a year or two,
             | will actually only get to 50k RPS in 5 years before the
             | "Our incredible journey" letter is issued.
        
             | rewmie wrote:
             | > Facebook had hundreds of millions daily active users
             | before writing a compiler.
             | 
             | In other words, Facebook felt compelled to write a compiler
             | when they had hundreds of millions daily active users.
        
               | depereo wrote:
               | If you have hundreds of millions of daily active users
               | then needing to write a compiler isn't the end of the
               | world.
        
         | steveBK123 wrote:
         | They wrote a new app in the language 99.9% of 0-5 year
         | experienced devs know.
         | 
         | Absolutely shocking.
        
         | ineedasername wrote:
         | Pfft. It would be 100x more performant if they'd written it in
         | asm.
        
           | baq wrote:
           | They also wouldn't release this year, perhaps neither next
        
       | babuloseo wrote:
       | Never forget https://github.com/twitter/the-algorithm/pull/447
        
         | aio2 wrote:
         | Finally, a difference between the two.
        
       | nunez wrote:
       | Can we block Twitter links on HN?
       | 
       | No point in supporting that user-hostile tire fire.
       | 
       | This could have been on Mastodon.
        
         | ambivalence wrote:
         | It is on Mastodon as well, actually:
         | https://mastodon.social/@ambv/110665808046065754
        
           | philipov wrote:
           | Could we get the mastodon link as the main link, please? Can
           | actually view that one.
        
             | aeyes wrote:
             | Is the Twitter link really not working for you at the
             | moment? I can open it without having an account.
        
               | rsynnott wrote:
               | You can't see replies on Twitter links anymore, though
               | that doesn't really matter in this particular case.
               | 
               | Of course, who knows what it'll be doing tomorrow. I'd
               | definitely think it's probably time for HN to prefer
               | Mastodon (or Threads, I suppose) links, where a post is
               | available on multiple microblogging things; Twitter has
               | had any one of about five behaviours for non-logged-in
               | linked tweets (work properly, work but without replies,
               | spin forever, redirect to login, generic error page) over
               | the last week, depending on when you try.
        
             | reledi wrote:
             | The tweet is currently viewable while logged out. Have you
             | tried?
        
               | timeon wrote:
               | Only thing I see is: "Something went wrong. Try
               | reloading."
               | 
               | (Mastodon link works.)
        
         | xk_id wrote:
         | The links are being submitted and voted on by other members of
         | the community. I'm not sure what gives you the impression
         | you're in a position of authority.
        
           | _Algernon_ wrote:
           | Asking a question is having the impression that one is in a
           | position of authority now?
        
             | the-dude wrote:
             | No, but appealing to 'we' is. Not that 'we' in this case
             | has much meaning, it actually could read 'you'.
        
               | johnfn wrote:
               | What other pronoun do you want OP to use? "Can I ban..."
               | - well, obviously not. "Can you ban..." who is "you" in
               | that case?
        
               | the-dude wrote:
               | > "Can I ban..." - well, obviously not
               | 
               | Of course he can. And he already does I assume,
               | considering his other remarks in the comment. So
               | including him in the 'we' does not really make sense ...
               | if you consider it a question. Which it is not.
               | 
               | > "Can you ban..." who is "you" in that case?
               | 
               | That would be us dude, but not him as there is no reason
               | to ask him, his position is clear.
               | 
               | So he is telling us what to do, dressed up as a question.
        
               | [deleted]
        
             | mr90210 wrote:
             | Exactly my thoughts. Seems like asking genuine questions is
             | not acceptable anymore because some people can't help but
             | to resort to personal attacks. Unbelievable.
        
               | the-dude wrote:
               | This is not a genuine question. This reads like 'Darling,
               | could you please put the garbage out?' is a question.
               | 
               | It is not a question.
        
           | mr90210 wrote:
           | Oh lord, the person asked a question, A QUESTION!!!!
           | 
           | Geez, it is you who is the authoritarian for coming so hard
           | at the person who asked a pertinent question.
           | 
           | Geez.
        
         | redleader55 wrote:
         | I think the cautionary tale here is not to do what Twitter did:
         | prevent links to competing products, blocking open access, etc.
         | 
         | Don't you find it ironic that you are proposing we do on HN
         | exactly what Twitter did, which is probably why you suggest not
         | linking to Twitter?
        
       | czardoz wrote:
       | I worked on Cinder, and on the web server. Happy to answer
       | technical questions if any :)
        
         | vmsp wrote:
         | Can you expand on the web server? Does it implement WSGI?
        
           | czardoz wrote:
           | It has some elements of WSGI, but has expanded into its own
           | thing which supports asyncio (this happened before ASGI was a
           | thing).
        
         | ntonozzi wrote:
         | What are the most impactful and likely features that you'd like
         | to see merged into CPython?
        
           | czardoz wrote:
           | I'd love to see lazy imports become part of the upstream at
           | some point.
        
         | feydaykyn wrote:
         | Hi,
         | 
         | Would you recommend using Cinder for stack made of - Django -
         | Cython - Numpy
         | 
         | Thanks!
        
           | czardoz wrote:
           | If this stack is heavily skewed towards numerical
           | computations, Cinder might require a lot of tuning to be
           | effective.
        
         | whalesalad wrote:
         | Who would benefit from Cinder? What kinds of workloads? The
         | README is kinda lacking on this.
        
           | czardoz wrote:
           | Cinder's feature set is highly optimized for IO bound web
           | services that run under a forked-worker model.
           | 
           | For example: you start a main process, warm it up with a few
           | requests, run the JIT compiler and then fork off worker
           | processes to handle the main chunk of traffic.
           | 
           | As of now, it requires hand-tuning to get the best possible
           | performance.
           | 
           | In terms of use cases, Cinder does the best when faced with
           | "business logic" code (lots of inheritance, attribute
           | lookups, method calls, etc). It can speed up numerical
           | computations too, but you're probably better off using a
           | library if that's the majority of the workload.
        
       | zengid wrote:
       | Ok but is the front-end in React Native?
        
         | akrymski wrote:
         | I'd suspect so, but would love to see that confirmed.
        
         | blkhp19 wrote:
         | Native
         | https://twitter.com/vinaygaba/status/1676968580712366080?s=6...
        
       | [deleted]
        
       | costco wrote:
       | There's definitely some overlap in the backends because
       | https://www.threads.net/t/C is the first Instagram post but on
       | Threads with the exact same amount of likes it has on Instagram
       | yet https://www.threads.net/@kevin shows no threads.
        
       | ddorian43 wrote:
       | No tweaks mentioned at all inside. The title is wrong. Kinda
       | weird to start a new project that needs to scale with Python
       | though (I use Python most of the time).
        
         | kleinsch wrote:
         | IG is built in Python (which is why this is also) and it seems
         | to scale fine.
        
         | [deleted]
        
         | threeseed wrote:
         | "JIT, lazy-loaded modules, pre-compiled static modules, and a
         | bunch of other interesting tweaks against vanilla Python 3.10"
        
           | ddorian43 wrote:
           | That is just CPython fork from instagram
           | https://github.com/facebookincubator/cinder/
        
             | ambivalence wrote:
             | The fork is where the tweaks live.
        
         | jackmott wrote:
         | [dead]
        
         | troupo wrote:
         | It seems they are re-purposing Instagram's backend for this.
         | And Instagram's backend is Python
        
         | super_flanker wrote:
         | As far as I can tell from my prior experience of working at
         | Insta is that all the work is done by various services written
         | in C++/hack. Python/Cinder would be just interface layer
         | routing the incoming request/response from/to client.
        
       | threeseed wrote:
       | Would love to know more details about the rollout.
       | 
       | Because it's adding well over a million new users each hour which
       | is pretty staggering.
        
         | phyrex wrote:
         | It's just running on the shared meta infrastructure which is
         | built to handle that scale
        
         | zokier wrote:
         | I would assume its massively overprovisioned at this point to
         | cover first few weeks if not months of projected growth.
         | 
         | But still there probably is all sorts of juicy details that
         | would be neat to hear about
        
           | sanderjd wrote:
           | One of the "fun" things when I worked at a big company was
           | thinking through how to make the very first launched version
           | of something work at millions-of-users scale. There is no
           | organic growth in traffic from zero when a non-negligible
           | subset of billions of active users will try it out right
           | away.
           | 
           | But I think on net it was still an easier problem than
           | scaling a startup product from zero, because the tooling and
           | processes to support huge usage is already so mature at these
           | big companies.
        
         | pavlov wrote:
         | Meta has 3.8 billion monthly active users across its products.
         | 
         | A few million new people is less than 0.1% of the total. It's
         | entirely within expected variance.
        
           | nomel wrote:
           | I think they're looking at it from a an individual product
           | perspective. % users of an entire company can be extremely
           | different than % growth of a new product, within that
           | company.
           | 
           | My naive assumption is that they have some neat simulated
           | user/load system to help identify any scaling issues that
           | might arise when launching a service, and the extreme %
           | growth. That 3.8 billion was spread over 15 years.
        
       | mattfrommars wrote:
       | Makes me wonder why did they not opt for Java or C# as their
       | backend choice. I better stop learning C# and move on.
        
       | manca wrote:
       | I've been using Threads since late yesterday and it's been very
       | slow... sometimes it wouldn't even load content when you browse
       | specific profiles (especially the active ones with tens of
       | thousands of followers). So I am not surprised the backend is in
       | Python :)
        
       | mlrpfx wrote:
       | Always the same people that get their names in the news.
       | 
       | I would say that the amount of tweaks that prop up Python is no
       | cause for celebration.
        
         | blitzar wrote:
         | Yet here it is ... powering a global scale app.
         | 
         | Too weak for someones hobby project, strong enough for global
         | scale.
        
           | mlrtph100 wrote:
           | So Google and Dropbox, who switched to Golang, are hobby
           | projects.
        
             | cmrdporcupine wrote:
             | Choice of language is not what makes things succeed or fail
             | in production.
             | 
             | Team best practices, enforcement of coding style and
             | technique, good project management, infrastructure, and
             | team cohesion etc.
             | 
             | And weeding out bad engineers who think that "switching to
             | a new [language|framework|religion]" will solve all
             | problems.
             | 
             | 10+ years ago Facebook itself did a remarkable job of
             | scaling up, with hardly any outages, a PHP LAMP stack
             | thingy to one of the hugest traffic websites in the world.
             | Meanwhile back then Twitter mucked around with every novel
             | technology they could find or invent, and had constant
             | outages.
             | 
             | Also Google didn't "switch to Golang." There's plenty of
             | C++, Java, Python there. I worked there for 10 years and
             | encountered only a handful of Go projects. Lots of cloud
             | services in C++.
             | 
             | Would I _personally_ choose Python for a project? No, I don
             | 't like it, and I work in Rust full time because I prefer
             | it. But if I worked at Meta / Instagram and had a team of
             | Python engineers and existing libraries & infrastructure,
             | this would be the right approach.
        
               | aeyes wrote:
               | Also, what's really different from Instagram? You change
               | the picture for a short text and many other features
               | already existed - comments, likes, subscriptions,
               | recommendations, personal messages...
               | 
               | I think this is just an Instagram skin with some feature
               | flags.
        
           | n_ary wrote:
           | Off-topic: it is not yet [global scale] because it is only
           | available in US. Here in EU, I still get "not available in
           | your country".
        
             | blitzar wrote:
             | It is only unavailable in the EU.
             | 
             | Something, something privacy regulations.
        
             | reaperducer wrote:
             | _it is not yet [global scale] because it is only available
             | in US. Here in EU, I still get "not available in your
             | country"._
             | 
             | It's not only available in the U.S. It's available in 100
             | countries. There's more to "global" than the E.U.
        
           | superq wrote:
           | Throw a trillion cores at anything and it can be global
           | scale.
        
           | Yoric wrote:
           | My few years of personal experience of using Python with even
           | small-ish scale professional projects have been...
           | underwhelming so far. I wonder how Meta (and Google, and all
           | the others who write serious Python code) manage to keep
           | things running - and I'm not talking (only) of performance,
           | but largely of bugs.
           | 
           | My personal suspicion is that the secret sauce is simply to
           | have a sufficient budget to hire an army of code quality
           | engineers. I could be wrong, though.
        
             | hospitalJail wrote:
             | >My few years of personal experience of using Python with
             | even small-ish scale professional projects have been...
             | underwhelming so far
             | 
             | You have years of experience but couldn't point to anything
             | in particular?
             | 
             | Heck, I like python and I can complain about dynamic typing
             | issues in for loops, or that they are adding features like
             | generators/decorators that make code more difficult to
             | understand, which goes against the zen of python.
             | 
             | (But I still think python is great)
        
               | Yoric wrote:
               | Why would I need to point to anything in particular? I
               | think that all Python developers know them already.
        
             | o1y32 wrote:
             | I cannot find any substance in your comment. At minimum it
             | should provide some rough examples instead of simply
             | "underwhelming".
        
               | Yoric wrote:
               | Well, I wasn't planning to spend time criticizing Python.
               | I think that we all know roughly its pros and cons, no
               | need to add fire.
               | 
               | My questions remain, though.
        
             | marcinzm wrote:
             | You can write crap code in any language. To avoid it you
             | need to enforce some decent set of best practices for that
             | specific language. That also means not using best practices
             | from language X in language Y and then complaining that
             | language Y is crap.
        
               | Yoric wrote:
               | Yes. And one way to rephrase my question is: which best
               | practices work for these Python mega-projects?
        
             | ambivalence wrote:
             | In short: static analysis, unit tests, staging
             | environments, effective log aggregation.
             | 
             | The first one is a big productivity booster as it shows you
             | bugs before you commit them.
             | 
             | Here's a post about this, not exactly new but still
             | describing the general principles very well:
             | https://instagram-engineering.com/static-analysis-at-
             | scale-a...
        
               | VWWHFSfQ wrote:
               | I would say that those things don't really contribute to
               | the actual runtime scalability of the application. Those
               | things ensure that the code is at least correct for what
               | it's supposed to do, not necessarily whether it's
               | performant. Even the most perfectly-written Python code
               | will still be dog-slow and require an enormous budget to
               | actually run at scale. It's obviously fine to do that and
               | there are many success stories, but all of them required
               | massive infrastructure investment.
        
               | ambivalence wrote:
               | > Even the most perfectly-written Python code will still
               | be dog-slow and require an enormous budget to actually
               | run at scale.
               | 
               | Nah, asyncio on uvloop is plenty fast.
        
               | Yoric wrote:
               | For context, I was asking about scaling code complexity,
               | not scaling performance.
               | 
               | That being said, my personal experience suggests that
               | both are really hard in Python, and not independent.
        
               | Yoric wrote:
               | Thanks. For context, I happen to have a PhD in static
               | analysis, so I'm not entirely stranger to the general
               | idea :)
               | 
               | Sadly, all the attempts I've had with static analysis in
               | Python screamed that while the language and tools make a
               | valiant effort at supporting a reasonable set of
               | annotations, 8-9 years after PEP 484, the libraries are
               | simply not yet ready for it (not even, in many cases, the
               | standard library).
               | 
               | Unit tests, staging environment, effective log
               | aggregation are all important tools, of course.
        
               | [deleted]
        
             | code_runner wrote:
             | Tests is likely a large part of it. There are some great
             | tools for code formatting, quality, and type checking. It's
             | a very flexible language...
             | 
             | When people are hyper critical of Python I feel it's
             | typically they haven't seen real professional Python before
             | and just throw scripts around... if you work with some
             | hardened Python pros you pick up the tricks really quickly
             | and it's very enlightening.
             | 
             | I've always struggled to find all of those tricks in one
             | online resource personally.
        
               | epgui wrote:
               | Every language will have its gurus and its own peculiar
               | with-the-grain approach, but the python language has a
               | ton of non-essential complexity baked into its very
               | design/core (especially for a language that people seem
               | to think is "simple"-- it's not, not at all; it's a
               | language full of "pragmatic compromises", apparently by
               | intentional philosophy), which places fundamental limits
               | on how safe your application can be.
        
             | smrtinsert wrote:
             | The secret sauce is Intellij and some community knowledge.
             | There's a library for everything and a great IDE makes it
             | pretty painless and fun. Not sure I'd always jump for it
             | thanks to the GIL but I wouldn't mind a job where it's my
             | primary instead of my secondary or third language.
             | 
             | The biggest pain with it is of course refactoring. It's
             | tedious, but I think still comes out ahead in terms of
             | productivity for high level stuff.
        
           | nobleach wrote:
           | But this is what FB/Meta are known for. They managed to scale
           | PHP for years. They did fork it into Hack. They do have their
           | own extensions. But they've put a ton of time and engineering
           | effort into it. They didn't do it because they thought PHP
           | was a great language. It was because they already had a
           | massive infrastructure and tons of engineers that were
           | already productive. Doing the math they realized it'd be
           | cheaper to keep using PHP.
           | 
           | The same is true here. They spent so much money getting
           | Instagram to scale. They created Cinder to try and fold their
           | hacks/tweaks back into Python. Spinning up some
           | Rust/Java/Nim/Zig/Pony/Whatever stack sounds like it'd be
           | fun... but spinning up a stack with which you're already VERY
           | familiar, sounds like a money-maker.
           | 
           | I'm totally agreeing btw, I realize this came off like it
           | might be a counterpoint.
        
       | Solvency wrote:
       | Why has everyone on here been comparing Threads to Twitter when
       | threads.com _literally_ says its a Slack alternative for
       | _makers_? The product has absolutely nothing to do with a Twitter
       | style social network.
        
         | PartiallyTyped wrote:
         | Threads.com is not Threads.net. The latter owned by
         | Instagram/Meta.
        
           | Solvency wrote:
           | Jesus Christ this is confusing.
        
       | nologic01 wrote:
       | No matter what the true combination of reasons, this is a massive
       | endorsement of the python ecosystem.
       | 
       | Its an interesting dynamic: popularity draws attention, attention
       | helps adress pain points, this makes the platform more attractive
       | and the popularity increases...
        
       | shaftoe444 wrote:
       | > It's running on Instagram's #Cinder fork that includes a JIT,
       | lazy-loaded modules, pre-compiled static modules, and a bunch of
       | other interesting tweaks against vanilla Python 3.10.
       | 
       | So not entirely just python 3.1.
        
         | roflyear wrote:
         | Python is a spec not an implementation, for what it is worth.
        
         | zitterbewegung wrote:
         | I have ported code from Cinder to CPython . The fork has some
         | optimizations that can be easily put in CPython and Facebook is
         | open to port features. I'm not sure if Facebook wants to
         | continually have a fork but CPython is open to have those
         | features merged in if they make sense.
        
           | amethyst wrote:
           | The Cinder team's longterm goal is to upstream as much as we
           | can, and make the rest available as pip-installable
           | extensions that anyone could theoretically install and use on
           | CPython. The fewer internal changes and patches we need to
           | maintain, the faster we can adopt upstream Python releases
           | and all of the associated performance and tooling wins.
        
       | andrewstuart wrote:
       | If you know you've got gigantic scale why would you not use a
       | compiled language?
       | 
       | Surely it makes massive difference to hardware utilization?
       | 
       | I like Python and javascript but there's perfectly fine tools
       | around now for building systems that allow one server to do a lot
       | more.
       | 
       | They say you should not prematurely optimize. Equally you should
       | not be prematurely unoptimised.
        
         | paulddraper wrote:
         | Using a compiled languages for web stuff is like using a
         | Ferrari for a commute.
         | 
         | It's better, but not the limiting factor.
        
           | andrewstuart wrote:
           | Threads is intended to be massive scale.
           | 
           | I'd use the analogy of a Japanese bullet train as the
           | compiled language and a diesel train as Python.
           | 
           | Python is my primary language along with JS but if I was
           | building a new service for Meta I wouldn't use either at the
           | back end.
        
         | callamdelaney wrote:
         | Because Zuck probably told his team to make this a few weeks
         | ago to annoy Musk. Weird ego thing going on.
         | 
         | Python is likely faster to push out an app like this than say
         | rust or even go.
         | 
         | Also why spend time optimising something that will be dead in a
         | few months?
        
           | [deleted]
        
           | smrtinsert wrote:
           | !remindme 2 months. After 30 million signups in a day I'm
           | pretty sure we just saw the birth of a new Meta social
           | network.
        
         | jajajajajajajaj wrote:
         | If you want the real answer, it's because Instagram was
         | acquired and the startup used Django. That's it.
        
           | oaiey wrote:
           | Exactly. This is all over engineered thinking. They probably
           | pulled the core Instagram engineering team and gave them a
           | new task. They used what they had.
        
       | TheAdamist wrote:
       | Ironic that a post about Threads backend is hosted on twitter.
        
         | [deleted]
        
         | HarHarVeryFunny wrote:
         | Is there a web-based version of threads at the moment, or just
         | the app ?
        
           | [deleted]
        
         | ambivalence wrote:
         | If you want real irony here, it's this: powering an app called
         | Threads with Python that's got a global interpreter lock which
         | makes threading hard.
         | 
         | Get it? Threads, threading...? OK, I'll see myself out.
        
           | hynek wrote:
           | I guess PEP 703 has to be auto-accepted now. I don't make the
           | rules.
        
           | nerdponx wrote:
           | Meta does have their own higher-performance JIT-compiled
           | implementation of Python, but I don't know if it still uses a
           | global interpreter lock.
        
             | sho_hn wrote:
             | ... wait, what? Did this morph out of HHVM? Any links on
             | this?
        
               | rat9988 wrote:
               | Click on the link, it's linked inside.
        
               | timeon wrote:
               | Seems to be behind login.
        
               | nerdponx wrote:
               | https://github.com/facebookincubator/cinder
        
           | schnable wrote:
           | Now that is Meta!
        
           | [deleted]
        
           | pookha wrote:
           | Gotten burned by that. Have a python utility that calls
           | something that's very CPU intensive (GDAL) and I literally
           | have to wrap this python process in java for real
           | concurrency. Not a python expert by any means but I'd love to
           | know what other people do when this happens. I'm assuming
           | it's something revolving around containers and messaging
           | (blech)...
        
             | antod wrote:
             | I thought Python calling a native lib was the one time you
             | could multithread your way around the GIL if you were cpu
             | limited? Are the GDAL bindings not releasing the lock or
             | something?
        
             | ska wrote:
             | Is your problem that the c part of GDAL isn't concurrent
             | and you are calling it multiple times, or are you blocking
             | on something you shouldn't?
             | 
             | Python calling a concurrent c thing is super common. For
             | more complex things there is multiprocessing, but that
             | essentially is simplification of IPC.
        
               | pookha wrote:
               | Had a massive amount of images that I was trying to
               | process for a specific purpose. I wound up spinning up
               | multiple python processes within a java wrapper using
               | concurrent linked queues. Got the job done but it's not
               | the proudest moment of my life.
        
               | seunosewa wrote:
               | Why did you need the Java wrapper? You can orchestrate
               | stuff like that within Python. The multiprocessing
               | module.
        
               | giantrobot wrote:
               | You could have done this with the multiprocessing module.
               | It interfaces like the threading library but it's
               | launching new processes for each "thread". I've done a
               | shitload of image processing in Python with it. You'll
               | end up with a 100% Python version of what you did.
        
         | mihaic wrote:
         | Why downvote this? It's a fair point.
        
           | RamblingCTO wrote:
           | Because it's not ironic. It's obvious. Twitter is just a
           | medium to post stuff. Threads is as well. I can post this on
           | mastodon even. Also: Europe still can't join in on the fun.
        
             | Dah00n wrote:
             | EU != Europe. Biggest country in Europe is Russia. Biggest
             | population is Russians.
        
               | ffhhttt wrote:
               | Just like America is not just the US. But nobody cares...
        
             | user_named wrote:
             | I'm in Europe and just downloaded the apk. All my Chinese
             | friends have joined too.
        
           | ambivalence wrote:
           | A fair point about what? Threads is out for less than a day,
           | I have 8 followers there so far. I originally posted this
           | information on Mastodon, actually:
           | https://mastodon.social/@ambv/110665808046065754
           | 
           | However, I have a much larger following on Twitter, so it was
           | the tweet that got picked up by someone here on HN.
           | 
           | We'll see how this looks like in twelve months.
        
             | sanderjd wrote:
             | A fair point that it's stupid to implement a mostly plain
             | text publishing site that doesn't support linking to any of
             | the text published there.
             | 
             | Edit: Someone kindly pointed out that you can see the
             | content in the browser without an account from following a
             | direct link. (You just can't have the app installed or it
             | will open that and drop you on the login page.) This makes
             | me a lot happier with the product!
        
               | gochi wrote:
               | You can link to it, what are you talking about? An
               | example fetched from a verge article ->
               | https://www.threads.net/t/CuVaOg1PgU4/
        
               | sanderjd wrote:
               | Oh nice! Thanks!
               | 
               | I hadn't seen one of these deep links before I installed
               | the app. After I installed the app they just opened the
               | app but did nothing because I don't have an account. But
               | I just uninstalled the app and now I see. This makes me a
               | lot more positive on this product!
        
               | Descon wrote:
               | That link doesn't work for me - just get a splash page
        
               | amadeuspagel wrote:
               | They don't even have the dotcom?
        
               | mihaic wrote:
               | This is the first time I'm seeing this. I'm surprised
               | they have a .net domain and that the design is so
               | terrible.
               | 
               | They just extracted Instagram comments and threw them in
               | a sea of whitespace. No thought is given to anchoring
               | content by importance or context, nor is there any
               | esthetic apeal to it.
        
               | rsynnott wrote:
               | Linking to it works better than linking to Twitter, in
               | fact, as of this week (Twitter no longer shows replies
               | unless you're logged in...)
        
               | sanderjd wrote:
               | Twitter no longer shows _anything_ unless you 're logged
               | in. Everything just redirects to a login page.
        
               | RoyGBivCap wrote:
               | [dead]
        
               | sillysaurusx wrote:
               | You can't link to threads? By design?
               | 
               | I was seriously considering trying it out, but if there's
               | no search engine visibility, there's no point. Most of
               | the searches for my real name pull up tweets.
               | 
               | Hopefully it's just a missing feature that will be added
               | later. Even TikToks are linkable, and those have very
               | little google visibility.
        
               | cma wrote:
               | I think Facebook/Instagram pioneered login walls for
               | public post content.
        
               | ipaddr wrote:
               | Pay walls have been around longer than facebook
        
               | sanderjd wrote:
               | It seems like you can generate links, but they don't do
               | anything for people who don't have the app (except direct
               | you to install the app). I haven't fiddled with it to see
               | if they've implemented something to make the content
               | visible to search engines. (Which presumably would mean I
               | could see it too with some effort.) But I haven't seen
               | such a thing yet.
               | 
               | Edit: Someone kindly pointed out that you can see the
               | content in the browser without an account from following
               | a direct link. (You just can't have the app installed or
               | it will open that and drop you on the login page.) This
               | makes me a lot happier with the product!
        
               | sillysaurusx wrote:
               | At least there are links. That's a good sign. TikTok
               | survived on mostly that model, though it's still
               | important to make the content visible without installing
               | an app.
        
               | sanderjd wrote:
               | Check my edit: It turns out this does work for deep links
               | to threads and user profiles (the homepage is still
               | entirely useless) as long as you _don 't_ have the app
               | installed. Definitely much better than I thought.
        
               | cbsks wrote:
               | I am able to view links to Threads from a web browser
               | without being logged in:
               | https://www.threads.net/t/CuVaOg1PgU4/
        
         | Asooka wrote:
         | Threads is a mobile-only app, like Instagram. You can't
         | actually use Threads on a desktop browser, so that's probably
         | why this is shared as a Twitter post. Even if it were a Threads
         | post, it would not be submittable to HN.
        
           | sanderjd wrote:
           | Right. But this demonstrates why that's dumb. People can't
           | even share press about their app without using a different
           | product.
           | 
           | Edit: Someone kindly pointed out that you can see the content
           | in the browser without an account from following a direct
           | link. (You just can't have the app installed or it will open
           | that and drop you on the login page.) This makes me a lot
           | happier with the product!
        
           | eatonphil wrote:
           | Not exactly true. They have a read-only browser view.
           | 
           | In fact a Thread thread has already been on the front of HN.
           | 
           | https://news.ycombinator.com/item?id=36609632
        
             | superq wrote:
             | > Not exactly true.
             | 
             | Still, _mostly_ true.. read-only is just dumb. Nice to be
             | back in 2008, when mobile-first turned into mobile-only.
             | 
             | The entire point of this thread was that this is a
             | Python/Django web app, but it's not actually a web _app_ at
             | all, so what 's the point of pointing out any of this?
        
               | [deleted]
        
             | HarHarVeryFunny wrote:
             | I see these https://www.threads.net/t/xxx links to specific
             | threads, but is there any top level page where you can
             | search for stuff or browse ? https://www.threads.net just
             | has a QR code (to install the app presumably).
        
               | eatonphil wrote:
               | You can get to users profiles too and view their posts:
               | https://www.threads.net/@zuck.
               | 
               | But there's no broader browser yet it seems.
        
           | SECProto wrote:
           | > Threads is a mobile-only app, like Instagram
           | 
           | Instagram can be used in a desktop browser just fine.
        
             | hotpotamus wrote:
             | Let's imagine that I don't have an Instagram account
             | because I don't. (I never really understood it because it
             | just felt like a slightly differently flavored Facebook and
             | I already had a Facebook account.) Up until this weekend, I
             | could simply view a Twitter post regardless of whether I
             | had an account. And while I've never been much of a Twitter
             | user, I did find that useful as a lot of local news and
             | weather type stuff would get posted and I could easily view
             | them. Is there any analogy to that with Threads?
        
               | Grazester wrote:
               | You can view single IG posts without an account.
        
               | uxp100 wrote:
               | You could view a tweet if linked to it, but it would
               | limit your ability to view a feed or navigate elsewhere.
               | Which is similar to instagram, but insta is more
               | aggressive with stopping you, whereas twitter gave a
               | slightly larger grace period.
               | 
               | Twitter removed that behavior under Elon, then developed
               | something even more restrictive. Great.
               | 
               | Neither are good, and unfortunate that tumblr has
               | recently ish copied this (for default css at least). Real
               | shithead behavior.
               | 
               | Unfortunately, Facebook looks the best out of all of
               | them. If a restaurant posts their specials on Facebook, I
               | can see them. Basically not true for other social sites.
        
           | lloeki wrote:
           | It also launched "in more than 100 countries" but none of the
           | the E.U. ones.
           | 
           | So until it either solves its privacy issues of becomes part
           | of the fediverse, audience is crippled.
        
           | snarkyturtle wrote:
           | Chill it's been in development for like 6 months and they
           | wanted it to scale to 10M users. I'm sure a web interface is
           | on the roadmap.
        
             | superq wrote:
             | Six months and it still doesn't have a basic and open web
             | interface. Not impressive. Many apps, including HN, operate
             | for years with only a web interface.
        
           | tredre3 wrote:
           | Instagram launched in 2010 (but really gained traction only
           | in 2011). It has been available via the web since 2012 but it
           | took until 2015 for the web version to reach feature parity
           | with the app version (namely posting and discovery).
           | 
           | Based on that, I think it's safe to assume that a full
           | featured web version of Threads should be up within a year.
           | Threads already has some web presence (you can view threads
           | and profiles).
        
         | mrits wrote:
         | Ironic people on MySpace talked about Facebook
        
         | smrtinsert wrote:
         | Or, "Threads" and the GIL maybe? Love Python though.
        
         | melling wrote:
         | Is the person from Meta? Doesn't appear so.
         | 
         | It's still perfectly acceptable for Twitter users to discus
         | other platforms on Twitter.
        
           | ctur wrote:
           | He was and worked on, among other things, Python while there.
           | He's a trustworthy source.
        
             | melling wrote:
             | So, no he doesn't work at Meta. We could have stopped this
             | thread an hour ago.
             | 
             | We need a social network that removes the noise from the
             | conversation
        
         | no_wizard wrote:
         | Their web app is not there. I think you can directly link to a
         | Thread but it only shows what you linked to, not even the
         | discussion tree or anything like that.
         | 
         | Its not open to the web yet. Supposedly, it will be
        
       | KolmogorovComp wrote:
       | I will probably be downvoted for this joke but I cannot resist. I
       | hope they use Threads over async/await.
        
       | beardedman wrote:
       | So glad we're condensing entire tech stacks down to a language
       | now.
        
       ___________________________________________________________________
       (page generated 2023-07-06 23:01 UTC)