[HN Gopher] Python's new t-strings
___________________________________________________________________
Python's new t-strings
Author : tambourine_man
Score : 534 points
Date : 2025-04-21 04:31 UTC (18 hours ago)
(HTM) web link (davepeck.org)
(TXT) w3m dump (davepeck.org)
| gnabgib wrote:
| Big discussion (414 points, 10 days ago, 324 comments)
| https://news.ycombinator.com/item?id=43647716
| damnitbuilds wrote:
| Maybe it's good to come back to a discussion after a few days,
| more informed and with a cooler head ?
|
| ;-)
| oulipo wrote:
| Nice, so it's a kind of "limited DSL" inside python that's easy
| to extend
| pauleveritt wrote:
| The PEP originally portrayed this as "for DSLs"
| avereveard wrote:
| what's the tldr difference between this and .format ?
| jitl wrote:
| It's injection safe and compostable, and the resulting object
| retains the original values you interpolate in. This makes it
| useful for building SQL queries with prepared arguments for
| example.
| pwdisswordfishz wrote:
| Compostable? The Green New Deal has gone too far.
| earnestinger wrote:
| It's custom .format implementation. (You access the placeholder
| and value and produce the string)
| hk__2 wrote:
| It's on a meta level: instead of formatting the string, it
| returns an object that contains both the format string and its
| argument. Library author can then implement whatever format
| function they want, for example one that escapes the
| interpolated strings.
| OJFord wrote:
| f-strings are syntactic sugar for .format, e.g.:
| f"foo is {foo} and {bar=}" "foo is {} and
| bar={}".format(foo, bar)
|
| are equivalent.
|
| t-strings are actually not strings, but Template objects,
| giving access to both the templating string and the parameters
| for processing. Sibling comments describe it as a custom
| .format implementation in that sense - it's f-string-like sugar
| where you're also allowed to take control of the .format
| function that it's sugar for.
| zahlman wrote:
| f-strings are actually translated into formatting and
| concatenation of the pieces at a bytecode level; they don't
| call `.format` under the hood and thus aren't what I'd call
| "syntactic sugar" in the traditional sense. But yes, the
| equivalence holds.
|
| t-strings, of course, (will) translate at the bytecode level
| into instantiation of the Template object (and whatever code
| is needed to compute the Interpolation values in-place).
| enescakir wrote:
| Not sure about introducing yet another string prefix. Between
| f-strings, raw strings, and i18n stuff, it's already getting
| crowded. Curious how readable this will be in large codebases.
| wodenokoto wrote:
| I'm of the opposite opinion. Let's set the prefixes free!
| from sql import sql query = sql"SELECT user_id,
| user_name FROM {user_table_versioned} WHERE user_name =
| {user_name}"
| dmurray wrote:
| How would this be different from a function sql() that
| operates on one of these new t-strings?
|
| The syntactic sugar of changing it from sql(t"...") doesn't
| seem particularly valuable. The novel thing about t-strings
| is that they change the parsing at compile-time.
| stavros wrote:
| It wouldn't be different, but it would be more convenient
| because we no longer have to count the number of %s, you'd
| put the variable inline.
| masklinn wrote:
| That's... already the case of t-strings?
| stavros wrote:
| Yes, that's my point. The GP was already talking about a
| t-string.
| masklinn wrote:
| dmurray was comparing a hypothetical sql"..." with
| sql(t"..."). There are no %s either way.
| Timwi wrote:
| > The syntactic sugar of changing it from sql(t"...")
| doesn't seem particularly valuable.
|
| It's valuable because:
|
| - IDEs could then syntax-highlight SQL inside of SQL
| strings and HTML inside of HTML strings
|
| - You can't accidentally pass an HTML string to your SQL
| library
| wodenokoto wrote:
| It's different from a function the same way f"" is
| different from f("") and t"" is different from t("")
|
| There's nothing stopping you from building a Python
| function that parses a string looking for {} and then
| searching globals for those variables. And you can extend
| that to also do some code execution and formatting.
|
| To me the real sugar of f-strings is that the editor knows
| that it's a template and not just a string. Expanding this
| to having SQL and regex syntax highlighting, linting and
| code formatting inside my Python code is a pretty cool
| prospect.
| mcintyre1994 wrote:
| This is how JavaScript does it with tagged template literals.
|
| Your sql there would just be a function that receives the
| array of strings/values and returns whatever.
| masklinn wrote:
| This was considered and rejected:
| https://peps.python.org/pep-0750/#arbitrary-string-
| literal-p...
| dsego wrote:
| This is what JS does with tagged template literals.
| https://github.com/dsego/sql_tag
| mulmboy wrote:
| Are there string prefixes for i18n stuff?
| Biganon wrote:
| They're probably talking about the convention of using _ as
| an alias for `translate`
| albert_e wrote:
| "Yet another" is not my main worry
|
| The concept of prefixes itself feels a little deviant from
| readable code that is close to human language -- which is the
| spirit of Python
| Timwi wrote:
| The single letter f or t does make it unnatural to read, but
| if it were sql"..." or html"...", I think that would help
| with that.
| empiko wrote:
| Additionally, it will probably be confusing that it is called
| a t-string but it is actually a constructor for a Template
| object and not string at all. I would rather see a new
| special term `template` than this.
| toxik wrote:
| Should have been a keyword. a = template
| "foo {bar}"
|
| As should raw and format.
| nezirus wrote:
| Maybe this could be useful to libraries like psycopg3 to use
| something more simple/natural instead of this:
|
| https://www.psycopg.org/psycopg3/docs/api/sql.html
|
| (while I also agree it gets crowded with yet another string
| prefix)
| dtech wrote:
| That's the exact use case. Basically these are syntactic sugar
| for a very similar function signature.
| pizza wrote:
| Looks useful for embedding interpreters
| damnitbuilds wrote:
| I enjoy f-strings, I guess some people need these.
|
| And I love Python but, having been through 2->3 ( occasionally
| still going through it! ) whenever I see a new language feature
| my first thought is "Thank goodness it doesn't break everything
| that went before it".
| stavros wrote:
| Yeah but it's been 17 years, maybe it's time to put the PTSD
| behind us. We're almost at a point where the current generation
| of programmers wasn't even programming when that happened.
| rglullis wrote:
| > We're almost at a point where the current generation of
| programmers wasn't even programming when that happened
|
| I've been programming with Python since 2006, I think most of
| the systems were based on 2.4 at the time. I've been one of
| those who switched to Python 3 somewhat late, waiting for
| some major libraries to ship python 3 packages - celery and
| Twisted were one of the biggest holdouts - so I remember that
| the first project where all my dependencies were ready for
| python 3 was around 2015.
|
| This is to say: even seasoned developers who were
| conservative around the migration have spent more time
| working with Python 3 than Python 2. There simply is no
| reason anymore to be talking about python 2.
| kstrauser wrote:
| The last time I touched a large Py2 project was in 2018
| when I ported it to Py3. So, I have 18 years of Py2,
| probably 6 years of overlap, and 7 years of pure Py3. That
| means I still have a lot more Py2 than Py3 time.
|
| Buuuttt, I'm so over the transition. It's ancient now and I
| agree that we can stop fretting about it.
| nightfly wrote:
| Python2 code didn't disappear when Python3 came out. At my
| work we're _still_ occasionally having to help people migrate
| code that was written for python2
| damnitbuilds wrote:
| Also my experience, alas.
|
| We are not completely _Post_ Traumatic Python2 Stress yet,
| I am afraid.
|
| Bad decisions can have looong-term repercussions.
| nhumrich wrote:
| We're at a point where the current generation of programmers
| weren't even _alive_ when that happened.
| pansa2 wrote:
| Yes, Python 3.0 was released 17 years ago. But the
| transition from Python 2.x was only completed with 2.7's
| end-of-life, 5 years ago.
| int_19h wrote:
| "It's still supported" is a strange metric for this. I
| mean, ActiveState still provides Python 2.7 builds with
| (paid) support.
| eichin wrote:
| And Ubuntu ESM got used as an excuse/"life support" for
| python 2 via 16.04 until horrifyingly recently. (With a
| layer of "you can still get ESM for 14.04, we're not
| _that_ far behind " :-)
| m2f2 wrote:
| If this is just for sql queries ... it'd be overkill especially
| where you need to compare the usual PREPARE statements with the
| hassle of keeping everyone on 3.14 and above.
| bazoom42 wrote:
| Could also be used to prevent html injection.
| nhumrich wrote:
| It's for SQL, HTML, and shell. But idk how solving injection, a
| top on the OWASP list forever is considered "overkill".
| orthoxerox wrote:
| It's also for logging: log.debug(f"The value
| of counter was {counter}, the nonce was {nonce}")
|
| builds a new string every time the interpreter hits this line.
| Whereas log.debug(t"The value of counter was
| {counter}, the nonce was {nonce}")
|
| passes a Template to the debug() function that bails out if
| debug mode is not on and doesn't build a string.
| 7734128 wrote:
| Sure, this avoids issues with SQL injections. However, I have a
| hard time imagining any developer who would both make such
| fundamental errors with f-strings currently and also switching to
| this option when it ships.
|
| Seems like a self selection which renders this meaningless, to
| some extent :/
| masklinn wrote:
| > However, I have a hard time imagining any developer who would
| both make such fundamental errors with f-strings currently and
| also switching to this option when it ships.
|
| t-strings are a different type (both static and dynamic),
| f-strings are not. So t-strings can be mandated at the API
| level, forcing the developer into "proper" usage.
|
| That is, you need third-party tools to differentiate between
| some_call("safe literal string")
|
| and some_call(f"unsafe dynamically created
| string")
|
| That is not the case when it comes to t-strings, `some_call`
| can typecheck internally that it got a t-string, and reject
| regular strings entirely.
|
| Although some space probably needs to be made for composing
| t-strings together in case of e.g. runtime variation in items
| or their count. Facetting for instance. I don't know if that's
| a native affordance of t-strings.
| Timwi wrote:
| But that would require any SQL library you're currently using
| to make the breaking change of no longer allowing strings.
| baggiponte wrote:
| sqlalchemy doesn't really accepts strings - if you do, you
| need to pass them into a "conn.execute(text(...))", so end
| users should not face a breaking change.
| nhumrich wrote:
| Yep. Probably worth it. You could also do this with a
| monkeypatch method to "opt in" to this change.
| masklinn wrote:
| Yes?
| sanderjd wrote:
| Yes. That's exactly what will happen, over time.
| sanderjd wrote:
| Eventually, you won't be able to pass a string into the
| commonly-used DB libraries.
| runekaagaard wrote:
| It feels a bit like "cheating" that new x-string features are
| built-in only. It would be cool to be able to do:
| from foo import bar bar"zoop"
| masklinn wrote:
| A t-string is a literal for a Template object which is a data
| holder, it doesn't actually do anything, so you would simply
| call bar(t"zoop")
| Timwi wrote:
| True. Then you could use sql"..."
| html"..."
|
| for each of the given examples and achieve some runtime type
| safety.
| jbverschoor wrote:
| And you'd end up with almost no improvement.
|
| If you pass a "t-string" to a framework, it can force
| escaping.
|
| What you suggest is to rely on escaping by the user (dev),
| who, if he was aware, would already escape.
|
| Unless you'd suggest that it would still return a template,
| but tagged with a language.
| mcintyre1994 wrote:
| FWIW the JS equivalent is a template but tagged with a
| language. It has all the benefits of this template, but
| IDEs can easily syntax highlight the string. That seems
| like it would be a bit trickier to do with the Python one
| which is a shame.
| thund wrote:
| Use a function? bar("zoop")
|
| It's just syntax, like we used to have print
| "foo"
|
| that later became print("foo")
| cb321 wrote:
| This is _exactly_ how Nim is. The f-string like equivalent uses
| a macro called "fmt" which has a short alias "&". So you can
| say: import std/strformat let world =
| "planet" echo &"hello {world}"
|
| The regular expression module does a similar thing with a
| `re"regular expression"` syntax or std/pegs with peg"parsing
| expression grammar" and so on. There are probably numerous
| other examples.
|
| In general, with user-defined operators and templates and
| macros, Nim has all kinds of Lisp-like facilities to extend the
| language, but with a more static focus which helps for both
| correctness and performance.
| nhumrich wrote:
| This was the original proposed idea in the PEP (750), but it
| changed overtime. There is a section in the PEP to explain why
| it changed to t-strings if you are interested.
| zahlman wrote:
| PEP 638 has always seemed to me like something of a
| generalization of the idea. But that really feels to me like
| a 4.0 feature, or rather something that needs to be designed
| for from the beginning. (Which is why I've done a bit of that
| in my own mind...)
| dec0dedab0de wrote:
| meh, the difference between bar"zoop" and bar("zoop") isn't
| really big enough to be worth it.
|
| I like F strings a lot, but for the most part I think all of
| the various X-strings should just be classes that take a string
| as an argument.
| zahlman wrote:
| In my own language design (nothing public yet - need to
| prioritize more practical things at the moment) this is
| definitely on the menu. Aside from a minimal set, keywords are
| implemented as compile-time macros; and it's intended that they
| can be extended, both "natively" and in the implementation
| language, by writing AST-manipulation code. But the handling of
| arithmetic expressions, as well as the broader line/block
| structure, is hard-coded. (I draw inspiration from both Python
| and the Lisp family.)
| TekMol wrote:
| Will this allow neat SQL syntax like the following?
| city = 'London' min_age = 21 # Find all users in
| London who are 21 or older: users = db.get(t'
| SELECT * FROM users WHERE city={city} AND
| age>{min_age} ')
|
| If the db.get() function accepts a template, it should, right?
|
| This would be the nicest way to use SQL I have seen yet.
| jbaiter wrote:
| Thanks, I hate it. While it's nice syntactic sugar, the
| difference between an SQL injection vulnerability and a
| properly parametrized query is now a single letter that's
| easily missed
| baggiponte wrote:
| Type checkers to the rescue ahaha I think db.get could also
| raise if the type does not match?
| TekMol wrote:
| I guess that is a misunderstanding on your side, about how
| templates work. Less hate and more love might help to avoid
| this type of hotheaded misconception ;-)
|
| Why do you think changing a letter would cause a
| vulnerability? Which letter do you mean?
| hyperbovine wrote:
| OP is referring to swapping t with f.
| TekMol wrote:
| That would result in a string passed to get() and raise
| an error as get() operates on a template, not on a
| string.
| baegi wrote:
| except if get() can also accept a raw string, which is
| likely
| orphea wrote:
| Why would it?
| mcintyre1994 wrote:
| No sane library is going to do that. If they do let you
| pass a raw string it should be a different function with
| the risks clearly documented.
|
| The thing this replaces is every library having their own
| bespoke API to create a prepared statement on their
| default/safe path. Now they can just take a template.
| crazygringo wrote:
| How about every library that wants to preserve backwards
| compatibility?
|
| Or are you suggesting that e.g. every database module
| needs to implement a new set of query functions with new
| names that supports templates? Which is probably the
| correct thing to do, but boy is it going to be ugly...
|
| So now you'll have to remember never to use 'execute()'
| but always 'execute_t()' or something.
| WorldMaker wrote:
| You don't have to remember it, you can use deprecation
| warnings and lint tools to remind you. (Until eventually
| the safe API is the only API and then you really have
| nothing to remember.)
| mcintyre1994 wrote:
| I'd assume their current safe function isn't taking a
| string, and is taking some sort of prepared statement? So
| they could have it take either their prepared statement
| or a template, and deprecate their prepared statement.
|
| If a library has functions taking a string and executing
| it as SQL they probably shouldn't make that take a
| template instead, but I'd hope that's a separate
| explicitly unsafe function already.
| crazygringo wrote:
| For sqlite3, it absolutely takes a regular string.
|
| If you want to substitute parameters, you put a '?' in
| the string for each one, and provide an additional
| (optional) tuple parameter with the variables.
|
| So no, there's no explicitly unsafe function. That's my
| point.
| mcintyre1994 wrote:
| Gotcha. I'd guess they'd want to deprecate that function
| and create a new one that only accepts a template then,
| which is definitely annoying! I figured they'd already
| have more separation between prepared and raw strings
| which would make it easier.
| codesnik wrote:
| f'' vs t'' probably.
| tannhaeuser wrote:
| Wow that's only slightly better than using the lowercase
| letter L vs the digit 1 or letter O vs zero to convey a
| significant difference.
| melodyogonna wrote:
| Those are two different types
| JimDabell wrote:
| The t-string produces a Template object without a __str__()
| method. You can't mistakenly use an f-string in its place.
| Either the code expects a string, in which case passing it a
| Template would blow it up, or the code expects a Template, in
| which case passing it a string would blow it up.
| politelemon wrote:
| And I'm guessing lots of code will expect strings to
| maintain backward compatibility.
| Mawr wrote:
| I'm guessing no existing functions will be extended to
| allow t-strings for this very reason. Instead, new
| functions that only accept t-strings will be created.
| xorcist wrote:
| There's an obvious risk here, same as with strcpy (no,
| strncpy.. no, strlcpy... no, strcpy_s) that documentation
| tends to outlive code, and people keep pasting from
| tutorails and older code so much that the newer
| alternatives have a hard time cutting through the noise.
|
| I would argue that as bad as some w3schools tutorials
| were, and copying from bad Stackoverflow answers, going
| back to MSA and the free cgi archives of the 90s, the
| tendency of code snippets to live on forever will only be
| excarbated by AI-style coding agents.
|
| On the other hand, deprecating existing methods is what
| languages do to die. And for good reason. I don't think
| there's an easy answer here. But language is also
| culture, and shared beliefs about code quality can be a
| middle route between respecting legacy and building new.
| If static checking is as easy as a directive such as "use
| strict" and the idea that checking is good spreads, then
| consesus can slowly evolve while working code keeps
| working.
| sanderjd wrote:
| It's pretty common for Python libraries to deprecate and
| remove functionality. It makes people mad, but it's a
| good thing, for this reason.
| tubthumper8 wrote:
| Do the python type checkers / linters / whatever have the
| ability to warn or error on calling certain functions?
| That would be nice to eventually enforce migration over
| to the newer functions that only take a t-string template
| mb5 wrote:
| They sure do, e.g.
| https://docs.astral.sh/ruff/rules/pandas-use-of-dot-is-
| null/
| mos_basik wrote:
| Yeah. A while back I was poking through some unfamiliar
| code and noticed that my editor was rendering a use of
| `datetime.utcnow()` as struck through. When I hovered it
| with my mouse, I got a message that that function had
| been deprecated.
|
| Turns out my editor (vscode) and typechecker (pyright)
| saw that `datetime.utcnow()` was marked as deprecated (I
| know one can use the `@deprecated` decorator from Python
| 3.13 or `__future__` to do this; I think it was done
| another way in this particular case) and therefore
| rendered it as struck through.
|
| And it taught me A) that `utcnow()` is deprecated and B)
| how to mark bits of our internal codebase as deprecated
| and nudge our developers to use the new, better versions
| if possible.
| sanderjd wrote:
| I think it's way more likely that existing libraries will
| introduce new methods that use t-strings and are type
| safe, rather than entirely defeat the purpose of having a
| t-string API.
| b3orn wrote:
| If it's not a completely new library written exclusively
| around templates, such code currently accepts strings and
| will most likely continue to accept strings for backwards
| compatibility.
| yk wrote:
| That's entirely implementation dependent. For existing
| libraries I would expect something like
| def get(self, query): if isinstance(query,
| template): self.get_template(query)
| else: self.get_old(query) #Don't break old
| code!
| ewidar wrote:
| it would likely be safer to have a safe (accepting
| Templates) and an unsafe (accepting strings) interface.
|
| Now whether maintainers introduce `getSafe` and keep the
| old behavior intact, or make a breaking change to turn
| `get` into `getUnsafe`, we will see
| darthwalsh wrote:
| And they could add deprecation warnings gradually
| solatic wrote:
| That would be a great argument if Python wasn't a language
| that let you reach into the internals and define __str__()
| for things you shouldn't be defining it for. And that is
| something people will _definitely_ do because, you know,
| they just need something to friggin work so they can get
| whatever ticket closed and keep some metric happy tracking
| time-to-close
| scott_w wrote:
| Programmers being lazy and shit at their jobs is not a
| reason to not improve the language.
| crazygringo wrote:
| > _or the code expects a Template, in which case passing it
| a string would blow it up._
|
| That's where the problem is though -- in most cases it
| probably won't blow up.
|
| Plenty of SQL queries don't have any parameters at all.
| You're just getting the number of rows in a table or
| something. A raw string is perfectly fine.
|
| Will sqlite3 really disallow strings? Will it force you to
| use templates, even when the template doesn't contain any
| parameters?
|
| You can argue it should, but that's not being very friendly
| with inputs, and will break backwards compatibility. Maybe
| if there's a flag you can set in the module to enable that
| strict behavior though, with the idea that in a decade it
| will become the default?
| WorldMaker wrote:
| db.execute(t"Select Count(1) from someTable")
|
| It's one extra letter to "force" for an unparameterized
| query over a "raw string". The t-string itself works just
| fine without parameters.
|
| There's definitely a backwards compatibility hurdle of
| switching to a template-only API, but a template-only API
| doesn't look that much "less friendly" with inputs, when
| the only difference is a `t` before every string,
| regardless of number of parameters.
| crazygringo wrote:
| Sure, but it's just I don't have to do that anywhere
| else.
|
| I never put an f in front of a string if I'm not putting
| variables within it.
|
| And I'm generally used to Python inputs being liberal. I
| can usually pass a list if it expects a tuple; I can pass
| an int if it expects a float; often I can pass an item
| directly instead of a tuple with a single item. Regex
| functions take regular strings or regex strings, they
| don't force regex strings.
|
| Being forced to use a single specific type of string in
| all cases is just very different from how Python has
| traditionally operated.
|
| It's safer, I get that. But it's definitely less
| friendly, so I'll be curious to see how module
| maintainers decide to handle this.
| scott_w wrote:
| Er... that's just not correct? Python can be more liberal
| but it's not always. It depends entirely on the tooling.
| Libraries will take time to catch up but I can definitely
| see people creating libraries that enforce t-strings,
| even if they're deconstructing them under the hood for
| legacy libraries.
| crazygringo wrote:
| What's not correct? Python inputs usually _are_ liberal.
| I didn 't say always.
|
| Are you claiming it's traditionally common in Python to
| be strict with inputs, and that being liberal is the
| exception?
| scott_w wrote:
| That Python lets you blindly interchange different types
| for no good reason. It simply doesn't.
|
| Yes, it's common for Python to be strict for inputs when
| the types are different. For example, try:
|
| Decimal('3.0') / 1.5
|
| You'll get an error and for good reason.
| crazygringo wrote:
| But... it usually _does_. For example, try:
| Decimal('3.0') / 2
|
| It works fine. It _doesn 't_ work with a float, which
| _is_ for good reason. That 's the whole point -- its
| general philosophy is to be pretty liberal with types,
| except when there's a good reason not to be. Heck, you
| can even do dumb things like: 4 + True
|
| And get 5 back. If that's not "blindly interchanging
| different types for no good reason" then I don't know
| what is. You can even multiply your Decimal object by
| False and get an answer...
|
| Or it's like my original example -- the Regex module
| isn't restricted to r-strings. It happily works with
| regular strings. Python's general philosophy is to handle
| input liberally. Even type hinting is an add-on. Now, it
| doesn't go as far as JavaScript in allowing e.g. "4"+1,
| but it's still awfully liberal. I just don't see how you
| can claim otherwise.
| scott_w wrote:
| I know about Decimal/int mixing but that's for a good
| reason: it's fine to intermix here. But not for floats
| (precision issues). The bool/int mixing isn't "good."
| It's a bad implementation detail that Python is stuck
| maintaining forever. I'm actually stunned that you think
| to use this as an example when I think I'd fire any
| programmer that did that in my team for gross negligence.
|
| The reason it works is because Python functionally has no
| bool type. True and False are just integers with names.
| It's stupid and shouldn't work like that but it does for
| historic reasons.
|
| Your example of regex makes no sense either. There is no
| difference between strings and r-strings. They're
| literally the same thing to the interpreter, so how could
| the regex functions enforce you use r-strings? Maybe they
| should be different but, for historic reasons, they can't
| be without Python 4.0.
| crazygringo wrote:
| > _I'm actually stunned that you think to use this as an
| example when I think I'd fire any programmer that did
| that in my team for gross negligence._
|
| You seem to be having a different conversation than I am.
|
| I'm just describing Python _as it is_. I 'm not defending
| it. I know _why_ you can add True to a number, or else I
| wouldn 't have come up with the example. And I know
| perfectly well that r-strings are just strings. Python
| easily could have made them a distinct object, to force
| people from ever making backslash errors, and restricted
| Regex functions to them, _but didn 't_.
|
| My only point has been, "Pythonic" things tend to be
| pretty liberal in what they accept. Type hints aren't
| even enforced, when they exist at all. You seem to think
| it _shouldn 't_ be that way. Great! But regardless,
| claiming it's _not_ that way -- that Python is somehow
| this strict language -- is just mischaracterizing it.
| scott_w wrote:
| > My only point has been, "Pythonic" things tend to be
| pretty liberal in what they accept
|
| Being able to use a string as a string and an int as an
| int are not "pretty liberal in what they accept," it's
| just programming language theory 101! I think you're
| mistaking duck typing for "liberal acceptance," which are
| not the same thing. There's always been an expectation
| that you should use compatible interfaces, even within
| the standard library. I've been bitten enough times by
| passing a generator in when a function expects a list,
| for example.
| crazygringo wrote:
| I'm not mistaking it at all. Yes, duck typing is very
| much liberal acceptance, but Python code tends to go much
| farther. I could give a million examples -- like how in
| Python's isinstance() the second argument can be a type
| or a tuple of types, or in sqlite3 that you can run
| queries on a connection _or_ on a cursor, and don 't even
| get me started on Matplotlib or Numpy. It's just
| idiomatic in Python to make things easy for the
| programmer by accepting multiple types when possible. If
| you don't recognize this as the common Python pattern, I
| literally don't know what else to tell you.
| 0cf8612b2e1e wrote:
| I never put an f in front of a string if I'm not putting
| variables within it.
|
| Linters will even complain if you have a f string without
| variables. I assume it will be the same for t strings.
| davepeck wrote:
| For the reasons discussed above, I'm not _sure_ that it
| will be the case for t-strings. I think it 'll take a
| little while for frameworks/libraries to adapt (while
| still maintaining backward compatibility) and a while for
| best practices to find their way into our linting and
| other tools.
| 0cf8612b2e1e wrote:
| If you can use a string anywhere you can use a t-string,
| then a non parametrized t-string is a code smell (lining
| error). If there is a dedicated template-string API, then
| there is the implicit threat you are breaking backwards
| compatibility to stop using regular strings.
| davepeck wrote:
| > If you can use a string anywhere you can use a t-string
|
| You can't; they're different types. t-strings are _not_
| `str`
|
| It's up to good framework/API design to take advantage of
| this.
| WorldMaker wrote:
| Yeah, "t-string" is possibly a misnomer, because they are
| in fact at runtime a Template object (from
| string.templatelib).
| 0cf8612b2e1e wrote:
| A library writer ultimately has to decide if they accept
| both types. For a database cursor, do you take regular
| strings + parameter arguments and template strings? Or
| dedicate a new API to the idea?
| cursor.execute("select * from x where foo=?", {foo=1})
| # while also allowing cursor.execute(t"select *
| from x where foo={foo}") #Vs
| cursor.executetemplate("select * from x where foo={foo}")
|
| If 'execute' takes string and t-string, then I would
| consider it a problem to use a t-string without
| parameters. If there is a novel API just for t-strings,
| then you are implying widespread breaking changes as you
| have a schism between the two ways of providing
| parameters.
| maleldil wrote:
| Linters complain because f"hello" and "hello" are the
| _exact_ same string. t"hello" isn't even a string.
| WorldMaker wrote:
| > Being forced to use a single specific type of string in
| all cases is just very different from how Python has
| traditionally operated.
|
| Maybe that's partly the disconnect here? "t-string" is
| probably a confusing colloquial name because they aren't
| strings, they are Templates. The runtime type is a
| Template. It is a very different duck-type from a string.
| As a duck-typable object it doesn't even implicitly or
| explicitly act like a string, there's intentionally no
| `__str__()` method and `str(someTemplate)` doesn't work
| like you'd expect. It shouldn't be a surprise that there
| is also no implicit conversion _from_ a string and you
| have to use its own literal syntax: it isn 't a string
| type, it's a Template type.
|
| Python here is still liberal with respect to Templates
| (it is still a duck type). If a function expects a
| Template and you don't want to use the t"" shorthand
| syntax nor use the Template constructor in
| string.templatelib, you just need a simple class of
| object that has an `__iter__()` of the correct shape
| and/or has `strings` and `values` tuples.
|
| Sure, it may make sense for some types of APIs to support
| a Union of str and Template as "liberal" options, but
| it's a different class of liberal support from Union of
| list and tuple or Union of int and float which are closer
| "domains" of types. A Template isn't a string and at
| runtime looks nothing like one (despite how syntactically
| it looks like one at "compile time"). Given `__iter__()`
| in Template, it may make more sense/would be more
| "natural" to Union Template with List or Tuple more than
| with a single string.
| yxhuvud wrote:
| Also I wonder how easy it will be to shoot oneself in the
| foot. It may be easy to accidentally make it to a string too
| soon and not get the proper escapeing.
| scott_w wrote:
| That's a library author problem, so it's less likely since
| library authors tend to be fewer in number and, for popular
| libraries, get a reasonable number of eyes on this type of
| change.
| fweimer wrote:
| The SQLite extension for Tcl offers something similar:
| db1 eval {INSERT INTO t1 VALUES(5,$bigstring)}
|
| https://sqlite.org/tclsqlite.html#the_eval_method
| wizzwizz4 wrote:
| As I understand, that's less powerful, because you can do:
| t"INSERT INTO mytable VALUES ({s}, {s[::-1]})"
|
| but you can't do: mydb eval {INSERT INTO
| mytable VALUES ($s, [string reverse $s])}
|
| Instead, you have to write: set t [string
| reverse $s] mydb eval {INSERT INTO mytable VALUES
| ($s, $t)}
|
| There's no reason you _couldn 't_ have such power in Tcl,
| though: it's just that the authors of SQLite didn't.
| meander_water wrote:
| This definitely seems like one of the motivations for
| implementing this feature in the first place -
| https://peps.python.org/pep-0750/#motivation.
|
| Having more control over the interpolation of string values is
| a win IMO.
| bombela wrote:
| Yes, it's quite delightful.
| neonsunset wrote:
| > This would be the nicest way to use SQL I have seen yet.
|
| EF/EF Core has existed for years :)
|
| https://learn.microsoft.com/en-us/ef/core/querying/sql-queri...
| nurettin wrote:
| I've used it for years. In order to generate the models you
| had to use the visual designer which was slow and buggy.
|
| Generally annoying experience if you have to clock in and out
| every day to watch that UI break your database relations
| whenever you click save.
| neonsunset wrote:
| No one uses it today, or in the last 5 years or so I
| presume. You use https://learn.microsoft.com/en-
| us/ef/core/modeling/#use-data...
|
| This was a completely separate, legacy extension of VS, not
| EF let alone EF Core.
| nurettin wrote:
| Completely separate is pushing it since it was
| recommended by Microsoft, but yes, I am old and times
| have changed.
| mcintyre1994 wrote:
| That's the sort of thing people have built with the equivalent
| feature in JavaScript, so it should do. Eg
| https://github.com/andywer/squid is a nice example.
| zelphirkalt wrote:
| Isn't the actually proper way to use prepared statements
| anyway? If we are doing that properly, then what does this t
| string business buy us for SQL usage from Python?
| scott_w wrote:
| Because, as the article states, people aren't using prepared
| statements. Instead, they pass f-strings because they're more
| convenient.
| hedgehog wrote:
| f strings are syntax rather than a type, the resulting
| templates look like a reasonable way to specify a prepared
| statement.
| vultour wrote:
| Except to maintain backwards compatibility we're probably
| going to get new methods that only accept templates,
| completely circumventing any effort to stop people passing
| in strings.
|
| Prepared statements were the recommended way to run SQL
| queries when I was starting with PHP 15 years ago, anyone
| writing code vulnerable to SQL injection at this point
| should not be writing code.
| scott_w wrote:
| Well yes but the alternative is to never make language
| improvements because legacy code exists.
| zahlman wrote:
| You would need triple-quotes to span multiple lines, but yes,
| that is exactly how it's intended to work. `db.get` will
| receive a Template instance, which stores string parts
| something like `('\n SELECT * FROM users\n WHERE city=', ' AND
| age>', '\n')` and interpolated parts like
| `(Interpolation('London'), Interpolation(21))`. It's then
| responsible for assembling and executing the query from that.
| sgt wrote:
| Will it be a performance boost if Django's template engine
| started using t-strings internally?
| pwdisswordfishz wrote:
| Why would mere syntax make a difference?
| sgt wrote:
| No, I mean that if Django's template engine internally
| switched over its implementation to enable templating (as a
| functionality) to use these new t-strings. If t-strings are
| more highly optimized within Python, it could mean a
| performance boost.
| sashank_1509 wrote:
| Why does this need to be a language feature. This could just be a
| separate library, we could use brackets instead of a letter
| before a string. I fear, Python is going down the path of C++
| jsnell wrote:
| It being a language feature gives you controlled access to the
| lexical scope, such that the template string can refer to
| variables by name rather than having to pass each value as a
| parameter. Doing it via parameters is repetitive and/or error-
| prone.
| linsomniac wrote:
| You _CAN_ get access to the calling scope of a function.
| Something like: current_frame =
| inspect.currentframe() env =
| current_frame.f_back.f_locals.copy()
|
| I did it in uplaybook so that you can do things like:
| for module in ['foo', 'bar', 'baz']:
| ln(path="/etc/apache2/mods-enabled", src="/etc/apache2/mods-
| available/{{ module }}.load")
|
| This is an ansible-like tooling with a python instead of YAML
| syntax, hence the Jinja2 templating.
| dsego wrote:
| Because JS has it in the form of tagged template literals.
| chrisrodrigue wrote:
| We have a handful of ways to create strings now.
|
| Meanwhile, pytest is still not part of the standard library.
| perlgeek wrote:
| If it's not a language feature, there's always a risk of
| fragmentation. Some people won't use it because it adds another
| dependency, that means fewer programmers will be familiar with
| it. Others will come up with their own, slightly incompatible
| implementation. See for example Perl and its various Object
| Orientation frameworks (Moose, Mouse, Moo, Mo, M, Mojolicious
| comes with its own...)
|
| Other languages have a policy of prototyping such things out of
| core, and only adding it to the core language if it gains
| traction. Of course that works better if the language has a
| mechanism for extending the syntax out of core.
| dec0dedab0de wrote:
| putting it in the standard library would handle those issues.
| Mawr wrote:
| It's not as simple as "more features" == "closer to C++".
| Features are not equal to each other in terms of impact on
| language complexity.
|
| t-strings don't interact with anything else in the language;
| they, as you yourself pointed out, could almost be an isolated
| library. That makes them low impact.
|
| This is also true syntactically; they're just another type of
| string, denoted by "t" instead of "f". That's easy to fit into
| one's existing model of the language.
|
| Moreover, even semantically, from the point of view of most
| language users, they are equivalent to f-strings in every way,
| so there's nothing to learn, really. It's only the library
| writers who need to learn about them.
|
| Then we have to consider the upsides - the potential to
| eliminate SQL and HTML injection attacks. The value/cost is so
| high the feature a no-brainer.
| nhumrich wrote:
| This feature actually can't be a library. It needs to be able
| to preserve the string before the variables are passed in,
| which a library would be unable to do because f-strings
| immediately replace all the values. The whole premise of
| t-strings is to know which values where "hard coded" and which
| ones are variables. A library can't possibly know that. And a
| function call wouldn't have access to the local variables. The
| only way to do this without language support is to pass
| `locals()` into every function call.
| dec0dedab0de wrote:
| Now you have me wondering how difficult it would be to have a
| class that takes a string and parses through globals to find
| the context it was called from. maybe causing an exception
| and abusing a traceback? or maybe we just find our own
| objectid.... gahh I have to try it now, but I'm setting a
| timer.
| dec0dedab0de wrote:
| It was easier than I thought, but also took longer than I
| wanted to. turns out the inspect module provides what you
| need to pull it off.
|
| This dummy example splits a string that it was given, then
| if one of those values is in the callers context it saves
| those in self.context, and has an output function to
| assemble it all together. Obviously this example is not
| very useful, but it shows how a library could do this as a
| class or function without the user having to pass in
| locals(). import inspect class
| MyXString: """ will split on whitespace and
| replace any string that is a variable name with
| the result of str(variable)""" def
| __init__(self, string): self.string =
| string caller_locals =
| inspect.currentframe().f_back.f_locals
| self.context = {} for key in
| set(string.split()): if key in
| caller_locals: self.context[key] =
| caller_locals[key] def output(self):
| output = self.string for k,v in
| self.context.items(): output =
| output.replace(k,str(v)) return output
| nhumrich wrote:
| This looks like a performance nightmare, and would likely
| never have IDE integration. So I guess I was wrong. It
| could be a library. But will be much better supported
| officially.
| zephyrfalcon wrote:
| You don't have to go through all the globals, you just have
| to get the caller's namespace, which is fairly simple. See
| e.g. https://stackoverflow.com/a/6618825/27426
|
| For this reason, I think it's not true that this absolutely
| had to be a language feature rather than a library. A
| template class written in pure Python could have done the
| same lookup in its __init__.
| dec0dedab0de wrote:
| That is exactly what I ended up doing:
|
| https://news.ycombinator.com/item?id=43752889
| sanderjd wrote:
| Counterpoint: It's good to add well designed and useful
| features to programming languages.
| toxik wrote:
| So should f strings now go away? They are just a special case of
| t strings.
|
| Also, don't get me started on g strings.
| perlgeek wrote:
| > So should f strings now go away?
|
| No
|
| > They are just a special case of t strings.
|
| Not really, because they produce a string right away instead of
| a template.
| nhumrich wrote:
| No. F-strings should still be the default. T-strings aren't
| usable directly, and don't have a `str` like API, so they
| aren't even compatible, intentionally.
| est wrote:
| https://peps.python.org/pep-0750/#approaches-to-lazy-evaluat...
|
| name = "World"
|
| template = t"Hello {(lambda: name)}"
|
| This looks cool
| fph wrote:
| How do these interact with i18n? Can I load a translated t-string
| with `_(t"")` from a .po file? Can it include variable names and
| arbitrary code inside lambdas?
| nhumrich wrote:
| I suggest you read the PEP, there is a section on i18n. Short
| version is it's not designed for this use case.
|
| As for variables and arbitrary code/lambdas, yes: t-strings can
| do that, just like f-strings
| serbuvlad wrote:
| All things considered, this is pretty cool. Basically, this
| replaces db.execute("QUERY WHERE name = ?",
| (name,))
|
| with db.execute(t"QUERY WHERE name = {name}")
|
| Does the benefit from this syntactic sugar outweigh the added
| complexity of a new language feature? I think it does in this
| case for two reasons:
|
| 1. Allowing library developers to do whatever they want with {}
| expansions is a good thing, and will probably spawn some good
| uses.
|
| 2. Generalizing template syntax across a language, so that all
| libraries solve this problem in the same way, is probably a good
| thing.
| pinoy420 wrote:
| Now instead of being explicit all it takes is someone
| unfamiliar with t strings (which will be almost everyone -
| still few know about f strings and their formatting
| capabilities) to use an f instead and you are in for a bad
| time.
| falcor84 wrote:
| That is an issue, but essentially it boils down to the
| existing risk of unknowledgeable people not escaping
| untrusted inputs. The solution should be more education and
| better tooling (linters, SAST), and t-strings are likely to
| help with both.
| masklinn wrote:
| t-strings allow building APIs which don't accept strings at
| all (or require some sort of opt-in), and will always error
| on such. That's the boon.
|
| Having to write cr.execute(t"...")
|
| even when there's nothing to format in is not a big
| imposition.
| mcintyre1994 wrote:
| Any sane library will just error when you pass a string to a
| function that expects a template though. And that library
| will have types too so your IDE tells you before you get that
| far.
| Dx5IQ wrote:
| Such library functions tend to also accept a string as a
| valid input. E.g. db.execute from the GP usually works with
| strings to allow non-parametrized SQL queries.
| kccqzy wrote:
| The library should just refuse strings. If a non
| parametrized query is desired, it could require the user
| to supply a t-string with no {}.
| _Algernon_ wrote:
| This would break backwardcompatibility pretty hard. In many
| cases it may not be worth it.
| eichin wrote:
| But now at least the language has the necessary rope (and
| an opportunity for a cultural push to insist on it.)
| hombre_fatal wrote:
| Javascript already has prior art here.
|
| A library can extend an existing database library like
| 'pg' so that PgClient#query() and PgPool#query() require
| string template statements.
|
| That way 'pg' can continue working with strings, and
| people who want nice templated strings can use the small
| extension library, and the small extension library makes
| it impossible to accidentally pass strings into the query
| functions.
| sanderjd wrote:
| No, because they don't return a string, so good library
| authors will raise a type error when that happens, for
| exactly this reason.
| hackrmn wrote:
| I suppose lack of overlap in the "interface surface"
| (attributes, including callables) between `str` and
| `Template` should nip the kind of issue in the bud -- being
| passed a `Template` and needing to actually "instantiate" it
| -- accessing `strings` and `values` attributes on the passed
| object, will likely fail at runtime when attempted on a
| string someone passed instead (e.g. confusing a `t`-string
| with an `f`-string)?
| Tenoke wrote:
| I don't see what it adds over f-string in that example?
| evertedsphere wrote:
| safety against sql injection
| ds_ wrote:
| The execute function can recognize it as a t-string and
| prevent SQL injection if the name is coming from user input.
| f-strings immediately evaluate to a string, whereas t-strings
| evaluate to a template object which requires further
| processing to turn it into a string.
| Tenoke wrote:
| Then the useful part is the extra execute function you have
| to write (it's not just a substitute like in the comment)
| and an extra function can confirm the safety of a value
| going into a f-string just as well.
|
| I get the general case, but even then it seems like an
| implicit anti-pattern over doing db.execute(f"QUERY WHERE
| name = {safe(name)}")
| ubercore wrote:
| Problem with that example is where do you get `safe`?
| Passing a template into `db.execute` lets the `db`
| instance handle safety specifically for the backend it's
| connected to. Otherwise, you'd need to create a `safe`
| function with a db connection to properly sanitize a
| string.
|
| And further, if `safe` just returns a string, you still
| lose out on the ability for `db.execute` to pass the
| parameter a different way -- you've lost the information
| that a variable is being interpolated into the string.
| Tenoke wrote:
| db.safe same as the new db.execute with safety checks in
| it you create for the t-string but yes I can see some
| benefits (though I'm still not a fan for my own codebases
| so far) with using the values further or more complex
| cases than this.
| ubercore wrote:
| Yeah but it would have to be something like
| `db.safe("SELECT * FROM table WHERE id = {}", row_id)`
| instead of `db.execute(t"SELECT * FROM table WHERE id =
| {row_id}")`.
|
| I'd prefer the second, myself.
| Tenoke wrote:
| No, just `db.execute(f"QUERY WHERE name =
| {db.safe(name)}")`
|
| And you add the safety inside db.safe explicitly instead
| of implicitly in db.execute.
|
| If you want to be fancy you can also assign name to
| db.foos inside db.safe to use it later (even in execute).
| ZiiS wrote:
| But if someone omits the `safe` it may still work but
| allow injection.
| thunky wrote:
| Same is true if someone forgets to use t" and uses f"
| instead.
|
| At least db.safe says what it does, unlike t".
| ewidar wrote:
| Not really, since f"" is a string and t"" is a template,
| you could make `db.execute` only accept templates, maybe
| have
|
| `db.execute(Template)` and `db.unsafeExecute(str)`
| thunky wrote:
| agreed. but then you're breaking the existing
| `db.execute(str)`. if you don't do that, and instead add
| `db.safe_execute(tpl: Template)`, then you're back to the
| risk that a user can forget to call the safe function.
|
| also, you're trusting that the library implementer raises
| a runtime exception if a string a passed where a template
| is expected. it's not enough to rely on type-
| checks/linting. and there is probably going to be a
| temptation to accept `db.execute(sql: Union[str,
| Template])` because this is non-breaking, and sql without
| params doesn't need to be templated - so it's breaking
| some stuff that doesn't need to be broken.
|
| i'm not saying templates aren't a good step forward, just
| that they're also susceptible to the same problems we
| have now if not used correctly.
| fwip wrote:
| Your linter can flag the type mismatch, and/or the
| function can reject f"" at runtime. This is because t""
| yields a Template, not a str.
|
| Template is also more powerful/concise in that the
| stringify function can handle the "formatting" args
| however it looks.
|
| Note also, that there's no requirement that the template
| ever become a str to be used.
| sanderjd wrote:
| This is just extra boilerplate though, for what purpose?.
|
| I think one thing you might be missing is that in the
| t-string version, `db.execute` is not taking a string; a
| t-string resolves to an object of a particular type. So
| it is doing your `db.safe` operation, but automatically.
| panzi wrote:
| Of course you can write code like that. This is about
| making it easier not to accidentally cause code injection
| by forgetting the call of safe(). JavaScript had the same
| feature and some SQL libraries allow only the passing of
| template strings, not normal strings, so you can't
| generate a string with code injection. If you have to
| dynamically generate queries they allow that a parameter
| is another template string and then those are merged
| correctly. It's about reducing the likelihood of making
| mistakes with fewer key strokes. We could all just write
| untyped assembly instead and could do it safely by paying
| really good attention.
| Izkata wrote:
| The first one already exists like:
| db.execute("SELECT * FROM table WHERE id = ?", (row_id,))
| NewEntryHN wrote:
| Some SQL engines support accepting parameters separately
| so that values get bound to the query once the abstract
| syntax tree is already built, which is way safer than
| string escapes shenanigans.
| ljm wrote:
| I'd always prefer to use a prepared statement if I can,
| but sadly that's also less feasible in the fancy new
| serverless execution environments where the DB adapter
| often can't support them.
|
| For me it just makes it easier to identify as safe,
| because it might not be obvious at a glance that an
| interpolated template string is properly sanitised.
| Mawr wrote:
| But you have to remember to call the right safe()
| function every time: db.execute(f"QUERY
| WHERE name = {name}") db.execute(f"QUERY
| WHERE name = {safe_html(name)}")
|
| Oops, you're screwed and there is nothing that can detect
| that. No such issue with a t-string, it cannot be
| misused.
| dragonwriter wrote:
| > and an extra function can confirm the safety of a value
| going into a f-string just as well.
|
| Yes, you could require _consumers_ to explicitly sanitize
| _each_ parameter before it goes into the f-string, or,
| because it has the structure of what is fixed and what is
| parameters, it can do _all_ of that for _all_ parameters
| when it gets a t-string.
|
| The latter is far more reliable, and you can't do it with
| an f-string because an f-string after creation is just a
| static string with no information about construction.
| zahlman wrote:
| > Then the useful part is the extra execute function you
| have to write
|
| Well, no, the library author writes it. And the library
| author also gets to detect whether you pass a Template
| instance as expected, or (erroneously) a string created
| by whatever formatting method you choose. Having to use
| `safe(name)` within the f-string loses type information,
| and risks a greater variety of errors.
| burky wrote:
| f-strings won't sanitize the value, so it's not safe. The
| article talks about this.
| Tenoke wrote:
| The article talked about it but the example here just
| assumes they'll be there.
| sanderjd wrote:
| What do you mean by "they"? You mean the template
| interpolation functions?
|
| Yes, the idea is that by having this in the language,
| library authors will write these implementations for use
| cases where they are appropriate.
| Tenoke wrote:
| The sanitization. Just using a t-string in your old
| db.execute doesn't imply anything safer is going on than
| before.
| masklinn wrote:
| Using a t-string in a db.execute which is not compatible
| with t-strings will result in an error.
|
| Using a t-string in a db-execute which is, should be as
| safe as using external parameters. And using a non-t-
| string in that context should (eventually) be rejected.
| Tenoke wrote:
| Again, just because a function accepts a t string it
| doesn't mean there's sanitization going on by default.
| tikhonj wrote:
| Yes, but if a function accepts a _template_ (which is a
| different type of object from a string!), either it is
| doing sanitization, or it explicitly implemented template
| support _without_ doing sanitization--hard to do by
| accident!
|
| The key point here is that a "t-string" isn't a string at
| all, it's a new kind of literal that's reusing string
| syntax to create Template objects. That's what makes this
| new feature fundamentally different from f-strings. Since
| it's a new type of object, libraries that accept strings
| will either have to handle it explicitly or raise a
| TypeError at runtime.
| Tenoke wrote:
| I'm not sure why you think it's harder to use them
| without sanitization - there is nothing inherent about
| checking the value in it, it's just a nice use.
|
| You might have implemented the t-string to save the value
| or log it better or something and not even have thought
| to check or escape anything and definitely not everything
| (just how people forget to do that elsewhere).
| sanderjd wrote:
| I really think you're misunderstanding the feature. If a
| method has a signature like: class DB:
| def execute(query: Template): ...
|
| It would be weird for the implementation to just
| concatenate everything in the template together into a
| string without doing any processing of the template
| parameters. If you wanted an unprocessed string, you
| would just have the parameter be a string.
| Tenoke wrote:
| I'm not. Again, you might be processing the variable for
| logging or saving or passing elsewhere as well or many
| other reasons unrelated to sanitization.
| nemetroid wrote:
| Sure, and the safe() function proposed upthread might
| also just be doing logging.
| Ukv wrote:
| The original comment said that it'd replace
| db.execute("QUERY WHERE name = ?", (name,))
|
| with db.execute(t"QUERY WHERE name =
| {name}")
|
| It's true that in theory `db.execute` could ignore
| semantics and concatenate together the template and
| variables to make a string without doing any
| sanitisation, but isn't the same true of the syntax it
| was claimed to replace?
|
| Just because templates (or the previous syntax of passing
| in variables separately) could be used in a way that's
| equivalent safety-wise to an f-string by a poorly
| designed library does not mean that they add nothing over
| an f-string in general - they move the interpolation into
| db.execute where it can do its own sanitization and,
| realistically, sqlite3 and other libraries explicitly
| updated to take these _will_ use it to do proper
| sanitization.
| sanderjd wrote:
| Taking a Template parameter into a database library's
| `execute` method is a big bright billboard level hint
| that the method is going to process the template
| parameters with the intent to make the query safe. The
| documentation will also describe the behavior.
|
| You're right that the authors of such libraries _could_
| choose to do something different with the template
| parameter. But none of them will, for normal interface
| design reasons.
|
| A library author could also write an implementation of a
| `plus` function on a numerical type that takes another
| numerical type, and return a string with the two numbers
| concatenated, rather than adding them together.
|
| But nobody will do that, because libraries with extremely
| surprising behavior like that won't get used by anybody,
| and library authors don't want to write useless
| libraries. This is the same.
| nemetroid wrote:
| Your "old" db.execute (which presumably accepts a regular
| old string) would not accept a t-string, because it's not
| a string. In the original example, it's a _new_
| db.execute.
| masklinn wrote:
| Because t-strings don't create strings, so if the library
| doesn't support t-strings the call can just error.
| teruakohatu wrote:
| If I pass an f-string to a method, it just sees a string. If
| I pass a t-string the method can decide how to process the
| t-string.
| sureglymop wrote:
| Wouldn't this precisely lead to sql injection vulnerabilities
| with f-strings here?
| sim7c00 wrote:
| it makes it so people too lazy to make good types and class
| will be getting closer to sane code without doing sane
| code...
|
| imagine writing a SqL where u put user input into query
| string directly.
|
| now remember its 2025, lie down try not to cry.
| NewEntryHN wrote:
| Assuming you also need to format non-values in the SQL (e.g.
| column names), how does the `execute` function is supposed to
| make the difference between stuff that should be formatted in
| the string vs a parametrized value?
| masklinn wrote:
| Same as currently: the library provides some sort of
| `Identifier` wrapper you can apply to _those_.
| NewEntryHN wrote:
| Fair enough. It would be nice if Python allowed to
| customize the formatting options after `:`
|
| This way you could encode such identifier directly in the
| t-string variable rather than with some "out-of-band"
| logic.
| mcintyre1994 wrote:
| The article does mention that the function receiving the
| template has access to those formatting options for each
| interpolation, so presumably you could abuse the ones
| that are available for that purpose?
| masklinn wrote:
| > Fair enough. It would be nice if Python allowed to
| customize the formatting options after `:`
|
| It does, the `Interpolation` object contains an arbitrary
| `format_spec` string:
| https://peps.python.org/pep-0750/#the-interpolation-type
|
| However I think using the format spec that way would be
| dubious and risky, because it makes the _sink_
| responsible for whitelisting values, and that means any
| processing between the source and sink becomes a major
| risk. It 's the same issue as HTML templates providing
| `raw` output, now you have to know to audit any
| modification to the upstream values which end there,
| which is a lot harder to do than when "raw markup" values
| are reified.
|
| > rather than with some "out-of-band" logic.
|
| It's the opposite, moving it to the format spec is out of
| band because it's not attached to values, it just says
| "whatever value is here is safe", which is generally not
| true.
|
| Unless you use the format spec as a way to signal that a
| term should use identifier escaping rules rather than
| value escaping rules (something only the sink knows), and
| an `Identifier` wrapper remains a way to bypass that.
| pphysch wrote:
| > Unless you use the format spec as a way to signal that
| a term should use identifier escaping rules rather than
| value escaping rules (something only the sink knows)
|
| This should be quiet common in the SQL applications. It
| will be nice to write t"select {name:id} from {table:id}
| where age={age}" and be confident that the SQL will be
| formatted correctly, with interpolations defaulting to
| (safe) literal values.
| amelius wrote:
| One thing it misses is compile-time checks for e.g. the format
| spec.
| karamanolev wrote:
| Doesn't all of Python miss that, having (close to) no compile
| time?
| amelius wrote:
| Python does some checks before it runs code. E.g.:
| print("hello") def f(): nonlocal
| foo
|
| gives: SyntaxError: no binding for
| nonlocal 'foo' found
|
| before printing hello, and note that f() wasn't even
| called.
| nomel wrote:
| I think it's just giving an error because a valid AST
| can't be made, which means valid bytecode can't be made.
| "<word> <word>" is only valid syntax if one is a reserved
| word. `nonlocal(foo)` is just fine, of course.
| pansa2 wrote:
| > _" <word> <word>" is only valid syntax if one is a
| reserved word._
|
| `nonlocal` is a keyword
| zahlman wrote:
| No, it gives an error because `nonlocal foo` requests
| that the name `foo` be looked up in a closure, but `f`
| doesn't have such a closure (the `foo` defined outside
| the function is global instead). `nonlocal` is the same
| sort of keyword as `global` but for enclosing functions
| instead of the global namespace; see also
| https://stackoverflow.com/questions/1261875 .
| rwmj wrote:
| I did a safe OCaml implementation of this about 20 years ago,
| the latest version being here:
|
| https://github.com/darioteixeira/pgocaml
|
| Note that the variables are safely and correctly interpolated
| at compile time. And it's type checked across the boundary too,
| by checking (at compile time) the column types with the live
| database.
| tasuki wrote:
| Yes, what you did is strictly more powerful than what the
| Python people did. And you did it 20 years ago. Well done,
| have an upvote. And yet, here we are in 2025 with Python
| popularity growing unstoppably and (approximately) no one
| caring about OCaml (and all the other languages better than
| Python). It makes me sad.
| sanderjd wrote:
| Network effects are a beast!
|
| But my two cents is that we're pretty lucky it's python
| that has taken off like a rocket. It's not my favorite
| language, but there are far worse that it could have been.
| psychoslave wrote:
| You mean like Cobol? Oh wait!
| rwmj wrote:
| I'm switching between C, OCaml, Python, bash & Rust roughly
| equally every day (to a lesser extent, Perl as well). Not
| everything is what gets on the front page of HN.
| skeledrew wrote:
| It's interesting how the majority has explicitly chosen NOT
| to use the "better" languages. Is the majority really that
| bad in their judgment? Or is it that "better" is actually
| defined by adoption over time?
| daedrdev wrote:
| It's clearly better in their opinion, they just aren't
| optimizing for the same metrics that you are. Python is
| better because it's easy for people to learn, imo.
| throwawaymaths wrote:
| its not easy to learn. its a challenge even getting it
| installed and running. what even is a venv? how do you
| explain that to a beginner?
|
| python is popular because its what teachers teach.
| acdha wrote:
| You don't need to teach it to a beginner. The first of
| learning doesn't need more than the standard library and
| when you need more than that you're either giving them
| the single command necessary to run or, more likely,
| having them use a template project where a tool like
| Poetry is doing that automatically.
|
| What this usually hits isn't that managing Python
| packages is hard in 2025 but that many people do not
| learn how their operating system works conceptually until
| the first time they learn to program and it's easy to
| conflate that with the first language you learn.
| zahlman wrote:
| On modern Linux you can type `python` at the command
| prompt and get a REPL. On Windows you download an
| installer from the official website (just like one
| usually does to install anything on Windows), then use
| `py` at the command prompt.
|
| You don't need to `import` anything to start teaching
| Python. Even then you can do quite a lot with the
| standard library. Even then, unless you're using 3.11 or
| later on Linux you can let Pip install with `--user`
| until you actually need to isolate things between
| projects. (And even with new Python on Linux, the
| instructor can typically avert this by just installing a
| separate Python in `/usr/local/bin` for example. Yes,
| that's "cheating", depending on the classroom
| environment. But that's part of the point: installation
| hurdles are hurdles for self-learners, not for students.)
|
| You only _need_ to learn about virtual environments once
| you have projects with mutually conflicting dependencies,
| and /or once you're at a point where you're ready to
| publish your own software and should be learning proper
| testing and development practices. (Which will be largely
| orthogonal to _programming_ , and not trivial, in any
| language.)
|
| And when your students do get to that point, you can give
| them a link such as
| https://chriswarrick.com/blog/2018/09/04/python-virtual-
| envi... .
|
| Teachers teach Python because it's easy to teach while
| still being relevant to the real world, in particular
| because boilerplate is minimized. You don't have to
| explain jargon-y keywords like "public" or "static" up
| front. You don't have to use classes for quite some time
| (if ever, really). You can express iteration naturally.
| Types are naturally thought of in terms of capabilities.
|
| In my mind, Python has all the pedagogical advantages of
| Lisp, plus enough syntactic cues to prevent getting "lost
| in a sea of parentheses". (Of course, it lacks plenty of
| other nice Lisp-family features.)
| daedrdev wrote:
| someone learning python as their first language knows so
| little its perfectly fine to let them pollute their
| global environment. Someone who knows other languages can
| understand what venv is for.
|
| Instead they can type python to open a shell and use
| python to immediately run their file.
| jyounker wrote:
| It has become successful largely because it has always
| had really good foreign function interface. If you have a
| scientific or mathematical library laying around in C,
| then you could wire it up to Python, and then suddenly
| you have all the flexibility of a (fairly clean)
| scripting language to orchestrate your high speed C.
|
| Good examples of this are numpy and tensorflow.
| psunavy03 wrote:
| If someone is challenged figuring out a venv and they're
| not an absolute beginner, perhaps they aren't cut out to
| work in technology. There are countless subjects in the
| field more challenging and complicated to wrap one's
| brain around.
|
| Also, in 2025, just use uv.
| dhruvrajvanshi wrote:
| I think you're being too unfair. People aren't dumb.
|
| It's also about how much better.
|
| Beyond a decent enough type system, the advantages start
| to flatten and other factors start to matter more.
|
| Can't speak too much for python, but as someone who's
| written large amounts of code in OCaml and Typescript,
| the strictest compiler options for Typescript are good
| enough.
| tetha wrote:
| Or you could use this in a library like sh with
| sh(t"stat {some_file}")
|
| With t-strings you could run proper escaping over the contents
| of `some_file` before passing it to a shell.
|
| I'd have to take a look at the order things happen in shell,
| but you might even be able to increase security/foot-gun-
| potential a little bit here by turning this into something like
| `stat "$( base64 -d [base64 encoded content of some_file] )"`.
| nhumrich wrote:
| You should check out PEP 787
| tetha wrote:
| Hmm, PEP-787 has some interesting discussions around it.
| I'll have to sort my thoughts on these aspects a bit.
| pauleveritt wrote:
| We really should just point most of these comments at that
| PEP. Thanks for getting it out so fast.
| zahlman wrote:
| Oh! I missed this one because I've been looking
| specifically at the Packaging forum rather than the PEPs
| forum. This looks like a brilliant use case. (I'm aiming
| for wide compatibility - back to 3.6 - with my current
| projects, but I look forward to trying this out if and when
| it's accepted and implemented.)
|
| Now if only the overall `subprocess` interface weren't so
| complex....
| dhruvrajvanshi wrote:
| Not Python but this is exactly the idea behind zx
|
| https://github.com/google/zx
| mikeholler wrote:
| A potential concern is how close this looks to the pattern
| they're trying to override.
| db.execute(f"QUERY WHERE name = {name}")
|
| versus db.execute(t"QUERY WHERE name =
| {name}")
| fzzzy wrote:
| But won't the f string version fail loudly because there's no
| name parameter?
| benwilber0 wrote:
| the {name} parameter is in the locals() dict like it always
| is
| fzzzy wrote:
| Good point. Perhaps the database api could refuse strings
| and require Templates.
| bshacklett wrote:
| That's a big breaking change around a brand new feature.
| I'm sure it could be done well, but it gives me the
| shivers.
| daedrdev wrote:
| much better would be execute_template(t"...")
| notatoad wrote:
| The key point is that t-strings are not strings.
| Db.execute(t"...") would throw an exception, because t"..."
| is not a string and cannot be interpreted as one.
|
| In order for a library to accept t-strings, they need to make
| a new function. Or else change the behavior and method
| signature of an old function, which I guess they could do but
| any sanely designed library doesn't do.
|
| Handling t-strings will require new functions to be added to
| libraries.
| gls2ro wrote:
| yes but the bug is writing f instead of t and I assume f
| will just work.
|
| To clarify even more:
|
| The problem is not writing by mistake t instead of f =>
| this is what we want and then for this we implement a new
| function
|
| The problem is writing f instead of t => and this will
| silently work I assume (not a Python dev just trying to
| understand the language design)
| masklinn wrote:
| > The problem is writing f instead of t => and this will
| silently work I assume (not a Python dev just trying to
| understand the language design)
|
| In the fullness of time it has no reason to. Even in the
| worst case scenario where you have to compose the query
| dynamically in a way t-strings can't support, you can
| just instantiate a Template object explicitely.
| benwilber0 wrote:
| Aren't there other benefits to server-side parameter binding
| besides just SQL-injection safety? For instance, using PG's
| extended protocol (binary) instead of just raw SQL strings.
| Caching parameterized prepared statements, etc.
|
| Also: db.execute(t"QUERY WHERE name =
| {name}")
|
| Is dangerously close to: db.execute(f"QUERY
| WHERE name = {name}")
|
| A single character difference and now you've just made yourself
| trivially injectible.
|
| I don't think this new format specifier is in any way
| applicable to SQL queries.
| VWWHFSfQ wrote:
| > I don't think this new format specifier is in any way
| applicable to SQL queries.
|
| Agree. And the mere presence of such a feature will trigger
| endless foot-gunning across the Python database ecosystem.
| masklinn wrote:
| > Aren't there other benefits to server-side parameter
| binding besides just SQL-injection safety? For instance,
| using PG's extended protocol (binary) instead of just raw SQL
| strings. Caching parameterized prepared statements, etc.
|
| All of which can be implemented on top of template strings.
|
| > A single character difference and now you've just made
| yourself trivially injectible.
|
| It's not just a one character difference, it's a different
| _type_. So `db.execute` can reject strings both statically
| and dynamically.
|
| > I don't think
|
| Definitely true.
|
| > this new format specifier is in any way applicable to SQL
| queries.
|
| It's literally one of PEP 750's motivations.
| VWWHFSfQ wrote:
| > It's literally one of PEP 750's motivations.
|
| Python is notorious for misguided motivations. We're not
| "appealing to authority" here. We're free to point out when
| things are goofy.
| willcipriano wrote:
| from string.templatelib import Template def
| execute(query: Template)
|
| Should allow for static analysis to prevent this issue if
| you run mypy as part of your pr process.
|
| That would be in addition to doing any runtime checks.
| benwilber0 wrote:
| The first mistake we're going to see a library developer
| make is: def execute(query: Union[str,
| Template]):
|
| Maybe because they want their execute function to be
| backwards compatible, or just because they really do want
| to allow either raw strings are a template string.
| masklinn wrote:
| > they really do want to allow either raw strings are a
| template string.
|
| I'd consider that an invalid use case:
|
| 1. You can create a template string without placeholders.
|
| 2. Even if the caller does need to pass in a string
| (because they're executing from a file, or t-strings
| don't support e.g. facetting) then they can just... wrap
| the string in a template explicitly.
| woodrowbarlow wrote:
| nitpicking:
|
| > It's not just a one character difference, it's a
| different type. So `db.execute` can reject strings both
| statically and dynamically.
|
| in this case, that's not actually helpful because SQL
| statements don't need to have parameters, so db.execute
| will always need to accept a string.
| anamexis wrote:
| You can just pass it a template with no substitutions.
| masklinn wrote:
| > db.execute will always need to accept a string.
|
| No. A t-string with no placeholders is perfectly fine.
| You can use that even if you have no parameters.
| tczMUFlmoNk wrote:
| > > I don't think
|
| > Definitely true.
|
| The rest of your comment is valuable, but this is just
| mean-spirited and unnecessary.
| rangerelf wrote:
| >> I don't think >Definitely true.
|
| I thought we left middle-school playground tactics behind.
| davepeck wrote:
| > Caching parameterized prepared statements, etc.
|
| I didn't explicitly mention this in my post but, yes, the
| Template type is designed with caching in mind. In
| particular, the .strings tuple is likely to be useful as a
| cache key in many cases.
| MR4D wrote:
| Dang! Thanks for pointing this out.
|
| I had to look SEVERAL times at your comment before I noticed
| one is an F and the other is a T.
|
| This won't end well. Although I like it conceptually, this
| few pixel difference in a letter is going to cause major
| problems down the road.
| pphysch wrote:
| How? tstrings and fstrings are literals for completely
| different types.
|
| CS has survived for decades with 1 and 1.0 being completely
| different types.
| Izkata wrote:
| Because they're both passed to "execute", which can't
| tell between the f-string and a non-interpolated query,
| so it just has to trust you did the right thing. Typoing
| the "t" as an "f" introduces SQL injection that's hard to
| spot.
| vlovich123 wrote:
| Assuming `execute` takes both. You could have
| `execute(template)` and `execute_interpolated(str,
| ...args)` but yeah if it takes both you'll have
| challenges discouraging plain-text interpolation.
| Izkata wrote:
| It would have to be the other way around or be a
| (possibly major) breaking change. Just execute() with
| strings is already standard python that all the
| frameworks build on top of, not to mention tutorials:
|
| https://docs.python.org/3/library/sqlite3.html
|
| https://www.psycopg.org/docs/cursor.html
|
| https://dev.mysql.com/doc/connector-python/en/connector-
| pyth...
| zahlman wrote:
| `execute` _can_ tell the difference, because `t "..."`
| does not create the same type of object that `f"..."`
| does.
| Certhas wrote:
| I had an extended debugging session last week that
| centered on 1 and 1. confusion in a library I have to
| use...
| pphysch wrote:
| Yeah, it's a real bummer when that happens. I wish JSON
| never tried to do types.
| MR4D wrote:
| Reread my comment. It's about noticing you have an "f" or
| a "t" and both are very similar characters.
| rocha wrote:
| Yes, but you will get an error since string and templates
| are different types and have different interfaces.
| Izkata wrote:
| Click "parent" a few times and look at the code example
| that started this thread. It's using the same function in
| a way that can't distinguish whether the user
| intentionally used a string (including an f-string) and a
| t-string.
| zahlman wrote:
| Yes, and the parent is misguided. As was pointed out in
| multiple replies, the library _can_ distinguish whether
| an ordinary string or a t-string is passed because _the
| t-string is not a string instance, but instead creates a
| separate library type_. A user who mistakenly uses an f
| prefix instead of a t prefix will, with a properly
| designed library, encounter a `TypeError` at runtime (or
| a warning earlier, given type annotations and a checker),
| not SQL injection.
| WorldMaker wrote:
| Templates are a very different duck type from strings and
| intentionally don't support __str__(). The SQL tool can
| provide a `safe_execute(Template)` that throws if passed a
| string and not a Template. You can imagine future libraries
| that only support Template and drop all functions that accept
| strings as truly safe query libraries.
|
| > Caching parameterized prepared statements, etc.
|
| Templates give you all the data you need to _also_ build
| things like cacheable parameterized prepared statements. For
| DB engines that support named parameters you can even get the
| interpolation expression to auto-name parameters (get the
| string "name" from your example as the name of the variable
| filling the slot) for additional debugging/sometimes caching
| benefits.
| rastignack wrote:
| Quite easy to detect with a proper linter.
| hombre_fatal wrote:
| You solve that with an execute(stmt) function that requires
| you to pass in a template.
|
| In Javascript, sql`where id = ${id}` is dangerously close to
| normal string interpolation `where id = ${id}`, and db libs
| that offer a sql tag have query(stmt) fns that reject
| strings.
| InstaPage wrote:
| t vs f going to be hard to spot.
| acdha wrote:
| This is true of many other things, which is why we have
| type checkers and linters to be perfectly rigorous rather
| than expecting humans to never make mistakes.
| kazinator wrote:
| But t"..." and f"..." have different types; we can make
| db.execute reject character strings and take only template
| objects.
| zahlman wrote:
| > A single character difference and now you've just made
| yourself trivially injectible.
|
| No; a single character difference and now you get a
| `TypeError`, which hopefully the library has made more
| informative by predicting this common misuse pattern.
| VWWHFSfQ wrote:
| > Allowing library developers to do whatever they want with {}
| expansions is a good thing, and will probably spawn some good
| uses.
|
| I completely disagree with this. Look what happened to Log4J
| when it was given similar freedoms.
| serbuvlad wrote:
| I think this would have solved the log4j vulnerability, no?
|
| As I understand it, log4j allowed malicious ${} expansion in
| any string passed to logging functions. So logging user
| generated code at all would be a security hole.
|
| But Python's t-strings purposely _do not_ expand user code,
| they only expand the string literal.
| int_19h wrote:
| Python is not the first one to get this feature. It's been
| present in JS for some time now, and before that in C# (not
| sure if that's the origin or they also borrowed it from
| somewhere). Python adopted it based in part on successful
| experience in those other languages.
| serbuvlad wrote:
| That's really cool. I don't use JS or C#, so I wasn't aware
| of this, but it's a good idea.
| zahlman wrote:
| 3. It _prevents_ the developer from trying
| db.execute(f"QUERY WHERE name = {name}")
|
| or db.execute("QUERY WHERE name = %s" % name,
| ())
|
| or other ways of manually interpolating the string - because
| `db.execute` can flag a `TypeError` if given a string (no
| matter how it was constructed) rather than a `Template`
| instance.
| franga2000 wrote:
| I wish they added the same thing JS has, where this "string
| literal prefix thingy" can be user-defined.
|
| html`<p>${value}</p>` will actually run the function
| html(template). This means you can use this to "mark" a function
| in a way that can be detected by static analysis. Many editors
| will, for example, syntax highlight and lint any HTML marked this
| way, same with SQL, GraphQL and probably some others too.
| conartist6 wrote:
| For the record the JS thing desugars to the exact same as the
| Python thing, so it is no more or less safe to do the syntax
| highlighting in Python as it is in JS.
| Timon3 wrote:
| It desugars similarly, but the Python version doesn't have a
| name. Any t-string is a t-string, there's no HTML t-string or
| SQL t-string or anything like that. It's just a t-string you
| can pass to a function: html_string =
| t"<something />" sql_string = t"SELECT * FROM
| something"
|
| In JS, the string has a prefix that can differ between
| languages, e.g.: const htmlString =
| html`<something />` const sqlString = sql`SELECT *
| FROM something`
|
| and so on. See the difference?
| masklinn wrote:
| Except your labels are incorrect because neither
| `html_string` nor `sql_string` are strings, they're both
| Template objects, and the sink function is the one which
| processes it. No processing has happened to them by the end
| of the snippet, beyond creating the template object itself.
| Timon3 wrote:
| Sure, choose different variable names, who cares. The
| essential difference is that the language is referenced
| at the declaration site, not the usage site, which makes
| the syntax highlighting far easier.
|
| Please engage with my point instead of criticizing
| trivialities.
| masklinn wrote:
| > Please engage with my point instead of criticizing
| trivialities.
|
| Your complete misunderstanding of what's happening is not
| a triviality.
|
| > The essential difference is that the language is
| referenced at the declaration site, not the usage site,
| which makes the syntax highlighting far easier.
|
| Javascript has no built-in template tags beyond
| `String.raw`. If tooling has the capabilities to infer
| embedded language from arbitrary third party libraries, I
| would hope they have the ability to do utterly trivial
| flow analysis and realise that
| html(t"<something />")
|
| means the template string is pretty likely to be HTML
| content.
| Timon3 wrote:
| Come on, you're just being rude for no good reason. A
| badly chosen variable name doesn't show a "complete
| misunderstanding". Yes, the variable should have been
| named `html_template` instead of `html_string` - how
| often do I have to acknowledge this before you accept it?
|
| And it's obviously more complex to do syntax highlighting
| when the declaration site and usage site are possibly
| split apart by variable assignments etc. Yes, in the case
| you showed syntax highlighting is easy, but what if the
| `html` function takes more parameters, doesn't take the
| template as the first parameter, etc? There's a lot of
| possible complexity that tagged template literals don't
| have. Thus they are easier to do highlighting for. This
| is objectively true.
| conartist6 wrote:
| Tagged template literals in JS have all that complexity.
| All of it. That tools you trust lie and pretend that's
| not the case doesn't make the language spec say anything
| different
| Timon3 wrote:
| No, they literally don't, because they don't support
| these features!
|
| You can't split apart the declaration of the template
| literal and the "tagging". The tag is always _part_ of
| the declaration, which it doesn 't have to be in Python,
| as I've showed.
|
| You can't pass additional parameters to the tag function,
| it's always just the template & values. In Python, you
| can pass as many parameters as you want to the usage
| site, e.g. some_value = html(True,
| t"<something />", 42)
| conartist6 wrote:
| It just sounds like you don't know JS very well, because
| in JS you can definitely split apart the declaration and
| the tag with ease. The thing implementing the tag is just
| a function after all: https://jsfiddle.net/rd2f1kot/
| Timon3 wrote:
| Sorry, but do just not want to understand what I'm
| talking about? Your example doesn't show what you're
| saying it does.
|
| In Python you can do this: bar = "hello
| world" template = t"<something foo={bar} />"
| string = html(template)
|
| This is _simply not possible_ in JS, because the template
| literal always must have the tag attached. You _can 't_
| split them apart. If you try: const bar
| = "hello world" const template = `<something
| foo=${bar} />`
|
| you _already have a string_ in the `template` variable.
| There 's no access to the individual values anymore. It's
| already done. It's a string. No template. You _can 't_
| pull apart the literal declaration and the tagging.
|
| Are we now done with this ridiculous game of deliberate
| misunderstandings?
| conartist6 wrote:
| I just don't see the same thing as you. This is what you
| say is simply not possible: let
| passthrough = (...args) => args; let bar = "hello
| world"; let template = passthrough`<something
| foo=${bar} />`; string = html(...template);
| Timon3 wrote:
| So now we have moved from "tagged template literals have
| all the complexity of Python templates" to "if you define
| a specific template tag and use that, you can replicate
| one feature of Python templates". No mention of the other
| example I mentioned (which, by the way, wasn't an
| exhaustive list).
|
| Now, I'll just ignore that you're _still_ deliberately
| misrepresenting me in the hopes of gaining some actual
| knowledge from this discussion - please show me: which
| editor supports syntax highlighting for the specific
| example you just mentioned? After all, that was the topic
| of discussion - not whether it 's possible to delay
| "tagging", but whether it's easier to do syntax
| highlighting for JS template tags. Please show me an
| existing editor, IDE or git repo for a tool that would
| highlight your specific example based on the
| `html(...template)` call in line 4 (so no heuristics
| based on line 3). Surely you're not just throwing random
| examples at the wall to see what sticks, right? You've
| actually been following the discussion and are arguing in
| the context of it?
| conartist6 wrote:
| Oh now I can reply. Thread continued above:
| https://news.ycombinator.com/item?id=43754585
| conartist6 wrote:
| Sorry for some reason I can't reply to the deeper posts.
| Depth limit I guess. You asked which editors could
| support syntax highlighting once the tag and the content
| that needs to be syntax highlighted are split apart.
| Currently there are none, though I am writing one that
| will be able to.
|
| Python should be able to detect the magic syntactic
| pattern just the same way JS does and use that for syntax
| higlighting. In JS the magic syntactic pattern for
| triggering HTML syntax highlighting is:
| html`<doc/>`
|
| In Python the magic syntactic pattern would be:
| html(t"<doc />")
|
| My point in showing that JS counterexample was to
| demonstrate that the real reason people don't do that
| kind of thing isn't that they _can 't_, it's that they
| like having syntax highlighting. That means the approach
| should work just as well in Python. This is the case even
| though the heuristic involved is very weak. Changing the
| name of the identifier or breaking into multiple
| expressions would be enough to break the heuristic in
| either language, which is why I think it's a really weak
| heuristic and dangerous pitfall for developers who might
| mistake the coloring for information about how the
| runtime sees that data (which is normally what syntax
| highlighting is for)
| Timon3 wrote:
| If I may give you some advice: you could have made this
| point without telling me I don't know JS well, without
| misrepresenting me and without wasting both of our time
| by just _making this point_ from the start. Because you
| are technically correct that, in the limited case of a
| template immediately being passed to a function, syntax
| highlighting can use the same heuristics that are
| currently used in JS. But, as I assumed previously, in
| doing so you 're simply ignoring my point: syntax
| highlighting for tagged template literals isn't
| applicable to only a subset of uses because they are by
| design less complex. Your counter example was using a
| different template tag and not using the actual tag as a
| template tag, which obviously doesn't work in the context
| of syntax highlighting for that template tag.
|
| Had you not played games with this technicality, we could
| both have saved a lot of time. Hope you had fun I guess.
| conartist6 wrote:
| Yes. It's a weak heuristic, but it's EXACTLY the same
| weak heuristic that JS is applying!
|
| IN other words, since custom template tags in JS *are
| literally just function calls* when a JS environment
| syntax highlights the code as HTML it's doing so based on
| an extremely weak heuristic (the identifier for the
| interpolation function is named "html"). Both Python and
| JS have the same problem.
| svieira wrote:
| I think his point would be clearer if we focused on
| typing the usages _statically_. Consider
| `html(this_is_a_sql_template)` vs. `html "SELECT * FROM
| ..."` or `thing_that_consumes_html_template_type(oops_a_s
| ql_template_type)`.
| conartist6 wrote:
| JS references the language at the usage site, _exactly_
| like Python. There is no difference here in how the two
| languages behave.
| Timon3 wrote:
| No, it doesn't, it references the language at the
| declaration site, because the declaration site _always
| is_ the usage site. You can 't split them apart. You
| _can_ split them apart in Python - see the example in my
| first comment.
| masklinn wrote:
| The JS version actually desugars to something much more
| primitive, and less convenient: in JS a template tag receives
| one parameter which is an array of n+1 strings, and then n
| parameters for the interpolated values, and has to iterate
| alternatively on both sequences.
|
| You can do that in python by accessing the `strings` and
| `values`, but I expect most cases will simply iterate the
| template, yielding a unified typed view of literal strings
| and interpolated values.
| nhumrich wrote:
| The PEP was initially proposed this way. But due to various
| reasons, making it an open namespace was considered to
| overcomplicate the language (read: understanding coffee when
| reading it). Alternatively, there doesn't really seem to be
| much loss of ability with t-strings. Libraries can require a
| template as it's accepted type, instead of having to invent
| their own custom type and named template.
| 0xFEE1DEAD wrote:
| Call me a monarchist, but I think Python has changed for the
| worse ever since Guido van Rossum stepped down.
| pansa2 wrote:
| I don't think things would be very different if Guido were
| still BDFL. He's second-author of the t-strings PEP, and has
| been an author of most other major PEPs in recent releases
| (including the walrus operator, PEG parser, and pattern
| matching).
| nhumrich wrote:
| I for one think python has never been better.
| ashwinsundar wrote:
| In what ways has it changed for the worse?
| sanderjd wrote:
| It's so fascinating to see so many people, when faced with a
| simple and useful programming language feature, get all up in
| arms about how it's the end of the world for that language.
|
| I honestly feel like a lot of people just seem bored and
| looking for stuff to be mad about.
| pansa2 wrote:
| > _t-strings evaluate to a new type,
| `string.templatelib.Template`_
|
| > _To support processing, `Template`s give developers access to
| the string and its interpolated values_ before* they are combined
| into a final string.*
|
| Are there any use-cases where processing a Template involves
| something other than (i) process each value, then (ii) recombine
| the results and the string parts, in their original order, to
| produce a new string? In other words, is the `process_template`
| function ever going to be substantially different from this
| (based on `pig_latin` from the article)? def
| process_template(template: Template) -> str: result =
| [] for item in template: if
| isinstance(item, str): result.append(item)
| else:
| result.append(process_value(item.value)) return
| "".join(result)
|
| I haven't seen any examples where the function would be
| different. But if there aren't any, it's strange that the design
| requires every Template processing function to include this
| boilerplate, instead of making, say, a `Template.process` method
| that accepts a `process_value` function.
| rfoo wrote:
| There are a lot of examples about SQL in comments. In the SQL
| case you want something like: def
| process_template(template: Template) -> tuple[str, tuple]:
| sql_parts = [] args = [] for item in template:
| if isinstance(item, str): sql_parts.append(item)
| else: sql_parts.append("?")
| args.append(process_value(item.value)) return
| "".join(sql_parts), tuple(args)
|
| (of course it would be more nuanced, but I hope you get the
| point)
| pansa2 wrote:
| Yes that makes sense, thanks.
|
| Also, my comment was about the amount of boilerplate
| required, but that can be vastly reduced by writing
| `process_template` in a more functional style instead of the
| highly-imperative (Golang-like?) style used in the article.
| The first `process_template` example is just:
| def process_template(template: Template) -> str:
| return ''.join(interleave_longest(template.strings,
| map(process_value, template.values)))
|
| And the second is something like: def
| process_template(template: Template) -> tuple[str, tuple]:
| return (
| ''.join(interleave_longest(template.strings, ['?'] *
| len(template.values))), map(process_value,
| template.values) )
| WorldMaker wrote:
| Templates don't even have to be processed into a string. The
| article shows an example where the Template is processed into
| an HTML mini-DOM. It's maybe not obvious because the DOM object
| is immediately stringified to show sample output, but you could
| imagine manipulating the DOM object in a few more steps before
| stringifying it, or maybe you are running in WASM in a browser
| and using that mini-DOM directly as a Virtual DOM passed to JS
| to work with.
|
| Also, in addition to the other SQL example using "?" to fill in
| the "holes" for parameters in an SQL friendly way, some DBs
| also support named parameters, so the "hole" in the string form
| might be naively replaced with something like
| `f"@{item.expression}"` and that also forms the key in a dict
| to pass as parameters. (You'd want to make sure that the
| expression inside the template is useful as a parameter name,
| and not something more exotic like {1 + 3} or {thing for thing
| in some_list}, in which cases you are probably auto-assigning
| some other parameter name.)
| pauleveritt wrote:
| Nearly everything you just described is being worked on. It's
| amazing how accurately you have described it. We hope to demo
| and explain at PyCon US.
| zahlman wrote:
| >But if there aren't any, it's strange that the design requires
| every Template processing function to include this boilerplate
|
| Other replies gave examples of other use cases. But the neat
| thing about Python is that you _don 't_ need to "include this
| boilerplate" for the common cases. It can be wrapped up in a
| decorator (which could be included in `templatelib`). Or, as
| you say, in a method on the Template class.
|
| I think I'd implement it as a generator, calling
| `process_value` (defaulting to `str`) on the Interpolations, so
| that the caller can still do more with the results (or just
| `''.join` them).
|
| But _these are separate considerations_ ; nothing prevents
| implementing them later, or indeed adding them to the
| implementation before the 3.14 release.
| nu11ptr wrote:
| Personally, this feels like a feature that is too focused on one
| problem to be a general feature. Python is getting huge. When
| people ask me if Python is easy and simple to learn I have to say
| "the basics, yes, but to to learn the whole language... not so
| much".
|
| I feel like in this sense Go really is interesting by rejecting
| almost every single feature. Honestly not sure generics were
| worth it as they add a lot of complexity, and while they are
| nice, I don't need them very much. The general idea to keep the
| language at its original focus is the right idea IMO. C++ would
| be the most extreme case where the language itself barely
| resembles what it started out as.
| murkt wrote:
| This is a pretty simple and useful feature. I wouldn't say that
| it bloats the language too much. Descriptors and metaclasses
| are much more complicated and have a lot more implications and
| have been in the language for a veeeeery long time. Is it
| decades already?
| pansa2 wrote:
| Yeah, Python hasn't been a simple language for a long time,
| if ever. That's probably the biggest misconception about the
| language - that its friendly syntax implies simple semantics.
| It's not true at all.
| tetha wrote:
| I would say python in it's entirety is one of, if not the
| deepest and potentially most complex language I know. C++
| is the other contender. The things you could do with
| metaclasses, multiple inheritance and operator overloading
| are quite staggering.
|
| I'm just glad you don't have to think or even use this as a
| normal user of the language, most of the time or at all.
| nu11ptr wrote:
| This feature is not complicated, but one must keep every
| feature that can possibly be seen in code in their head. Even
| if it is familiar now, what happens when you use the feature
| in the one small section of code where it fits, nowhere else,
| and then read that code 2 years later? This is the problem
| with adding useful features that are only used in a few key
| places. I'm not saying Go is a perfect language, far from it,
| but limiting # of features as a general goal is something
| more languages should strive for IMO.
| murkt wrote:
| I am not arguing against that language ideally should have
| less features. I am arguing with "Python is getting huge",
| because it's huge and has been for many-many years :)
| nu11ptr wrote:
| True - cat is well out of the bag at this point
| oliwarner wrote:
| Python has always been a batteries-included language, so having
| a go at templated string interpolation --a feature other
| languages have had for decades-- seems like a strange gripe.
|
| It's far more essential than little utilities like textwrap or
| goliath packages like Python's bundled tkinter implementation.
| nu11ptr wrote:
| What other languages have this feature? I'm not aware of any
| pansa2 wrote:
| > _[T-strings] are the pythonic parallel to JavaScript's
| tagged templates._
| hocuspocus wrote:
| Scala since 2014.
|
| Java 22 had the feature as preview but it was removed in
| 23, it'll come back after some refinement.
| thatnerdyguy wrote:
| C# has InterpolatedStringHandler which isn't quite the same
| thing, but (in my understanding), trying to address the
| same core issues.
| WorldMaker wrote:
| C# InterpolatedString is very close, with the twisty bit
| being that C# can rely on static typing for safety so the
| "f-string" and "t-string" variants use the same literal
| syntax and depend on what function they are passed to,
| whereas in both Python and Javascript they have different
| literal syntaxes. Python chose to use a different literal
| prefix to its literals ("f" versus "t") and Javascript
| chose to use a function-call syntax as prefix
| (`templateString` versus html`templateString` where html
| is a function in scope).
| neonsunset wrote:
| For the case like here it's closer to FormattableString
| that's what EF Core's composable FromSql method works on
| top of. Both address custom interpolation but from
| different angles / for different scenarios.
| martinky24 wrote:
| They literally discuss this in the article.
| jerf wrote:
| Many languages have similar features.
|
| For instance, Python has the % operator that is a template
| format that allows interpolating values based on a template
| string with a variety of printf-like features:
| https://python-
| reference.readthedocs.io/en/latest/docs/str/f...
|
| Also, Python has the .format method on strings, which is a
| template format that allows interpolating values based on a
| template string: https://www.geeksforgeeks.org/python-
| string-format-method/
|
| As another example, Python has f-strings that are a
| template format that allows interpolating values based on a
| template string: https://www.geeksforgeeks.org/formatted-
| string-literals-f-st...
|
| Also, you can also find languages like Python that have a
| rich ecosystem of third party templating solutions. These
| are often intended for things like rendering entire web
| pages but many of them have relatively simple ways of using
| their templating functionality in a fairly reasonable
| amount of code, if you just want to have a template format
| that allows interpolating values based on a template
| string.
|
| So, as you can see, many other languages have this feature,
| as you can tell from all the examples I have shown you
| here.
|
| (To spell it out for those who may find this too subtle...
| somehow... I'm not a fan of this simply because Python has
| gone from "There should be one-- and preferably only one
| --obvious way to do it." to "there's half-a-dozen ways to
| do it and if they are all wrong Python 3.x+1 will introduce
| a seventh" and I'm just not seeing the benefits worth the
| tradeoffs here.)
| phreeza wrote:
| Batteries included to me also meant a large standard library,
| not a large language.
| sanderjd wrote:
| I'm truly glad that Go exists for people who like languages
| that are simple even to the point of frustration and I hope it
| never changes. But I'm glad that other languages exist for
| those of us for whom learning some syntax is not a barrier and
| having convenient ways to do common things is highly valued.
| acdha wrote:
| There's an interesting trade off around maintenance. Python's
| stdlib means there's more consistency across projects and you
| can assume basic things like error handling which you had to
| check individually on each Go program, which is the kind of
| stuff which adds up when you have a lot of small programs.
|
| This is especially noticeable with AWS Lambda where you can
| have a lot of useful stuff running for years without doing more
| than bumping the runtime version every year or two, but also
| that is one highly opinionated architecture so it's not
| globally optimal for everyone.
| Smithalicious wrote:
| I really was on the side of being generally willing to accept new
| python features, but this is getting ridiculous. What an utterly
| pointless thing to bloat the language with. At this point my
| moving to clojure as my first line language of choice is only
| accelerating.
|
| This is of the category "things I wouldn't want to use even for
| the specific hyper niche things they're intended for". What even
| does a "t-string" represent? Because it's clearly not a string of
| any kind, it's a weird kind of function call notation. The
| programmer sees something that looks like string formatting, but
| the program executes some arbitrary procedure that might not
| return a string whatsoever.
| nhumrich wrote:
| For me, this is the best feature to land in python for 6 years.
| JS has had this and it allows my code to be completely safe
| from SQL injection, which is an absolutely incredible feature,
| given SQL injection has been the #1 vulnerability for a long
| time.
| pauleveritt wrote:
| Thanks Nick for this response and all the time you've spent
| explaining. It's funny, I looked back at the comments on
| f-strings before they landed. They also got similar
| complaints about bloat. And yet, my uneducated guess: very
| popular.
| Smithalicious wrote:
| But this doesn't prevent SQL injection, does it? It adds a
| grammar feature that you can then use to build SQL injection
| prevention following a novel idiom. Someone still needs to
| write the actual SQL building logic somewhere!
|
| I don't think this is the right idiom for doing this. Frankly
| I don't think SQL query generation should look like string
| templating at all!
|
| The sell seems to be "now you can write code that looks like
| an SQL injection vulnerability, without it actually being
| vulnerable!". I'd rather write code that isn't a
| vulnerability, and doesn't look like one, and doesn't have to
| bend the language grammar either.
| stefan_ wrote:
| What? Maybe in 2005. This is just strictly worse than
| parameterized because now you are also wasting time
| "escaping" strings which perpetuates the whole mixing data &
| query thing nobody even wants anymore.
|
| It's like even the one case identified nobody has even
| thought all the way through. Now your SQL library only
| accepts t-strings, I get an obscure error passing in a simple
| static query. Ah yes, put the useless t on it. That sorted,
| now the SQL library escapes all the parameters it wasn't
| previously doing, to then hand the final unique query to the
| actual underlying SQL library which would much rather have
| the parameterized one so it can cache parsing. Jesus.
| sanderjd wrote:
| The "bloat" is that you can now put the letter "t" in front of
| a string, rather than "f" or "r"?
| kccqzy wrote:
| I'm not the OP but I'm guessing with OP moving to Clojure
| that the bloat is basically any special syntax or special
| language feature that basically boils down to a function call
| or a let statement. A lot of functional programming languages
| have minimalistic expression syntax and they are just as
| expressive as OP needs them to be.
| Smithalicious wrote:
| It's a pattern moreso than this specific feature. There's an
| important qualitative distinction for me between something
| that's on the level of a library (even if it's the stdlib) on
| the one hand and first class features (ie things that require
| parser level support) on the other.
|
| Python has historically been very conservative about this but
| in recent years has had one controversial language extension
| after another, while parts of the language that actually need
| love are left to languish IMO.
|
| I wanna be very clear that this is me _changing my mind_ -- I
| was (still am) very on board with the highly controversial
| assignment expressions ( "walrus operator") for instance.
|
| I don't have much faith about what the Python language will
| look like if you project the current rate of changes forward
| 10, 15, 20 years. It really doesn't help that I consider this
| new thing an active antifeature.
| sanderjd wrote:
| I feel like this is begging the question... This t-string
| feature is only controversial inasmuch as comments like
| yours here are criticizing it as being controversial...
| pragma_x wrote:
| Debate around the usefulness aside, are there any linter rules
| for warning about f-strings in light of this? I can easily see
| where mistaking one for the other would cause problems. For
| context, I'm thinking specifically about tools like Black and
| MyPy.
| sanderjd wrote:
| This will be trivial to implement (it is just a check against
| the type that t-strings return), so I'm sure it will be done,
| if it hasn't already.
| xixixao wrote:
| The runtime and static type of the template string is different
| to fstring (Template vs str). Unless you're passing them
| somewhere that accepts strings, you can't go wrong.
|
| I actually quite like the simplicity of this design over tagged
| literals in JS.
| russfink wrote:
| I feel like this can be solved another way. S=f"my good code ####
| {potentially_evil_user_input} #### my good code again" then work
| around the ####. Of course, even better, S=evil_user_input and do
| a scrub on S first.
| morkalork wrote:
| Honestly think this is a more useful feature and elegant solution
| than the walrus operator that was added. Formatting query strings
| has always felt messy especially with different DBs having their
| own non-standard ways of doing it.
| dec0dedab0de wrote:
| Seems pretty neat so far, but I don't understand the motivation
| behind not allowing you to call str(template) and get the
| template as a normal string. I could imagine it being very useful
| to be able to gather up the template itself in a string to do
| stringy things with.
|
| The only reason I could imagine, is if you are trying to protect
| developers from themselves, which kinda goes against the "we're
| all adults here" mentality that makes Python so great. I suppose
| it's easy enough to add that functionality, but come on.
| gaogao wrote:
| The traditional way at the megacorp for something like that is
| unsafe_str_DO_NOT_USE_OR_YOU_WILL_BE_FIRED(template)
| TheRealPomax wrote:
| > If you've worked with JavaScript, t-strings may feel familiar.
| They are the pythonic parallel to JavaScript's tagged templates.
|
| The syntax is template literals, not just "tagged templates".
| Which is a _huge_ difference: template literals still act as real
| strings. They don 't _need_ a tag prefix to work, you have the
| _option_ to tag them if and when needed.
|
| As far as I understand it, t-strings can't do that. They're not
| strings, and you can't even coerce them into strings, you _have_
| to run them through a processor before they become a string. So
| they 're nothing like JS's template literals, they're syntactic
| sugar for forming "an instance of an object that needs to be
| passed into a function that returns a string".
|
| So I don't look forward to folks preferring f-strings over
| t-strings even when they really shouldn't, simply because "having
| to constantly convert them from not-a-string to a string is a
| hassle". If only they'd worked like JS template literals.. that
| would have been _fantastic_.
| haberman wrote:
| TL;DR: like f-strings, all {foo} expressions in the t-string are
| evaluated immediately, but instead of immediately concatenating
| everything into a single result string, the t-string evaluation
| returns a Template object that keeps the interpolation results
| and the surrounding strings separate. This lets subsequent logic
| decide whether the interpolation results need any special
| escaping before concatenating them with the strings around them.
|
| In other words, t-strings are basically f-strings where the final
| concatenation is delayed. And indeed, you can trivially implement
| f-strings using t-strings by performing a simple, non-escaped
| concatenation step: https://peps.python.org/pep-0750/#example-
| implementing-f-str... f'...' -> str
| t'...' -> Template foo(t: Template) -> str
| dheera wrote:
| > This lets subsequent logic decide whether the interpolation
| results need any special escaping before concatenating them
| with the strings around them
|
| This sounds like unnecessary fluff in what was supposed to be a
| simple language. I'm worried Python is turning into C++42 with
| 65535 ways to do one simple thing.
|
| Why not just: f'SELECT * FROM `{esc(table)}`
| WHERE name = "{esc(name)}"'
|
| Nice and simple.
| pansa2 wrote:
| While your code is a valid alternative way to implement
| @haberman's description, the feature is actually much more
| flexible.
|
| The "subsequent logic" has full access to the interpolation
| results and strings. Not only can it escape the results, it
| can do whatever it wants to them. It can also do whatever it
| wants to the strings, and then combine everything in any way
| it likes - it's not even necessary that the final result is a
| string.
| pauleveritt wrote:
| The other PEP example shows generating HTML attributes from
| a passed-in dictionary. HTML has a number of places where
| this is helpful, if you have original data.
| WorldMaker wrote:
| It's easy to forget the `esc` function. How does the
| recipient check (or type check) that it was called in all the
| right places?
|
| Most DBs support parameterized queries which can be cached
| for performance. How do you pick out the parameters from that
| and replace those parts of the strings with the DB's
| parameter placeholders? t'Select * from
| {table} where name = {name}'
|
| Looks very similar, but execution engine has access to all
| the individual parts, making it very easy to add placeholders
| such as: ('Select * from ? where name = ?`,
| table, name)
|
| Or even (if the DB supports it), has access to the
| expressions inside the string and can use named parameters:
| ('Select * from @table where name = @name', { "table": table,
| "name": name })
|
| That's really nice for debugging, depending on your DB
| engine.
|
| In every DB engine that supports it, parameterized SQL is
| even safer than escape syntaxes because parameters are passed
| in entirely different parts of the binary protocols and don't
| need to rely on just string manipulation to add escape
| sequences.
| mounir9912 wrote:
| What I really don't get is how it's any different than applying
| whatever function you would apply to the template, on the
| f-string variables. So instead of: evil =
| "<script>alert('bad')</script>" template = t"{evil}"
| safe = html(template)
|
| Why not just: evil =
| "<script>alert('bad')</script>" safe = f"{html(evil)}"
|
| Or even before creating the f-string. Is it just about not
| forgetting the sanitization/string manipulation part and forcing
| you to go through that?
| scott_w wrote:
| Pretty much, yeah. The article highlights that people were
| using f-strings directly, and they wanted to provide an
| alternative for lightweight template/interpolation.
| mounir9912 wrote:
| I feel like I'm still missing something when they're saying
| this about the example(s):
|
| "Neither of these examples is possible with f-strings. By
| providing a mechanism to intercept and transform interpolated
| values, template strings enable a wide range of string
| processing use cases."
|
| As far as I can see, anything you do with the template, you
| could do before building the f-string or inline as in my
| intial example.
| scott_w wrote:
| You wouldn't really do your example, though. If you're
| using an f-string, you'd just directly interpolate, because
| it's convenient. You wouldn't use an extra library to
| properly make it safe, otherwise you'd just use a proper
| template library and language.
|
| This gives you a convenient middle ground where you don't
| need to learn a template library but still get safety. I
| can't think of the code right now but I could see this
| being useful to pass in some dynamic HTML to, say, Django
| without having to remember to turn off escaping for that
| section. It can also be convenient for writing raw SQL
| without having to use prepared strings.
| davepeck wrote:
| With f-strings, you _cannot_ write code to determine which
| portions of the resulting `str` were static and which were
| dynamic; with t-strings, you can. *
|
| (As to your initial example: it's worth considering what
| will happen as you compose multiple bits of HTML via
| nesting to generate a large final page. The developer
| experience may become... unideal.)
|
| * (barring undesirable hacks with inspect, etc.)
| zahlman wrote:
| > Is it just about not forgetting the sanitization/string
| manipulation part and forcing you to go through that?
|
| This is a very big deal! It's also about _centralizing_ that
| work. Now that sanitization can occur in the _consumer of_ the
| t-string (for example, the API to your HTML renderer), rather
| than in every f-string.
| mont_tag wrote:
| How does this Python docs example work with t-strings?
| cur.executemany("INSERT INTO movie VALUES(?, ?, ?)", data)
|
| Can SQLite3 cache the query as it does now?
| roywiggins wrote:
| It seems to me that it would be straightforward to parse a
| Template's parameters back into ? placeholders?
| mont_tag wrote:
| No, it would fail upstream before the template post-
| processing even begin.
|
| The template object itself could not be formed because the
| each name must be a visible variable name.
|
| This is the same error that you would get with an f-string
| that contained an undefined variable name.
| sanderjd wrote:
| I'm intrigued but confused by your comment. I think it
| could be done like this:
| cur.executemany(t"INSERT INTO movie VALUES({data})")
|
| Then the Template object would have a parameter that is a
| list, and it could turn that into the right number of "?"s
| to pass into the database driver along with the data.
|
| What am I missing?
| nikisweeting wrote:
| This is pretty cool, if we're porting JS features do we get
| dictionary unpacking/destructuring next?
|
| I just want this so badly, it's the main reason I drift back to
| JS: >>> {a, b=45, c=None, **d} = {'a': 234,
| xzy: 32456} >>> print(a, b, c, d) 234 45 None
| {'xyz': 32456}
| enragedcacti wrote:
| just use this very sane pattern /s: >>> match
| {'b': 45, 'c': None}|{'a': 234, 'xyz': 32456}: >>>
| case {'a': a, 'b': b, 'c': c, **d}: pass >>> print(a,
| b, c, d) 234 45 None {'xyz': 32456}
| zahlman wrote:
| Another alternative: >>> a, b, c, d = (lambda
| a, b=45, c=None, **d: (a, b, c, d))(**{'a': 234, 'xyz': 32456})
|
| The parentheses for this are admittedly a bit tricky.
| kamikaz1k wrote:
| by making it a generic `t` you lose explicit syntax highlighting.
| Where something like JS template`string` could determine which
| syntax to use based on the template value.
|
| I supposed when assigning it to a, variable:
| SyntaxRecognizableTemplate, you could give it the hint necessary.
|
| was this discussed in the PEP?
|
| *edit: reading the PEP-750[1] it doesn't seem like it..
|
| [1] https://peps.python.org/pep-0750/#the-interpolation-type
| davepeck wrote:
| We pushed these questions out of the PEP to keep its scope
| constrained. (See https://peps.python.org/pep-0750/#mechanism-
| to-describe-the-...)
|
| But yes, the PEP leaves open an important question: how will
| tools decide to work with common types of content in t-strings,
| like HTML or SQL?
|
| There are simple approaches that can be taken in the short term
| (content sniffing) and more robust approaches (type
| annotations, perhaps) that will take time and the broader
| tooling community to develop.
| pauleveritt wrote:
| It was discussed in the first revision and discussion of the
| PEP. The decision was made to move that to follow-on work, as
| we discovered more about what tooling needs.
|
| As an example, I was excited about using `Annotated` on the
| function to indicate the language it expected. Turns out, a lot
| of linters know nothing about the type system.
| davepeck wrote:
| Hi! I wrote this. :-)
|
| I'm a little late to the conversation (and a bit surprised to see
| this trending on HN) but am happy to answer any questions; I'll
| try to pop in throughout the day.
| 18172828286177 wrote:
| This is super cool, thank you.
| maxloh wrote:
| Hi. I come from a JavaScript background.
|
| I am wondering what is the reason behind not using a similar
| syntax to JavaScript? Seems simpler to me. #
| Compare this: template = t"<p>{evil}</p>" safe =
| html(template) # To this: safe =
| html"<p>{evil}</p>"
| davepeck wrote:
| The PEP originally started with a similar-to-javascript
| syntax but over time we decided it wasn't the right way to
| expose these ideas in Python. There's more detail about why
| this approach was rejected in the PEP:
| https://peps.python.org/pep-0750/#arbitrary-string-
| literal-p...
| varunneal wrote:
| Would be interested in inclusion of PEP 292 [1] in your
| discussion here, which introduced `string.Template`. Is this
| Template going to be deprecated?
|
| [1] https://peps.python.org/pep-0292/
| davepeck wrote:
| PEP 292's `string.Template` will remain; there are no plans
| to deprecate it.
|
| PEP 750's `string.templatelib.Template` is a separate and
| unrelated type. Amongst many differences, unlike PEP 292,
| `Template` has a literal form too.
|
| I'm hopeful that the confusion will be minimal; in practice,
| PEP 292 (aka $-strings) is used only in specialized cases,
| like flufl.i18n, a really deep I18N framework.
| florbnit wrote:
| > In addition, I hope that the tooling ecosystem will adapt to
| support t-strings. For instance, I'd love to see black and ruff
| format t-string contents, and vscode color those contents, if
| they're a common type like HTML or SQL.
|
| This is such a strange take on t-strings. The only way for
| anything to infer that the template string is supposed to turn
| into valid HTML or SQL is to base it of the apparent syntax in
| the string, which can only be done in an ad-hoc fashion and has
| nothing to do with the template string feature.
|
| The way the feature has been designed there is no indication in
| the string itself what type of content it is or what it will
| eventually be converted to. It's all handled by the converting
| function.
|
| As others have added, something like sql"select * from {table}"
| would have been able to do this, but there's not even any
| guarantees that something that is in a template that will be
| converted into valid sql by a converting function should be any
| type of valid sql prior to that conversion. For all you know
| t"give me {table} but only {columns}" might be a converted into
| valid sql after the template is processed.
| pauleveritt wrote:
| The original PEP and the original discussion had this in scope.
| We removed it to let this emerge later. There are different
| ways to signal the language -- some more friendly to tooling,
| some more robust.
| pphysch wrote:
| Python has had type annotations for a decade, and modern IDEs
| can interpret them.
|
| Writing `query: SQL = t"SELECT ..."` is a small price to pay
| for such a DX boost.
| acdha wrote:
| Couldn't you do this with a type annotation? e.g. SQLAlchemy
| could have a SQL type so tools like mypy could see a Template
| instance and confirm you're using it safely but Black, Ruff, or
| SQLFluff could look for the more specialized
| Annotated[Template, SQL] to realize that the template could be
| formatted as SQL, and something like Django could even have
| Annotated[Template, Email], Annotated[Template, HTML], or
| Annotated[Template, JSX] to indicate what context the same
| templating syntax is targeting.
| pauleveritt wrote:
| This is what we discussed in the first revision of the PEP
| (the use of `Annotated`.) But we found out: linters don't
| know anything about the Python type system.
|
| We hope to get a community around all of this, stuff at PyCon
| US, EuroPython, etc. and work some of this out. The JSX/TSX
| world really has good tooling. We can provide that for those
| that want it, perhaps better on some aspects.
| acdha wrote:
| Interesting, thanks for the background. I've been curious
| what Astral is going to do in the space but also worry
| about what happens when their funding runs out.
| Someone wrote:
| > The only way for anything to infer that the template string
| is supposed to turn into valid HTML or SQL is to base it of the
| apparent syntax in the string
|
| Not the only thing. You can also look at how it is used. Your
| editor could know of how some popular libraries use t-strings,
| track which t-strings get passed into functions from those
| libraries, and use that to assume what grammar the t-string
| should follow.
|
| Is that cheating? In some sense, yes, but it also is useful and
| likely will be worth it for quite a few programmers.
| spankalee wrote:
| I think you'll just see a pattern like:
| html(t"<h1>Hello</h1>")
|
| And highlighters and static analyzers will key off of this.
|
| JavaScript's tagged template literals are actually about as
| flexible as this, since you can dynamically choose the tag
| function, it's just very rare to do so, so tools assume a lot
| based on the name of the function. Python tools can basically
| do the same thing, and just not support t-strings that aren't
| nested inside a well-named processing function.
| dandellion wrote:
| Another option would be type hints, something like `title:
| HTMLTemplate = t"<h1>Hello</h1>"`.
| davepeck wrote:
| > This is such a strange take on t-strings
|
| I understand why this seems strange at first!
|
| As Paul mentioned, we spent quite a lot of time considering
| these issues as PEP 750 came together. In the end, we concluded
| (a) the PEP leaves open quite a few potential approaches for
| tools to adopt (not just the one you suggest, as others here
| have pointed out), and (b) it's ultimately something that the
| broader tooling community needs to rally around and should
| probably be out of scope for the PEP itself.
|
| So, with that background in mind, I am _indeed_ hopeful we 'll
| see the ecosystem adapt! :-)
| permo-w wrote:
| is this just a copy paste of the PEP announcement?
| metrognome wrote:
| Zen of Python in 2025: There should be one-- and
| preferably only one --obvious way to do it.
|
| Python String Formatting in 2025:
|
| - t-strings
|
| - f-strings
|
| - %-operator
|
| - +-operator
|
| - str.format()
| a_t48 wrote:
| The last three generally shouldn't be used (`+` is sometimes
| fine, but not really for formatting), but I doubt we would ever
| get a py4 that removes them, given the stomachaches that py3
| caused. It does feel odd that a t-string is just an f-string
| without args applied.
| alfalfasprout wrote:
| It's my understanding that they're still recommended for
| logging since with f-strings you always pay the formatting
| cost (but with str.format it's deferred).
| davepeck wrote:
| And $-strings (PEP 292) as well. :-)
|
| I see each of these as distinct but overlapping; I'm (slowly)
| writing a guide to string formatting with all of these in mind,
| trying to emphasize when I'd choose one over the other. (fwiw I
| personally avoid % and + these days; $ is pretty uncommon in
| practice; f-, t-, and .format() all seem to have good unique
| uses.)
| sanderjd wrote:
| t-strings aren't really like these others.
|
| It's definitely true that those four string formatting
| techniques violate the "one obvious way" advice.
| hughw wrote:
| I could scarcely believe this new t-string wasn't a joke. As an
| occasional, grudging Python programmer, I rue the accretion of
| string formatters over the years. My code bears the history of
| successive, imperfectly understood (by me)formatters.
|
| "Situation: There are 14 competing standards...."
| https://xkcd.com/927/
| feraidoon wrote:
| I had something like this implemented for safely executing shell
| commands, by using metaprogramming:
|
| name="A$ron"
|
| z("echo Hello {name}")
|
| Note that this is not an f-string. The z function expands the
| variables by parsing this string and accessing its caller's local
| variables.
|
| https://github.com/NightMachinery/brish
| mos_basik wrote:
| Landing in 3.14? Nice, but also oof, that's probably not getting
| to my employer's codebase for a year or two. And it sounds like
| it could really solve some problems for us, too.
|
| Paging asottile - any plans to make a `future-tstrings`? :)
|
| `future-fstrings` (https://pypi.org/project/future-fstrings/) was
| a real QOL improvement for our team for a year or 2 around 2019
| before we got onto Python 3.x.
| pauleveritt wrote:
| Some prior art: https://pypi.org/project/tagged/
|
| In fact, the repo of a companion project from the author has
| the ticket that spawned the work leading to t-strings:
| https://github.com/jviide/htm.py/issues/11
| ayhanfuat wrote:
| I asked asottile for his opinion on PEP 750 (or rather, a
| previous version of it), and he wasn't really in favor. So I
| think it's unlikely that we'll see a future-tstrings from him.
| :)
| vFunct wrote:
| Would like to see Django fully use these to replace a lot of it's
| own complicated template syntax.
___________________________________________________________________
(page generated 2025-04-21 23:00 UTC)