[HN Gopher] Please restore our registers when you're done with them
___________________________________________________________________
Please restore our registers when you're done with them
Author : jmillikin
Score : 173 points
Date : 2022-11-22 12:53 UTC (10 hours ago)
(HTM) web link (randomascii.wordpress.com)
(TXT) w3m dump (randomascii.wordpress.com)
| secondcoming wrote:
| Ugh, I had a similar issue when I forgot to clobber ecx when
| using inline assembly for rdtscp. Very annoying.
| von_lohengramm wrote:
| I'm sure some C++ lawyer can correct me, but isn't branching off
| of `m_ptr` (e.g. `CHECK()`ing the value) after `std::move(m_ptr)`
| technically Unspecified Behavior since `std::move(m_ptr)` leaves
| `m_ptr` as an Unspecified Value? It would be up to the compiler
| to define the behavior if they so pleased, but the C++ spec would
| not require such behavior to be defined at all.
| tux3 wrote:
| Moves in C++ don't make the whole object invalid, it just
| leaves them in a valid but unspecified state, so that at least
| the destructor can still run.
|
| This means calling operator bool on your unique_ptr ought to be
| fine, because the unique_ptr still has a valid state (you don't
| know what that state is, it's unspecified, but it's guaranteed
| to not be radioactive on mere contact. It has to be a _valid_
| unspecified state.)
| twoodfin wrote:
| It's an unspecified but required to be valid value for the
| moved type. The author mentions it's a smart pointer type,
| which could easily be defined to act like this.
| CamperBob2 wrote:
| It appears that you need to be really smart in order to not
| blow your own foot off with this "smart pointer." I'll stick
| to the regular dumb kind, thanks.
| s28l wrote:
| There is a big difference between "undefined" and "unspecified"
| behavior. In this case, the behavior of
| `unique_ptr(unique_ptr&&)` is in fact specified. [0]
|
| However, the bigger issue with that code is that it can easily
| stop working with a simple refactor. Consider:
| void foo(std::unique_ptr<int> ptr) {} void
| bar(std::unique_ptr<int>&& ptr) {} int main()
| { std::unique_ptr<int> p1{new int{1}};
| std::unique_ptr<int> p2{new int{2}};
| foo(std::move(p1)); assert(p1 == nullptr);
| bar(std::move(p2)); assert(p2 != nullptr);
| }
|
| Neither of the above asserts will fire, but from the calling
| site, they look exactly the same. In my opinion, the more
| explicit option would be to do something like
| `bar(std::exchange(p2, nullptr))`
|
| [0]: overload (5)
| https://en.cppreference.com/w/cpp/memory/unique_ptr/unique_p...
| mkj wrote:
| It seems that the chrome developers should be able to perform the
| same binary analysis on the suspect Mcafee software. I guess it's
| a bit harder without source code to reference side-by-side
| though.
| brucedawson wrote:
| That would be possible, but we'd have to install the software,
| then guess which binary was the culprit, and then have some way
| of finding the function boundaries. My crude analysis technique
| required on having symbols for chrome.dll to indicate where
| functions started, so I'd have to have switched tools to
| something else that could find those.
| altilunium wrote:
| kaapipo wrote:
| How does this add to the conversation?
| namdnay wrote:
| looks like someone spamming links to all their personal
| projects over HN
| dang wrote:
| We ban accounts that post like this, because the community
| considers it spamming.
|
| I'm not going to ban you because you've also posted other
| things to HN and seem like a legit user. But you've been
| posting these links much too often, so please stop doing that.
| dreamcompiler wrote:
| In this particular example, why not just PXOR
| XMM7 XMM7
|
| before using XMM7 to zero anything? That way the compiler doesn't
| have to _assume_ that XMM7 is zero, it can _know_.
|
| Yes it's an extra instruction but XORing a register with itself
| is such a common metaphor for zeroing that register that CPU
| designers try to make it fast.
|
| Edit: Just noticed that Veliladon essentially made the same
| comment herein and explained the reason why it's not done this
| way.
| puffoflogic wrote:
| It's not just that. If you can't trust the ABI then everything
| else is wrong too. Everything. Not just zeroing registers. It
| only just so happens that in this case XMM7 is used for
| zeroing, but it could be used to save a variable across a
| function call. Then there is no trick to get it set back to the
| right value.
|
| The ABI is not optional, or best effort, or best practice, or
| any other BS that passes in the ordinary world. It is just as
| required as the correct operation of instructions (e.g., add
| should actually add things, mul should actually multiply them,
| and so on).
| mrkeen wrote:
| My reading of it was that the bug was caused by XMM7 not being
| restored to its previous value.
|
| E.g. on Linux, functions should restore the values of ebx, esi,
| edi, ... once they're done with them. The article says (on
| Windows) that XMM7 needs to be restored too.
|
| If you can't trust one register being preserved (per the ABI),
| then you really can't trust the values of any registers.
| KingLancelot wrote:
| codeflo wrote:
| Why does a ,,3rd party encryption software" appear in the call
| stack of a user mode process in the first place? Is this another
| case where a "security" software injects broken DLL files into
| all processes in your system?
| Cloudef wrote:
| The irony of security and anti-cheat software acting like a
| invasive malware
| cpeterso wrote:
| Yes. That's pretty common, unfortunately.
| bombcar wrote:
| It's interesting that there is a zero stored in a register and
| used for hours - is that significantly faster than just using
| some actual zero each time? Perhaps CPUs need a "always zero"
| register or some similar menomic to help harden.
| Veliladon wrote:
| Intel has never really needed to have a zero register because
| xor register, register as a zeroing idiom is so fast and so
| recognized that Intel have optimized the hell out of it. In
| Sandy Bridge and onward it doesn't even go through an execution
| port, even for the vector registers.
|
| The problem is really whether to indulge bad programmers who
| don't respect the ABI at the cost of a minimal sliver of
| performance (even though it's not taking up an execution port
| the extra instruction still takes up cache space, bandwidth,
| and decode). Yeah they should probably zero the register before
| they zero the pointer but they shouldn't have to if other
| people respected the ABI.
| kazinator wrote:
| I think it's not just the xor trick, but that Intel has lots
| of addressing modes, including ones with immediate operands
| that you can use in many situations.
|
| In RISC-like machines, most of the operations are register-
| register, and you have load/store instructions for
| referencing memory.
|
| To use an immediate operand (literal constant in the code
| itself), you may have to load it into a register, like
| move r7, #42 add r1, r1, r7 ;; ok, now we have 42 in
| r7, we can increment r1 by 42.
|
| Whereas in a CISC you would have add r1, #42
| ;; two operand form
|
| or maybe add r1, r1, #42 ;; three operand
| form
|
| When you need a zero, you just use the immediate operand
| zero, and thus you don't need to to pick some register to
| clear.
|
| In summary, zero registers in RISC-like instruction set
| architectures effectively provide a literal zero that can be
| used wherever a register is required, which helps because
| only register operands can be used in many instructions.
| Veliladon wrote:
| That's a good point. But all of the x86 SIMD stuff is
| register/register and we don't have xmm0/ymm0/zmm0 being 0
| like we'd expect on a load store style RISC architecture.
| CodesInChaos wrote:
| RISC-V has such a register. It returns zero when read, and
| ignored writes.
| [deleted]
| ddingus wrote:
| Where it's not optimized away, getting "an actual zero"
| requires a memory operation of some kind. Register ops are
| faster in that they are right there, no fetch needed.
| AnimalMuppet wrote:
| Depends on the instruction architecture. 68000 had some ways
| of burying a small literal operand in the instruction.
| MOVEQ.L would let you move zero to a register without
| touching memory (other than the instruction fetch), and it
| wasn't a long instruction.
| gumby wrote:
| Many architectures do have a 0 register because the value is
| useful. Others have a zero instruction (or both).
|
| What would an "actual zero" be -- a literal?
| bombcar wrote:
| Yeah, something like "mov ax, 0h" but I suppose that is way
| more memory intensive as you have to load a 0 into memory
| somewhere and then copy it into the register.
|
| It strikes me as somehow the _compiler_ is making assumptions
| that aren 't being enforced by the ... OS? Language? not sure
| what, but it's assuming functions restore registers used but
| that isn't enforced by anything. From my (long ago) time
| there was PUSHA and POPA but I assume those take quite a bit
| of "oomph" and are avoided if possible.
| gumby wrote:
| > It strikes me as somehow the _compiler_ is making
| assumptions that aren 't being enforced by the ... OS?
| Language?
|
| One case of this problem was in a handwritten assembly
| file. The other was a compiler bug.
|
| This is a case where the ABI requires that if you use a
| certain register you must save its previous value and
| restore it afterwords; the two independent bugs were cases
| of forgetting to look after a certain register.
|
| An ABI is simply an agreement as to how things should work:
| what registers you are free to clobber, which you must look
| after when you use, how certain data must be laid out in
| memory, etc. ABIs are typically language specific, though
| there may be a lot of commonality at the very high level
| (i.e. how you use sections in an ELF file) and low (anybody
| using unboxed integers probably will do the same thing).
|
| You are welcome to violate the ABI as you see fit in your
| own code. The OS doesn't care; it has its own constraints
| (how to make a system call, how to pass arguments to each
| -- though cf above when I talked about ints). So, say, a
| Lisp compiler can lay out stack frames differently from a
| C++ compiler because of the languages' different semantics)
| but if your Lisp program wants to call a library written in
| C++ it must make sure memory at the call site follows the
| C++ ABI because that's what the C++ compiler will have
| assumed.
| bombcar wrote:
| It seems to me something that could be found by some kind
| of valgrind-like tool - it'd be much slower than normal
| code but "ABI exception detected" or something.
| gumby wrote:
| Not worth checking for. The few people who write assembly
| code these days know what they are doing, and, bugs
| aside, the compiler knows what to do.
| titzer wrote:
| The old Russian proverb is "Trust but verify."
|
| I, for one, hate debugging asm. I do it a lot, and would
| prefer bugs be caught automatically, preferably soon
| after they are introduced.
| gpderetta wrote:
| Also compilers violate the ABI when they know the
| violation can't be observed, so the external tool would
| have too many positives.
| AnimalMuppet wrote:
| Why would you have to load 0 into memory somewhere (other
| than in the instruction itself)? Or did you mean that it is
| in fact in the instruction?
| pitaj wrote:
| I don't know anything about x86, but on Arm you can use an
| immediate 0 that is stored in the instruction data.
| rocqua wrote:
| To what degree is this possible to check statically?
|
| It feels like at least simple breaks of the ABI rules like this
| can be detected somewhat statically. The author already started
| with a very simple and incomplete version.
|
| In general, I wonder, are there any (many?) static analyzers for
| assembled binaries.
| fweimer wrote:
| Compilers do inter-procedural register allocation and use
| custom calling conventions for local calls (where "local" can
| be quite large with LTO), while preserving ABI externally. This
| means that clobbering a callee-saved register without
| saving/restoring it in the same function is not necessarily a
| bug.
|
| Curiously, I found a register clobber bug in the NaCl
| cryptography library today. Apparently, they used a custom
| assembler-preprocessor (qhasm) that avoids certain classes of
| bugs and aids with porting, but while the tool seems to
| actually model the register in some way, it does not treat it
| as callee-saved.
| otikik wrote:
| Naive question about ABIs: shouldn't the caller be responsible
| for that? If I want a function to restore certain registers,
| wouldn't it be simpler if I was the one that save them on my
| memory, call the function, and then override the registers with
| whatever values the function set? Otherwise it seems we're
| just... asking for trouble, so to speak.
| dannymi wrote:
| It would just be slow for the caller to have to push and pop
| (say 30) registers in general that the specific callee (and
| transitive callees) may not even use.
|
| Most ABI specify some registers caller-saved, some registers
| callee-saved (retained unchanged from the perspective of the
| caller) and some registers scratch (not-saved).
|
| In the end it depends on the architecture and on typical
| workload which are the fastest--and measurements can be made
| and it can be found out which combination is the fastest on
| average.
| dzaima wrote:
| You usually wouldn't need to push&pop all registers, just
| ones that you want to preserve across the call. Regardless,
| yeah, non-volatile aka callee-saved registers are extremely
| important for good performance of code that calls functions
| (esp. loops - without callee-saved registers, you'd have to
| store the loop counter & length on the stack!)
| dreamcompiler wrote:
| ...which is exactly what the callee within the loop would
| have to do if it used those registers. But I guess your
| point is that in the callee-save case it only happens when
| it needs to happen, while in the caller-save case it
| happens every time whether it needs to or not.
| dzaima wrote:
| yep, hence why calling conventions usually have both
| callee-saved and caller-saved registers, so that you only
| have "unnecessary" stack usage when you need to use more
| than roughly half of the registers.
| chrisseaton wrote:
| > If I want a function to restore certain registers, wouldn't
| it be simpler if I was the one that save them on my memory,
| call the function, and then override the registers with
| whatever values the function set?
|
| That's a waste if the callee doesn't use them.
| otikik wrote:
| Indeed. But isn't it also a waste when someone doesn't follow
| through and then we have situations like the one described on
| the article?
| catiopatio wrote:
| Such cases are rare bugs in a small amount of code --
| mostly just compilers and hand-written assembly.
|
| Preemptively saving and restoring all registers in all
| callers would appreciably slow down every single function
| call on every device in the world using that ABI.
|
| The cumulative cost would be astronomical.
| 323 wrote:
| Sounds like there should be an option when you write assembly
| code to tell the compiler "please save/restore any register that
| I'm modifying in this asm code according to the target you are
| compiling for"
| wyldfire wrote:
| When you use inline assembly, in fact the compiler does
| preserve the semantics of the surrounding program considering
| the target's ABI -- _if_ you tell it correctly what impacts the
| inline asm has. Lots of rules must be followed in order to get
| this behavior just right. One of the most subtle ones is
| "early clobbers" [1].
|
| In many cases, you can get all the benefits of inline assembly
| from compiler intrinsics while letting the compiler handle all
| the details of register allocation and scheduling.
|
| Note that in OP's case IIUC this was assembly code and not
| inline assembly. If you write functions in assembly you are
| solely responsible for calling conventions and ABI conformance.
|
| [1] https://stackoverflow.com/a/15819941/489590
| pjc50 wrote:
| That kind of feature is very .. un-assembler. You certainly
| could, but the assembler doesn't keep track of any of the
| relevant information, so it would be a larger feature than you
| expect. It doesn't know the calling convention. It doesn't have
| a map of registers dirtied by which instructions. It doesn't do
| reachability analysis, so it doesn't even necessarily know
| what's "in the function".
| Joker_vD wrote:
| Doesn't GCC inline assembly actually track that info, with
| the "clobber" section?
| masklinn wrote:
| Yes, but it's not a property of the assembly (or
| assembler), it's a necessity for the compiler to correctly
| codegen around the inline assembly.
|
| Historically, assemblers have been really dumb, so ABI is
| not a thing they'd track, especially as... I don't think
| they know what functions are? So while they can notice
| call/ret, they have no knowledge of a label being a jump or
| call target per-se, do they?
|
| So you'd need an assembly-like language to encode this sort
| of information.
| ElevenLathe wrote:
| I mean if you have a macro assembler you should be able
| to write a macro that generates "save registers"
| instructions before a section of code and "restore
| registers" instructions after it. The assembler doesn't
| need to know our care that this section of code is a
| function.
|
| It's interesting how blurry the line is between a good,
| full-featured assembler and a crappy compiler!
| masklinn wrote:
| > I mean if you have a macro assembler you should be able
| to write a macro that generates "save registers"
| instructions before a section of code and "restore
| registers" instructions after it.
|
| Except you want a "macro" which:
|
| - saves only the registers you touched
|
| - which are callee-saved
|
| - according to the ABI you're targeting
|
| And you really only want that for functions, because...
| that's where ABIs come into play.
| denton-scratch wrote:
| > I don't think they know what functions are?
|
| It's a long time since I tangled with x86 assembler; but
| as I recall, ENTER and LEAVE were specifically for
| functions, and I'm not aware of any other use for them.
| tremon wrote:
| ENTER and LEAVE were specific x86 instructions though,
| not some kind of assembler special sauce. The assembler
| just translated your ENTER instruction into the
| corresponding machine opcode; no ABI knowledge required
| whatsoever.
| matthews2 wrote:
| Maybe we need some kind of higher level language that gets
| boiled down into assembly.
| pencilguin wrote:
| And, a compiler that knows custom SIMD optimizations for
| every algorithm anyone might ever need, and can recognize
| when you have coded one of them so it can substitute its SIMD
| version.
| dfox wrote:
| THat is essentially how inline assembly in GCC works. You have
| to declare which registers you are changing and in which
| registers you expect input and which contain results from your
| inline assembly block.
|
| This for obvious reasons does not work if you have separate
| compilation unit written in assembly, then you have to follow
| the ABI.
| flanfly wrote:
| I not sure whether that satire. In case its not, most languages
| that allow inline assembly (like C) have an optional "clobber
| list" argument that tells the dataflow analysis of the compiler
| that your assembly snippet overwrites certain registers [1].
| Inline assembly doesn't have target specific clobber lists
| because it's assumed that the code only works on one target and
| the programmer has to take care of making it work.
|
| 1: https://www.ibiblio.org/gferg/ldp/GCC-Inline-Assembly-
| HOWTO....
| tom_ wrote:
| The System V x64 ABI is different from the Windows x64 ABI.
| [deleted]
| bumholio wrote:
| If one could reliably detect this DLL injection in the process
| address space, then the correct "fix" is to crash immediately.
| Authors of such tools should seek another way of accomplishing
| their goal, preferably one that does not export their own bugs to
| innocent bystanders.
| dblohm7 wrote:
| I used to take point for Mozilla's efforts dealing with third-
| party interference in our binaries. Browsers are ripe targets for
| this kind of shit. The stories we could tell...
| harry8 wrote:
| And from our perspective you should! Would be interesting.
| pjc50 wrote:
| One of the linked pull requests:
| https://github.com/cisco/openh264/commit/db956674bbdfbaab5ac...
|
| Macros in a macro assembler .. not very nice.
| userbinator wrote:
| He even mentions that it's Windows-specific, both in the commit
| message and in the article... and then seemingly fails to make
| it Windows-only? That's "not very nice" either.
| electroly wrote:
| Those PUSH_XMM/POP_XMM macros appear to be Windows-only; I
| think they expand to nothing on other platforms because they
| contain their own guard for Windows internally. If that's the
| case, the call sites don't need to guard for it. I'm guessing
| that obeying this calling convention is the purpose of those
| macros.
|
| https://github.com/cisco/openh264/blob/db956674bbdfbaab5acdd.
| ..
|
| https://github.com/cisco/openh264/blob/db956674bbdfbaab5acdd.
| ..
| brucedawson wrote:
| Exactly. My understanding of the conventions and macros in
| those source files is that you declare what registers you
| will be trashing, and then the registers are saved/restored
| as required by that platform. On Linux it would be a NOP,
| and on Windows it saves and restores XMM6 and XMM7
| (XMM0-XMM5 are volatile).
| brucedawson wrote:
| The webrtc fix was thematically similar in that the
| programmer declared what registers were trashed and then
| the compiler knows which registers need to be saved. I'm
| not sure why the compiler doesn't notice when registers are
| used without being declared as being trashed - I'm really
| not an expert at _writing_ assembly language.
| pjc50 wrote:
| Traditionally, an assembler neither knows nor cares about
| such information, it just turns lines of assembly into
| bytes and does some address fixup for you.
| tialaramex wrote:
| It's _literally undecidable_ in principle whether some
| assembler correctly restores some register R. That 's a
| non-trivial semantic property, Rice's theorem applies. So
| the compiler's only practical option if it worked this
| way would be a conservative option - any time it's
| unclear whether register R is clobbered, treat it as
| clobbered.
|
| As a trivial example of why a register might _not_ be
| clobbered even though my code touched it and it seems
| like I didn 't restore it...
|
| Suppose if R is divisible by 12 I branch, in the other
| branch I don't change R, but in that branch I do change
| R, XORing it with a value which is difficult to explain
| but has a value between 1 and 3 inclusive, sometimes more
| than once. At the end of the branch I also clear the
| bottom two bits of R.
|
| R is actually not clobbered by this function! If the
| bottom two bits weren't zero before, R isn't divisible by
| 12, so we didn't change R, and if they _were_ zero, we
| restore that, the other bits are never changed.
|
| Having the human programmer promise they they wrote a
| correct clobber list means if their assembler _does_
| somehow restore / preserve register R, the human can just
| say so, and needn't prove to the compiler somehow that
| this works. This sort of code is mostly in performance
| critical components, e.g. video decoding, where we are
| already trading reliance on fallible humans for better
| performance, so adding one extra promise feels OK.
| twic wrote:
| > It's literally undecidable in principle whether some
| assembler correctly restores some register R.
|
| No, it's literally undecidable in principle whether
| _every_ bit of assembler correctly restores some register
| R. For any given bit of inline assembler, it 's quite
| likely to be trivial.
|
| In any case, we can have a useful safety feature without
| requiring the compiler to decide. The compiler can easily
| work out all the registers which get written to (right?),
| just not which get restored. So in addition to the
| clobber list, we could have a list of registers which the
| programmer asserts that the code restores. A register
| which is written to has to be on either the clobber list
| or the restore list (or be an output). This certainly
| isn't foolproof, but it would catch accidental clobbers.
| Animats wrote:
| > No, it's literally undecidable in principle whether
| every bit of assembler correctly restores some register
| R. For any given bit of inline assembler, it's quite
| likely to be trivial.
|
| Exactly. And if it's non-trivial to decide something that
| basic, you're doing it wrong. Saving and restoring all
| the registers is always an option. Only saving and
| restoring some of them is an optimization which must be
| shown to be sound.
| [deleted]
___________________________________________________________________
(page generated 2022-11-22 23:01 UTC)