[HN Gopher] The Single Byte That Kills Your Exploit: Understandi...
___________________________________________________________________
The Single Byte That Kills Your Exploit: Understanding Endianness
Author : andwati
Score : 26 points
Date : 2025-11-09 13:56 UTC (3 days ago)
(HTM) web link (pwnforfunandprofit.substack.com)
(TXT) w3m dump (pwnforfunandprofit.substack.com)
| MrBuddyCasino wrote:
| What first confused me about endianness is that it is about byte
| order, not bit order. The latter would have seemed more logical,
| or is this just me?
| andwati wrote:
| Learning this initially was confusing for me too, aren't we
| arranging bits?
| kazinator wrote:
| The words are divided into bytes. The bytes are rearranged,
| but the bits stay the same. The bits are not addressable and
| so represent pure binary values.
|
| For instance given the word DEADBEEF, the least significant
| byte is EF.
|
| That is a specific binary value: the value 239.
|
| That value stays the same whether the bytes are EF BE AD DE
| in memory, or DE AD BE EF.
|
| EF is just 239. We don't think about reversing the bits; they
| are not addressable. They have an abstract order determined
| by the binary system. The most significant bit of the value
| contributes 128 and so on.
|
| The order matters when the bits have to be transmitted over a
| wire to another machine. Then we have to decide: do we
| transmit the low bit of EF first, or the high bit 1? If the
| two sides of the data link are inconsistent, then one side
| transmits 11101111 and the other receives 11110111, which is
| F7.
| cobbal wrote:
| Little endian does appear strange at first, but if you consider
| the motivation it makes a lot of sense.
|
| Little endian's most valuable property is that an integer
| stored at an address has common layout no matter the width of
| the integer. If I store an i32 at 0x100, and then load an i16
| at 0x100, that's the same as casting (with wrapping) an i32 to
| an i16 because the "ones digit" (more accurately the "ones
| byte") is stored at the same place for both integers.
|
| Since bits aren't addressable, they don't really have an order
| in memory. The only way to access bits is by loading them into
| a register, and registers don't meaningfully have an
| endianness.
| IshKebab wrote:
| I'm not sure I've ever seen that _actually_ come in to play.
| Little Endian is obviously the best Endian, but I don 't
| think that argument really makes sense.
|
| The most obvious argument is that little Endian is clearly
| the most natural order - the only reason to use Big Endian is
| to match the stupid human history of mixing LTR text with RTL
| numbers.
|
| I've seen one real technical reason to prefer little endian
| (can't remember what it was tbh but it was fairly niche) and
| I've never seen any technical reasons to prefer big endian
| ("it's easier to read in a hex editor" doesn't count).
| jojomodding wrote:
| You can't address individual bits. There is no way of telling
| if the LSBit is "left" or "right" of the MSBit. So endianness
| can't be about that.
|
| For bytes, you can distinguish them, as you can look at the
| individual bytes produced from a larger-than-byte store.
| tadfisher wrote:
| Your CPU (probably) has left and right variants for shift and
| rotate operations, which is certainly an avenue for
| confusion. There's a "logical" bit order that these
| operations follow, which starts with the MSBit and ends with
| the LSBit, even when the physical connections are all
| parallel and don't really define a physical bit order.
| NobodyNada wrote:
| > There's a "logical" bit order that these operations
| follow, which starts with the MSBit and ends with the LSBit
|
| Well, normally when bits are numbered, "bit 0" is the least
| significant bit. The MSB is usually written on the left,
| (such as for left and right shifts), but that doesn't
| necessarily make it "first" in my mind.
| kazinator wrote:
| The Common Lisp ASH (arithmetic shift) instruction has
| positive shifts for left, nefgative for right.
|
| But even if machines were like this, it would not cause any
| interoperatibility issue. Because it is the data links
| between machines which ensure that bits are transmitted and
| received in the correct order, not the semantics of machine
| instructions.
|
| It would be something to worry about when translating code
| from one language or instruction set to another.
|
| Data link and physical protocols ensure that when you
| transmit byte with a certain decimal value like 65 (ASCII
| 'A') it is received as 65 on the other end.
|
| The bits are "addressable" at the data link level, because
| the hardware has to receive a certain bit first, and the
| one after that next, and so on.
| kazinator wrote:
| The concept of order can only matter down to the units that are
| addressable.
|
| Bits are typically not addressable, therefore do not have
| endiannness.
|
| Bits are manipulated by special instructions, and those
| instructions are tied to arithmetic identities, due to the bits
| being interpreted as a binary number: like that a shift left is
| multiplication by 2.
|
| In many instruction sets, the shift is a positive amount, and
| whether it is left or right is a different instruction. If it
| were the case that shifting one way is positive and the other
| way negative, then you have a kind of endiannness in that one
| machine uses positive for multiplication by powers of two,
| whereas another one for division. That would not result in an
| incompatible storage format though.
|
| When data is transferred between machines as a sequence of
| bytes, there is a bit order in question, but it is taken care
| of by the compatibility of the data links.
|
| Classic Ethernet is little endian at the bit level: the
| baseband pulses that represent the bit of a byte are sent into
| coax cable least-significant-bit first. RS-232 serial
| communication is the same: least significant bit first.
|
| I think I2C is an example of a data link / physical protocol
| that is most-significant-bit first. So if you somehow hooked up
| an RS-232 end to I2C and got the communication to work, the
| bytes would be reversed.
|
| We rarely, if ever, see bit endian effects because nobody does
| that --- transmit bytes between incompatible data links. If
| won't work for other reasons, like different framing bits,
| signaling conventions, voltages, speeds, synchronization
| methods, checksums, ...
|
| Endianness of bits shows up in some data formats which pack
| individual bitfields of variable length.
|
| Bitfields in C structures reveal bit endianness to some extent.
| What typically happens is that on a big endian target, bit
| fields are packed into the most significant bit of the
| underlying "cell" first. E.g. struct {
| unsigned a : 1, b : 1 };
|
| the underlying cell might be the size of an int, like 32 bits.
| So where in the cell do "a" and "b" go? What you see under GCC
| is that on a big endian target, _b_ will go to the most
| significant bit of the underlying storage cell, and _b_ to the
| second most significant one. Whereas on little endian, _a_ goes
| to the least significant bit, and _b_ to the second least. In
| both cases, the bits map to the first byte, at the lowest
| address.
|
| So in a certain sense, the allocation of members in C, as such,
| is little endian: the earlier struct members go to the lowest
| address, regardless of machine endian. It is probably because
| of that the bit order follows. Since putting bitfield _a_ at
| the lowest address, as mandated by C field layout order, means
| that it has to go into the first byte, and that first byte is
| the most significant byte under big endian, it makes sense that
| the bit goes into the most significant bit position, for
| consistency.
|
| That way we only have two possibilities to deal with for, say,
| a memory mapped status register: struct
| port_status_word { #if HAVE_BIG_ENDIAN unsigned
| transmit_ready : 1; unsigned data_received : 1;
| unsigned carrier_present : 1; // [ ... 29 more]
| #else // [ ... 29 more] unsigned
| carrier_present : 1; unsigned data_received : 1;
| unsigned transmit_ready : 1; #endif };
|
| If we had separate byte and bit order, we would need two levels
| of #if nesting and four possibilities, which is even more ugly.
| scottlamb wrote:
| This is a weird take. I've never put together this kind of
| exploit, but still I know enough to not buy this. Do people ever
| really craft exploits that are perfectly valid except for using
| the wrong endianness?
|
| > If you've ever crafted a perfect shellcode and ROP chain only
| to have your exploit immediately crash with a SIGSEGV(a signal
| sent by the operating system to a program when it attempts to
| access a protected or invalid memory location) or EIP(a 32-bit
| CPU register in the x86 architecture that holds the memory
| address of the next machine instruction to be executed) pointing
| to garbage, you've likely met the silent killer of beginners:
| Endianness.
|
| Aren't there a million other ways to get addresses wrong?
|
| > Using x86/x86_64 gadgets and packers on a MIPS/PowerPC target
| (different endianness and instruction set) will not work.
|
| "and instruction set" is carrying a lot of weight here.
|
| This isn't like a coin flip thing: even considering architectures
| with configurable endianness, in 2025 it's overwhelmingly likely
| both host and target are little-endian. And on old, big-endian
| platforms, that's just one of many things you have to get right.
| benmmurphy wrote:
| it does seem like the audience the article is explicitly
| targeted for is an edge case. people who understand enough to
| be writing an exploit but are somehow unaware of their target
| architecture works.
|
| but i guess the real target audience is probably people that
| are just starting out on CTFs and just trying to string stuff
| together without a proper understanding of the fundamentals.
| everyone has to start somewhere and i guess if people are just
| using packers and tools to generate exploit code then its quite
| easy to use the wrong flags and not know what is going on.
| tadfisher wrote:
| > Disclaimer: This article was written with AI assistance,
| for a bit of brainstorming and proofreading.
|
| I suspect the target audience is "whoever will subscribe on
| Substack" more than someone who has ever written or
| contemplated writing shellcode. I'm seeing more and more
| articles like this that focus the prose on some weird subset-
| of-a-niche aspect of a subject, then end with a set of bullet
| points for fixing the problem as if this is something one
| regularly encounters.
| Retr0id wrote:
| People learn things in different orders. For many people, low-
| level CTF challenges are their _introduction_ to computer
| architecture (a good way to learn if you ask me!) If so,
| endianness is a novel concept to them.
|
| While I personally learned about endianness before writing my
| first exploit, I've definitely made endianness-related mistakes
| before.
| kazinator wrote:
| > _you've likely met the silent killer of beginners: Endianness._
|
| No, you've more likely made one or more of any number of bugs, or
| the conditions are not right in the target host for the exploit
| to work.
___________________________________________________________________
(page generated 2025-11-12 23:01 UTC)