[HN Gopher] C++ patterns for low-latency applications including ...
___________________________________________________________________
C++ patterns for low-latency applications including high-frequency
trading
Author : chris_overseas
Score : 86 points
Date : 2024-07-08 18:58 UTC (4 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| jeffreygoesto wrote:
| Reminds me of
| https://github.com/CppCon/CppCon2017/blob/master/Presentatio...
| munificent wrote:
| This is an excellent slideshow.
|
| The slide on measuring by having a fake server replaying order
| data, a second server calculating runtimes, the server under
| test, and a hardware switch to let you measure packet times is
| so delightfully hardcore.
|
| I don't have any interest in working in finance, but it must be
| fun working on something so performance critical that buying a
| rack of hardware just for benchmarking is economically
| feasible.
| sneilan1 wrote:
| I've got an implementation of a stock exchange that uses the LMAX
| disruptor pattern in C++ https://github.com/sneilan/stock-
| exchange
|
| And a basic implementation of the LMAX disruptor as a couple C++
| files https://github.com/sneilan/lmax-disruptor-tutorial
|
| I've been looking to rebuild this in rust however. I reached the
| point where I implemented my own websocket protocol,
| authentication system, SSL etc. Then I realized that memory
| management and dependencies are a lot easier in rust. Especially
| for a one man software project.
| JedMartin wrote:
| It's not easy to get data structures like this right in C++.
| There are a couple of problems with your implementation of the
| queue. Memory accesses can be reordered by both the compiler
| and the CPU, so you should use std::atomic for your producer
| and consumer positions to get the barriers described in the
| original LMAX Disruptor paper. In the get method, you're
| returning a pointer to the element within the queue after
| bumping the consumer position (which frees the slot for the
| producer), so it can get overwritten while the user is
| accessing it. And then your producer and consumer positions
| will most likely end up in the same cache line, leading to
| false sharing.
| sneilan1 wrote:
| >> In the get method, you're returning a pointer to the
| element within the queue after bumping the consumer position
| (which frees the slot for the producer), so it can get
| overwritten while the user is accessing it. And then your
| producer and consumer positions will most likely end up in
| the same cache line, leading to false sharing.
|
| I did not realize this. Thank you so much for pointing this
| out. I'm going to take a look.
|
| >> use std::atomic for your producer
|
| Yes, it is hard to get these data structures right. I used
| Martin Fowler's description of the LMAX algorithm which did
| not mention atomic.
| https://martinfowler.com/articles/lmax.html I'll check out
| the paper.
| fooker wrote:
| You say it's easier in Rust, but you still have a complete C++
| implementation and not a Rust one. :)
| deepsun wrote:
| Linus said he wouldn't start Linux if Unix was ready at that
| time.
| sneilan1 wrote:
| It took me about a year to build all of this stuff in C++. So
| I imagine since I've had to learn rust, it will probably take
| me the same amount of time if I can save time with
| dependencies.
| temporarely wrote:
| fun fact: the original LMAX was designed for and written in
| Java.
|
| https://martinfowler.com/articles/lmax.html
| sneilan1 wrote:
| I think it made sense at the time. From what I understand,
| you can make Java run as fast as C++ if you're careful with
| it and use JIT. However, I have never tried such a thing and
| my source is hearsay from friends who have worked in
| financial institutions. Then you get added benefit of the
| Java ecosystem.
| gedanziger wrote:
| Very cool intro to the subject!
| globular-toast wrote:
| Is there any good reason for high-frequency trading to exist?
| People often complain about bitcoin wasting energy, but oddly
| this gets a free pass despite this being a definite net negative
| to society as far as I can tell.
| arcimpulse wrote:
| It would be trivial (and vastly more equitable) to quantize
| trade times.
| FredPret wrote:
| Non-bitcoin transactions are just a couple of entries in
| various databases. Mining bitcoin is intense number crunching.
|
| HFT makes the financial markets a tiny bit more accurate by
| resolving inconsistencies (for example three pairs of
| currencies can get out of whack with one another) and obvious
| mispricings (for various definitions of "obvious")
| jeffbee wrote:
| That's a nice fairy tale that they probably tell their kids
| when asked, but what the profitable firms are doing at the
| cutting edge is inducing responses in the other guys' robots,
| in a phase that the antagonist controls, then trading against
| what they know is about to happen. It is literally market
| manipulation. A way to kill off this entire field of endeavor
| is to charge a tax on cancelled orders.
| blakers95 wrote:
| Yep and what's worse is many hft firms aren't in the
| market-making business at all but actually REMOVE
| liquidity.
| FredPret wrote:
| What's the mechanism for removing liquidity?
| FredPret wrote:
| If one outlawed / disincentivized hostile the bot behavior
| you described, there would still be the opportunity to do
| the good and profitable things I described.
| torlok wrote:
| To go a step further, I don't think you should be required to
| talk to middlemen when buying stocks, yet here we are. The
| house wants its cut.
| astromaniak wrote:
| Under the carpet deals would make it less transparent. It
| would be hard to detect that top manager sold all his stock
| to competitors and now is making decisions in their favor.
| bravura wrote:
| Warren Buffett proposed that the stock market should be open
| less frequently, like once a quarter or similar. This would
| encourage long-term investing rather than reacting to
| speculation.
|
| Regardless, there are no natural events that necessitate high-
| frequency trading. The underlying value of things rarely
| changes very quickly, and if it does it's not volatile, rather
| it's a firm transiton.
| astromaniak wrote:
| > Warren Buffett proposed that the stock market should be
| open less frequently
|
| This will result in another market where deals will be made
| and then finalized on that 'official' when it opens. It's
| like with employee stock. You can sell it before you can...
| munk-a wrote:
| Not if we disallow it. We have laws in place to try and
| prevent a lot of natural actions of markets.
| htrp wrote:
| most trading volume already happens at open or before close.
| Arnt wrote:
| AIUI the point of HFT isn't to trade frequently, but rather
| to change offer/bid prices in the smallest possible steps
| (small along both axes) while waiting for someone to accept
| the offer/bid.
| kolbe wrote:
| Seems like a testable hypothesis. Choose the four times a
| year that the stock is at a fair price, and buy when it goes
| below it and sell when it goes above it?
| idohft wrote:
| How far have you tried to tell, and do you buy/sell stocks?
|
| There's someone on the other side of your trade when you want
| to trade something. You're more likely than not choosing to
| interact with an HFT player at your price. If you're getting a
| better price, that's money that you get to keep.
|
| *I'm going to disagree on "free pass" also. HFT is pretty often
| criticized here.
| jeffreyrogers wrote:
| Bid/ask spreads are far narrower than they were previously. If
| you look at the profits of the HFT industry as a whole they
| aren't that large (low billions) and their dollar volume is in
| the trillions. Hard to argue that the industry is wildly
| prosocial but making spreads narrower does mean less money goes
| to middlemen.
| sesuximo wrote:
| Why do high spreads mean more money for middlemen?
| Arnt wrote:
| When you buy stock, you generally by it from a "market
| maker", which is a middleman. When you sell, you sell to a
| market maker. Their business is to let you buy and sell
| when you want instead of waiting for a buyer/seller to show
| up. The spread is their profit source.
| sesuximo wrote:
| Wouldn't the price movement overwhelm the spread if you
| sell more than a few days after you buy? I guess if
| spreads were huge it would matter more
| Arnt wrote:
| The price movement does indeed overwhelm the spread. Half
| the time it goes up, half the time down.
| pi-rat wrote:
| Generally gets attributed with:
|
| - Increased liquidity. Ensures there's actually something to be
| traded available globally, and swiftly moves it to places where
| it's lacking.
|
| - Tighter spreads, the difference between you buying and then
| selling again is lower. Which often is good for the "actual
| users" of the market.
|
| - Global prices / less geographical differences in prices.
| Generally you can trust you get the right price no matter what
| venue you trade at, as any arbitrage opportunity has likely
| already been executed on.
|
| - etc..
| munk-a wrote:
| > Tighter spreads, the difference between you buying and then
| selling again is lower. Which often is good for the "actual
| users" of the market.
|
| I just wanted to highlight this one in particular - the
| spread is tighter because HFTs eat the spread and reduce the
| error that market players can benefit from. The spread is
| disappearing because of rent-seeking from the HFTs.
| mkoubaa wrote:
| Why does it exist?
|
| Because it's legal and profitable.
|
| If you don't like it, try to convince regulators that it
| shouldn't be legal and provide a framework for
| criminalizing/fining it without unintended consequences, and
| then find a way to pay regulators more in bribes than the HFT
| shops do, even though their pockets are deeper than yours, and
| then things may change.
|
| If that sounds impossible, that's another answer to your
| question
| munificent wrote:
| _> The noted efficiency in compile-time dispatch is due to
| decisions about function calls being made during the compilation
| phase. By bypassing the decision-making overhead present in
| runtime dispatch, programs can execute more swiftly, thus
| boosting performance._
|
| The other benefit with compile-time dispatch is that when the
| compiler can statically determine which function is being called,
| it may be able to inline the called function's code directly at
| the callsite. That eliminates _all_ of the function call overhead
| and may also enable further optimizations (dead code elimination,
| constant propagation, etc.).
| xxpor wrote:
| OTOH, it might be a net negative in latency if you're icache
| limited. Depends on the access pattern among other things, of
| course.
| munificent wrote:
| Yup, you always have to measure.
|
| Though my impression is that compilers tend to be fairly
| conservative about inlining so that don't risk the inlining
| being a pessimization.
| binary132 wrote:
| The real performance depends on the runtime behavior of the
| machine as well as compiler optimizations. I thought this talk
| was very interesting on this subject.
|
| https://youtu.be/i5MAXAxp_Tw
| astromaniak wrote:
| Just in case you are a pro developer, the whole thing is worth
| looking at:
|
| https://github.com/CppCon/CppCon2017/tree/master/Presentatio...
|
| and up
___________________________________________________________________
(page generated 2024-07-08 23:00 UTC)