[HN Gopher] Case Study: Algorithmic Trading With Go
___________________________________________________________________
Case Study: Algorithmic Trading With Go
Author : WestCoastJustin
Score : 255 points
Date : 2023-06-30 17:49 UTC (5 hours ago)
(HTM) web link (polygon.io)
(TXT) w3m dump (polygon.io)
| spockz wrote:
| Excellent write-up! Is there somewhere where I can read up on all
| the trading jargon?
|
| Also I wonder how can there be changes in the price of a stock
| after market if the exchange has closed? Isn't the whole point
| that trades need to happen for stocks to get a certain value?
| WestCoastJustin wrote:
| https://www.investopedia.com/ is probably the best resource and
| chatgpt. I honestly, just google stock related terms and all
| roads lead to investopedia.
| xyzelement wrote:
| There's after market trading on venues. But because most of the
| liquidity has "gone home" you can't transact as much at those
| prices and those prices are based on relatively few/small
| transactions.
| WestCoastJustin wrote:
| Yeah, this is where a system like this shines. In that you
| can build a tool, like top, but for the stock market. You
| have all the real-time data. So, you can look at where the
| party is happening at. Just by looking at the trade flows.
| This is where building your own system absolutely kills
| anything else.
| cempaka wrote:
| While I was laid off and looking for work, I connected with a
| cryptocurrency market making firm that had access to a private
| broker feed which is not attached to any matching, so would
| occasionally go crossed and offer a pure arb opportunity for the
| same pairs. I had done some algo trading with Java in Kospi 200
| options in 2011-2012 so I decided to put together a simple bot
| for them to try and grab the crossed markets when they occur.
| Even an incredibly simple trade like this requires quite a lot of
| work to get the risk management in place. I also took it as a
| chance to catch up on new Java features since I had been out of
| that ecosystem for awhile.
|
| It did successfully grab the arbs but there wasn't enough juice
| to justify more work on it and I got a job in the meantime, so I
| open sourced the whole thing: https://github.com/abissell/cempaka
| carabiner wrote:
| Cool. Did you think Java was a good choice for this kind of
| work? Why not Go like OP or something else?
| cempaka wrote:
| I mostly went with Java because it's what I knew best, and I
| thought was likely to be "fast enough" for the market I was
| in. I'm sure Go would be similar in this case but didn't want
| to have to take the time to learn it.
|
| I do think that if Java can deliver on the combination of the
| foreign function/memory interface and value types, it might
| really start to look competitive for certain strategies which
| are just a bit too complex for the "do everything in the
| network card" approach. When the Aeron guys first implemented
| their protocols in Java, C#, and C++, C# was actually the
| fastest, which they attributed to the presence of both the
| optimizing runtime and value types.
| afterburner wrote:
| I see the new bull market is bringing back these kinds of
| articles.
| WestCoastJustin wrote:
| Yeah, I've been messing around with this for 4+ years so. Got
| to see COVID nose dive, meme stocks, massive build up, market
| taking a dump, bank collapses, and now a bull run again. I have
| no idea if it was always like this but it has been in insane.
| joshu wrote:
| also having worked in the space: HFT should not exist. Break up
| the day into segments and have a single crossing. Do it every
| five minutes or whatever for sufficient timeliness. the
| millisecond race does not make anything better for anybody except
| the people doing the trading.
| alchemist1e9 wrote:
| Ridiculous. This opinion is held by those who couldn't cut it
| in the field because they don't understand it. Out of
| bitterness they reach this conclusion.
|
| HFT has consistently driven economic costs lower and lower and
| lower. Of course people can point to bad actors and apples here
| and there, but overall it performs a valuable economic function
| which the alternative suggested would not do as efficiently.
|
| To understand this requires understanding markets very deeply
| and can't be debated using liberal arts rationalizations.
| grog454 wrote:
| > but overall it performs a valuable economic function
|
| Can you explain what that is?
|
| > To understand this requires understanding markets very
| deeply
|
| Apparently you aren't the only person on this forum who does.
| Are we just supposed to take your word for it?
| mrcode007 wrote:
| He's referring to the fact that the HFT "provides
| liquidity" as in getting a share off your hands and
| flipping it to someone else that is not you a moment later.
| This way you didn't have to wait 3 seconds or 1 minute or
| whatever to sell your shares because someone knew they
| could flip them (alpha) on a short notice so they
| participated in a transaction with you. He's assuming that
| without the HFT, the transactions wouldn't be as frequent
| and you'd have to either a) wait for the fill on your trade
| , or b) lower your ask price if selling under time
| deadline. Analogously for buy orders.
|
| The only parties with semi-global visibility are prime
| brokers by definition; they see every position of everyone
| who custodies with them.
|
| The parties with global visibility of the US market are TRF
| (trade reporting facility) and those are the only parties
| who can sort of evaluate the HFT claims without bias or
| vested interest. Most of the studies in the field have some
| sort of an angle or vested interest so it's hard to
| evaluate the veracity of the claims one way or another.
|
| A counterpoint to HFT is that stock markets existed before
| the the advent of computers and they had runs, panics and
| blow ups just like regular markets do now.
| alchemist1e9 wrote:
| I don't have time to reply properly until a few hours
| from now.
|
| In the meantime what I can say very simply in the hope
| that someone else knowledgeable can contribute earlier.
|
| It's simply an empirical fact that the costs of
| intermediation to the system are the lowest they have
| ever been. The US and other global electronic markets are
| incredibly efficient and deliver unmatched liquidity,
| information efficiency, and the lowest costs to the
| entire capital markets than at anytime in history. That
| march forward is continuous and brutally competitive
|
| There are many many way to see this and measure it, any
| serious quantitative analysis, by professionals, for
| instance trained in econometrics and with access to the
| raw data, like those at say the Fed, or those operating
| exchanges, as examples.
| ye-olde-sysrq wrote:
| I think you would still end up with races, they'd just move
| around. IEX's "speed bump" was just marketing and didn't really
| matter to actual HFTs.
|
| Plus, some of it is unavoidable if you don't have a single
| unified exchange. Where there's latency, there's
| inefficiencies, and where there's inefficiency, there's profit
| to be made. And competition among exchanges is healthy for the
| ecosystem, so I don't think we'd want to consolidate.
|
| And lastly, HFT has consolidated so much that I don't think
| it's worth worrying about. Virtu literally switched sides and
| make most of their money on order execution. Industry-wide, HFT
| revenues are down like 80% over the last 5-8 years. Between
| wholesaling/PFOF taking non-toxic order flow off the lit
| exchanges, and banks finally wising up on not being pants-on-
| head about their order execution, it's literally just sharks in
| the pool now, there's not even any water.
| HWR_14 wrote:
| Why is competition among exchanges a good thing? Why not a
| single, very regulated, exchange which serves as a neutral
| playing field?
| tick_tock_tick wrote:
| Single point of failure is a simple justification right off
| the top of my head. What happens if you have a critical
| failure that brings down an exchange for several days?
| HWR_14 wrote:
| If the NYSE went down, wouldn't most trading stop because
| of people being frightened about using old mismatched
| prices?
|
| Besides, when has an exchange gone done for hours, let
| alone days? Absent intentional breaks in trading as
| speedbumps.
| chollida1 wrote:
| > If the NYSE went down, wouldn't most trading stop
| because of people being frightened about using old
| mismatched prices?
|
| Why would you assume that.
|
| Every exchange is a valid place to trade and the sip
| ensures you always have the correct NBBO
| HWR_14 wrote:
| I assumed that some exchange provides enough volume that
| without it there would be worries that you aren't getting
| true price discovery. Isn't that a concern with low
| trading volume? Not that that would necessarily be true,
| but there would be enough concern to cause some people
| not to trade. And others to want to trade specifically
| with those kind of nervous people wouldn't because the
| people they expected to profit by were staying put. And
| then it would cascade.
| quickthrower2 wrote:
| As a counter I had some VW shares I couldn't access for 6
| months because they were moved to another exchange. 20
| calls to the broker to find out what had even happened.
| Eventually I was able to sell them.
| tomcam wrote:
| Well, your first problem was investing in VW (KIDDING!).
| But seriously, I usually stick to self-serve stock
| purchases. Was this a situation where you had to contact
| a human to make the buy in the first place, or was it
| that you bought it yourself, then VW changed up their
| infrastructure and inadvertently locked you out?
|
| Shitty situation either way.
| TheAlchemist wrote:
| What do you mean << moved to another exchange >> ?
|
| Shares are not held at an exchange.
| xoranth wrote:
| Lower fees.
|
| Also, "exchange competition" is a thing in US stocks, but
| not, for example, in the futures' market. And there's HFTs
| in futures too, so having a single exchange wouldn't
| "remove" HFTs (not that you'd want to).
| auntienomen wrote:
| Running the matching engine at discrete times doesn't eliminate
| the speed race, as that's mainly about jockeying for queue
| position. You could put in an auction instead, but that's just
| changing the rules of the game rather than eliminating the
| game.
|
| But really, the key thing is that liquidity provision actually
| is a service -- market makers are basically selling insurance.
| There's money to be made doing his, so people will compete to
| do so. That's what the millisecond race is about. Faster
| trading -> less risk for the MM -> less capital needed -> lower
| profits acceptable. Contra what you're claiming, the race is
| cutting into profits, not raising them.
|
| If you want to reduce market-maker profits, crack down on
| payment for order flow, and let everyone compete for a chance
| to trade against it.
| ilyt wrote:
| ...that's kinda entire stock exchange.
|
| It could be all run once a day if it was just to give ability
| to raise funds via investment
| Ntrails wrote:
| We tried this in Taiwan. They reverted to continuous matching.
| WestCoastJustin wrote:
| Amazing, I just checked out your profile. I used del.icio.us
| religiously back in the day. Thank you!
| blibble wrote:
| periodic auctions are common on european exchanges (pesky
| limits on dark pool volume)
|
| instead of order entry + continuous matching: order entry is
| allowed, but no matching (like pre-open phase)
|
| then after a random period of time the auction algo runs on the
| entered orders
|
| then repeat the entire thing every second
| [deleted]
| Eddy_Viscosity2 wrote:
| There should be transaction fees on all trades on all exchanges
| that go to the government, yes I mean a tax.
| alchemist1e9 wrote:
| Why?
| ye-olde-sysrq wrote:
| I worked in HFT for a while. I find the whole space fascinating.
| I'm glad OP found similar thrills.
|
| "This aspect, the platform itself, seems to be often overlooked
| in most discussions. Many conversations revolve around strategies
| (mean reversion, trend following, linear regression, etc.), and
| backtesting, without fully addressing the practical mechanics or
| logistics of strategy implementation, particularly in the context
| of live, intraday trading."
|
| I'm glad you had fun, OP, but also I think I can shed some light
| on why most people discuss strategy.
|
| Trading is a perfect storm of ridiculously high tech,
| ridiculously complicated, ridiculously regulated (Not _over_
| -regulated, mind you, this isn't a value judgement. But the
| amount of regulation _is_ extremely high.), and ridiculously
| competitive.
|
| But that said, it's the last bit that drives it all. Since it's
| so competitive, even though building an order entry system, and a
| risk system, and a position-tracking system, etc is a huge
| accomplishment (again, congrats OP!), it's table-stakes to even
| dip your toes in the pool here. Trading shops can attract top
| talent and robust, bespoke trading systems are basically cost of
| entry.
|
| So people talk about strategy because everyone already _has_ the
| table-stakes stuff and are now trying to make money with it.
|
| It doesn't help, too, that lots of market participants aren't
| even playing the same game. In HFT, we operated on trades with
| alphas that lasted a few seconds, where races to entry/exit were
| battled in shaving nanoseconds off FPGAs being able to shoot out
| orders and microseconds off wireless networks flying market data
| around new jersey. Meanwhile, banks are more concerned with
| elections and geopolitics than they are about the weather in
| Carteret. (Rain = no microwave network for the day). And then
| there's a million strategies in the middle with alphas that last
| from hours to weeks.
|
| So it makes it really hard to even speak the same language to
| each other when talking in common forums.
|
| It's a fun world. I miss it sometimes.
| wwilim wrote:
| Sample size one. Contracting for a trading shop for 2 years
| taught me that it's not technical aptitude that keeps the
| business rolling.
| mr_o47 wrote:
| If you don't mind me asking how did you land a contract gig
| at trading company
| blibble wrote:
| however ineptitude very quickly brings the entire thing down
| skummetmaelk wrote:
| Seems more pointless than crypto to be honest.
| WestCoastJustin wrote:
| YES, 100%! I honestly, cannot agree more. I've read pretty much
| every book that I could get my hands on and no one talks about
| the platform. This makes so much sense. Thank you for your
| perspective. It seems like so much of this stuff is silod off
| into each company and no one is talking about it.
|
| HFTs are definitely playing a completely different game. I was
| reading about the exchange architectures and how things are
| actually wired. I'm getting my data from SIPs while HTFs are
| directly connected to the exchanges [1]. I'm transacting in
| seconds and they, like you said, are transacting in
| microseconds, so there is no comparison. Which, in a way is
| actually nice in that I'm not really competing with them. Or,
| maybe I am but I can still make some money. haha.
|
| Cheers and thanks for the awesome comment!
|
| [1] https://www.researchgate.net/figure/Latencies-in-the-
| Electro...
| kasey_junk wrote:
| Not only are the platforms table stakes they are the more
| straightforward part to build. Even at the bleeding edge of
| latency you can usually work your way to the limits of your
| platform budget without having to find anything novel.
|
| The strategies though are where the discovery is. There are a
| few strategies that are well known and still profitable but
| those are largely consolidated to the biggest firms. For
| everything else it's a discovery process. And done strategies
| are only profitable for very short regimes.
|
| I miss it sometimes too, but so much has been consolidated it's
| largely a big firm world now.
| ye-olde-sysrq wrote:
| "they are the more straightforward part to build"
|
| More straightforward, heh, sure. But still damnably
| complicated. Which just goes to show how much money and how
| much engineering talent is invested in this world that these
| things are so taken for granted.
| ilyt wrote:
| So many intelligent people applied for something that has
| no value whatsoever to humanity regardless of how you look
| at it.
| allenrb wrote:
| Essentially yes. When I worked in algo trading, it never
| bothered me that we were extracting profits from the
| markets, nor that we served little social good. It felt
| like a step up from where I'd been before (being told
| that we were making the world a better place, when every
| engineer knew otherwise.) At least we weren't making
| things obviously worse.
|
| What did bother me, and was acknowledged by my coworkers,
| was how much top talent was being pulled away from
| productive tasks to essentially wank around. Not just
| technologists either, but all sorts of mathematicians and
| scientists were drawn to the flame.
|
| We as a society have managed to allocate so many of the
| "best and brightest" to either fintech wankery or placing
| ads in front of eyeballs. It's enough to make one want to
| give up on capitalism, except that everything else
| appears to be worse.
| tomcam wrote:
| Seems like every person I've ever met in M&A has ended up
| hating life because of their career choice. They seem to
| burn out in 5 years or so after salting away a million
| bucks or 3. Was that your feeling doing the HFT thing?
| finexplained wrote:
| I work at an HFT firm. Most fun I've ever had.
| [deleted]
| makestuff wrote:
| This is really interesting. Have your strategies out performed
| buy and holding index funds, or are you mainly just doing this
| with a small amount of capital to learn how the markets work? I
| have always wanted to try algorithmic trading to learn about it,
| but I have always read it is a fools errand to think you will
| beat just buy and hold.
| nemothekid wrote:
| 1. I think buying and holding is more probable to have a higher
| return
|
| 2. Sometimes you get a cool api and think wow this would be
| fun, and next thing you know you've lost thousands on
| boneheaded trades.
|
| I did something similar during the pandemic with Rust with the
| Polygon API (and instead of interactive brokers, I used
| tradier). Eventually I learned I actually had more fun building
| the thing than actually trying to beat the market.
| WestCoastJustin wrote:
| I have won and lots thousands for sure. Haha. When stocks
| were on a rip the bot was making lots of money just because
| everything was going way up. Then, in 2022 when everything
| went way down, like tons of tech stocks, my bot sucked. So, I
| really need to add shorting or something. I'm still exploring
| things on the strategy side.
| pqdbr wrote:
| Isn't the trend detection equally applicable to shorting as
| well, with swapped signals?
|
| Or do you believe there are more fundamental changes needed
| so your app can trade in shorting as well?
| WestCoastJustin wrote:
| Yeah, right now I'm only set up to buy stocks. I haven't
| tried to short anything yet. I want to get to this
| eventually since it would be nice to make money when the
| market goes down too.
| Fripplebubby wrote:
| I'm sure you know this, but for others reading this who
| are novices at finance / trading, like I am - the gotcha
| here is that while the strategy may be symmetric, the
| risk is not - when buying a stock, there is a floor to
| how much money you can lose (the price you paid for the
| stock), while with short-selling, there is no such floor,
| since the price can rise to any amount and increase your
| losses to infinite. I believe that traders will use
| hedges to account for this, however, these hedges will
| eat into your profits if they are not exercised (but may
| save your bacon if they are!).
| stouset wrote:
| Aka, everyone thinks they're an investing genius when the
| whole market is going up.
| WestCoastJustin wrote:
| Yes, 100%. Everything looks amazing while the markets are
| going up. Sometimes, I've just shut the entire thing down
| when there is Fed news or the markets are taking a dump.
| That's a legit strategy too. Only run it while the
| markets are going up.
| mhh__ wrote:
| Out perform relative to a number or your expectation?
|
| You might want returns that aren't correlated just to an index
| - this is a major reason to look to invest in (say) a hedge
| fund.
| WestCoastJustin wrote:
| This is going to sound crazy given all the scams out there. But
| I was interested in testing the idea of small compounding
| returns. Like, could you get a daily 0.5% compounding return.
| Sure, you could go all in on TSLA for example and get a 1%
| daily return. But, could you do that with automation, using
| lots of small bets, across the entire market. You can, but
| there is a scale issue here. In that you need to make
| exponentially large bets as your bank roll increases. So, it's
| capped. Well, that's what I've seen. So, yes you can beat it
| but only with smaller amounts of money.
| rgbrenner wrote:
| Just want to point out with 260 working days in the year:
|
| 1.005^260 = 366%
|
| and
|
| 1.01^260 = 1329%
|
| In case anyone sees "0.5%" _daily_ and thinks _low risk_.
| WestCoastJustin wrote:
| Yeah, I'm not saying this is low risk. It's more about
| trying to build a system to be in the right place at the
| right time. It is massively risky. I was sort of hesitant
| to even write that comment because you constantly see all
| these youtube day traders selling courses on making 1%
| daily. If you try that you'll lose money extremely quickly.
| the__alchemist wrote:
| > Like, could you get a daily 0.5% compounding return
|
| It seems that this is the key to your approach. How is this
| part achieved?
| WestCoastJustin wrote:
| That's sort of the secret sauce. But, having a platform
| like this is more than 75% of the solution. The rest is
| more around trend following.
| the__alchemist wrote:
| I tend to frame it as the _secret sauce_ is 99% of the
| solution, and the platform /API-interaction is _trivial_
| , as the term is used in science.
| WestCoastJustin wrote:
| Yeah, you could look at it like that too. But you need to
| have a platform before you can even do something like
| this. So, that's why I framed it like that.
| stouset wrote:
| It's also stunningly easy to convince yourself you have a
| profitable algorithm when you actually have a money
| loser.
|
| Gaining 0.5%/day 60% of the time and breaking even 39% of
| the time looks great until you run into the 1% of the
| time where you lose 50%.
| WestCoastJustin wrote:
| Yes, 100%. Like I said in the article there. I once lost
| like 40% in a few minutes in the pre-market because my
| app didn't have the correct sell logic. Honestly, I don't
| even know if I had the correct logic that would have even
| saved me, because the market was falling so quickly I
| probably could not have even for the fills I wanted. So,
| I've moved to breaking things into tons of small bets and
| really trying to manage risk.
| mhh__ wrote:
| https://en.m.wikipedia.org/wiki/Martingale_(betting_system)
| makestuff wrote:
| That makes sense to me and lines up with what buffet says
| about there are alot of people on wall street who can average
| 50% returns with 100k but once it gets into the millions it
| is much harder to find alpha.
| cholmon wrote:
| I don't follow Berkshire Hathaway much, so I'm not aware of
| what Warren Buffet has said on this topic, but it sounds
| super interesting.
|
| After a bit of digging, I found this Q&A from the 2019
| shareholder meeting:
| https://www.youtube.com/watch?v=geRIJQJXRVo&t=17980s
|
| And the meeting minutes in PDF form (see page 120, question
| #32): https://s3.amazonaws.com/static.contentres.com/media/
| documen...
|
| The text from that document...
|
| 32. It's easy to make 50% on a million, but much more
| difficult on larger amounts
|
| WARREN BUFFETT: Station 9. We're just about -- yeah, we've
| got time for a couple more.
|
| AUDIENCE MEMBER: My name is John Dorso (phonetic), and I'm
| from New York. Mr. Buffett, you've said that you could
| return 50 percent per annum if you were managing a one-
| million-dollar portfolio. What type of strategy would you
| use? Would you invest in cigar butts, i.e., average
| businesses at very cheap prices? Or would it be some type
| of arbitrage strategy? Thank you.
|
| WARREN BUFFETT: It might well be the arbitrage strategy,
| but in a very different, perhaps, way than customary
| arbitrages, a lot of it. One way or another, I can assure
| you, if Charlie was working with a million, or I was
| working with a million, we would find a way to make that
| with essentially no risk, not using a lot of leverage or
| anything of the sort. But you change the one million to a
| hundred million and that 50 goes down like a rock. There
| are little fringe inefficiencies that people don't spot and
| you do get opportunities occasionally to do, but they don't
| really have any applicability to Berkshire. Charlie?
|
| CHARLIE MUNGER: Well, I agree totally. It's just you used
| to say that large amounts of money, they develop their own
| anchors. It gets harder and harder. I've just seen genius
| after genius with a great record and pretty soon they've
| got 30 billion and two floors of young men and away goes
| the good record. That's just the way it works. It's hard as
| the money goes up.
|
| WARREN BUFFETT: When Charlie was a lawyer, initially, I
| mean, you were developing a couple of real estate projects.
| I mean, if you really want to make a million dollars -- or
| 50 percent on a million -- and you're willing to work at it
| -- that's doable. But it just has no applicability to
| managing huge sums. Wish it did, but it doesn't.
|
| CHARLIE MUNGER: Yeah. Lee Louley (phonetic), using nothing
| but the float on his student loans, had a million dollars,
| practically, shortly after he graduated as a total
| scholarship student. He found just a few things to do and
| did them.
| tiahura wrote:
| So flip houses and do the work myself?
| mhh__ wrote:
| One of the issues arbitrageurs have is that as they get
| good and bigger they can run out of things to arbitrage.
| oh_sigh wrote:
| Are there any examples out there ofactual algotrading strategies
| that made money in a real market, say within the past 10 years? I
| know these are closely guarded secrets, but figured someone may
| have published a strategy that ceased being profitable.
| nurettin wrote:
| I love IB. They give you a papertrading account as a testbed. The
| executions are pretty close to reality, even in low liquidity
| markets.
| mr_o47 wrote:
| I'm always fascinated by fintech especially when it comes to
| technology and trading and the author did a really great job of
| explaining the concepts
| WestCoastJustin wrote:
| Thank you!
| pyrrhotech wrote:
| One of the most misunderstood concepts about algorithmic trading
| is that speed is not a critical factor for the majority of
| systems. My systems like https://grizzlybulls.com/models/vix-ta-
| macro-mp-extreme have been crushing the market with live trading
| for 3+ years and yet trade on a frequency of only once ever 18
| trading days on average (some a little more often, some even
| less), and are set up to only generate signals around hourly
| breakpoints.
|
| The last 18 months have been weaker than the first given the
| enormous structural shift in the market in this high inflation
| and rapidly rising interest rates environment, but we've still
| managed to deliver a return of +14.11% since the site launched in
| Jan 2022 compared to -7.83% for the SPX. We've managed to do it
| without any use of leverage and also with lower drawdowns as well
| of -16.48% vs. -27.57% for the SPX over that time frame.
| TechBro8615 wrote:
| Nice writeup. I appreciate the concrete discussion and especially
| the screenshots. I understand not discussing strategies, but can
| you elaborate on whether your trading is entirely based on
| technical analysis, or do you use external (or "alt") data feeds?
| In other words, is this a mostly closed system with inputs from
| Polygon and outputs to the IB API, or do you have a more
| extensive setup of custom data feeds of news sites, twitter,
| reddit, etc? If so, I'd be curious what challenges you face
| balancing those with historical backtesting, when some data
| sources might only have partial historical coverage.
| WestCoastJustin wrote:
| All my data is from polygon.io and nothing external. I have
| been exploring using lookup tables of pre-computed historical
| values to do things like anomaly detection (ie, is this normal
| activity for this stock). A good example would be BXRX today
| [1]. I have been looking at options trading activity too and
| trying to use that as a signal.
|
| [1] https://www.google.com/search?q=BXRX
| marsupialtail_2 wrote:
| Hi Justin, you might be interested in my blog:
| https://github.com/marsupialtail/quokka/blob/master/blog/bac...
| advocating a cloud based approach.
|
| You don't have to use the system I am building, but it's worth
| thinking about that design.
| WestCoastJustin wrote:
| Cool, thanks. I'll check it out!
| sneak wrote:
| Slightly related: when running 3 screens like that, put the
| middle (primary) one in landscape, so the array is like a big H.
|
| It works better for videos and code, IMO. You still have two huge
| portrait fields for documentation/webpages/etc, or 8 portrait-
| oriented quarters.
|
| I usually run 8 portrait-shaped windows quartered on the sides,
| then two side by side on the middle, which are each approximately
| square.
|
| I don't like my editor window getting too tall.
| WestCoastJustin wrote:
| Yeah, personally, it's been a game changer just being able to
| load so much more into your field of view. Having two side-by-
| side makes it really easy to look at other parts of your code
| and make changes. I'm loving it! I'll give the H method a try
| for a few days and see how it goes.
| magundu wrote:
| What is h method?
| WestCoastJustin wrote:
| Oh, sorry. I was referencing the OP comment there.
|
| > 3 screens like that, put the middle (primary) one in
| landscape, so the array is like a big H.
| ilyt wrote:
| Eh, honestly only thing horizontal is good for is media, any
| documents/code is better in narrower vertical columns.
|
| Honestly my perfect screen _for work_ would probably had
| something like 1:1 ratio, so I can have 2 nice columns of code
| that are tall enough that can be split horizontally if needed
| while still being useful.
| ghoshbishakh wrote:
| Who noticed the Chat GPT running on the right screen?
| pqdbr wrote:
| If you read the entire article, chatGPT is mentioned as
| something that increased OP's productivity 3x.
| botdan wrote:
| If anyone from Polygon is reading, there's a typo in the URL:
| https://polygon.io/blog/case-study-algorithmict-trading-with...
| (algorithmict has an erroneous "t" on the end). Normally I
| wouldn't mention this but the corrected spelling 404's.
| 0xdeadbeefbabe wrote:
| Has the network or the power ever gone out at a bad time?
| WestCoastJustin wrote:
| Yes, both. Basically, I learned early that I needed a backup
| power supply. I have a couple that power my computer, monitors,
| and the internet router. For the network, that one is really
| tricky. In that, you're screwed. I've only had this happen once
| and then I needed to manually sell all my positions. Luckily, I
| didn't lose any money.
| gochi wrote:
| >I went from not knowing how to solve a problem, blindly googling
| around and reading books, to just telling ChatGPT the problem,
| and then asking how it would solve it, then asking it to code the
| solution. This is absolutely insane and has easily 3x my
| productivity.
|
| Very interesting. We can facetiously say that ChatGPT is using
| you as a medium between setting up algorithmic trading in Go!
| victorbjorklund wrote:
| Nice. Been wanting to build a trading bot in elixir.
| RomanPushkin wrote:
| Go itself, or any other language won't give you too much of an
| advantage. What gives you advantage is trading algo, which is
| always hard to find. I've spent months on figuring out the best
| parameters for trading. Ended up this working only on historical
| data, while in reality it was totally different.
|
| I could use Visual Basic, and it would be better than Go, Rust,
| or whatever it is out there, given the algo and strategy are
| flawless. Language is just a tool. It's great you used Go, but I
| think the title is a bit misleading - people think of it as some
| kind of advantage. It isn't.
|
| And for HFT trading a language with Garbage Collector is not a
| great choice IMO.
| asavinov wrote:
| > _What gives you advantage is trading algo, which is always
| hard to find._
|
| At the end it is necessary to make a decision whether to buy or
| sell (and how much), which will compete with other decisions
| made based on some logic. Developing such a logic (strategy)
| manually is of course quite difficult. I developed an
| intelligent trading bot which derives its trading strategy from
| historic data:
|
| https://github.com/asavinov/intelligent-trading-bot
|
| Currently it works for cryptocurrencies but can be applied to
| other markets:
|
| https://t.me/intelligent_trading_signals
|
| > _I 've spent months on figuring out the best parameters for
| trading. Ended up this working only on historical data, while
| in reality it was totally different._
|
| It is a typical situation. The whole problem is to develop a
| strategy which works for future (unseen) data. Even backtesting
| algorithms should be designed in such a way that future
| experience (data) does not leak to the past.
| IshKebab wrote:
| You overfitted. That can happen due to human optimisation as
| well as computer optimisation. After you have experimented on
| some historical data for a while it becomes less useful.
| WestCoastJustin wrote:
| Hey, I write this. I did program this in Go. The reason is that
| it's my go to programming language. Using it is actually a
| disadvantage in that the industry uses C++ and Python but
| that's just what I know how to program in. I wasn't trying to
| be misleading. Go does work really well for taking in data,
| doing something with it, and then calling a remote API. So, it
| actually works really well but if you wanted to get a job based
| off this it probably wouldn't help you.
| hu3 wrote:
| Go is great for network heavy apps like algo trading!
|
| I used Go to write trading algos that would find small
| windows of triangle arbitrages in crypto exchanges. Made me
| some money but the risk of a big loss made me stop pursing
| crypto trade and it required too much time and attention.
| It's a full time job from my experience. Reasons I could lose
| big at any given time if I scaled up the stakes:
|
| - Exchanges temporarily pausing some specific crypto trading
| for N reasons (happened very often) while I'm in the middle
| of the arbitrage.
|
| - Getting caught in the middle of a pump and dump event (also
| frequent)
|
| - Any algo mistake that would perform excessives trades in
| succession would incur huge losses because crypto exchanges
| charge %.
|
| Also, most cryptos have too low volatility. Every time I
| tried to scale my bot would start interferring oo much with
| the market. And it wasn't even much money.
| moneywoes wrote:
| Have you tried non crypto markets
| tomcam wrote:
| It was my (admittedly ignorant) impression that C++ was
| preferred over garbage-collected languages because you'd be
| more likely to avoid GC pauses. Yes, of course I know that
| Go's is super efficient, but if you're doing a million
| trades a day even a tiny percentage of slowdowns that would
| be otherwise preventable could be a career-limiting move.
| Am I just behind the times?
| hu3 wrote:
| I think you're right! Go will never beat C++ in this
| aspect.
|
| I used Go because that's what I knew and for the non-
| professional trading I was aiming, C++ wouldn't have made
| much difference. My bottlenecks were network (1s+ per
| trade roundtrip) and chaotic unreliable crypto markets.
|
| Just note that Java is used in HTF, but it's a different
| beast than our average CRUD Java. For example this
| article states:
|
| "Essentially, we use a contrived form of Java that avoids
| all the Java constructs that make things go slow. We only
| use the constructs that are fast and efficient, and we
| avoid all the garbage."
|
| https://www.efinancialcareers.co.uk/news/2020/11/low-
| latency...
| gkedzierski wrote:
| I think there's a way to disable GC in Go too, no?
| cachvico wrote:
| A language should be considered with its ecosystem, which is
| why the author started delegating parts to Python by the end.
|
| Therefore the title was actually on point.
| pdimitar wrote:
| Really funny coincidence that I am seeing Interactive Brokers
| mentioned in this good article. Story + rant time, feel free to
| skip if you are not interested how one guy gambled and lost. It's
| also a tentative call for partnership if somebody is interested.
| And a call for chat if anyone has any interest in the topic.
|
| _(It 's also kind of off-topic, my apologies for that. To me it
| seems semi-related but would agree with mods' assessment if it
| doesn't match mine.)_
|
| ---
|
| I recently "broke up" with some extremely toxic "investors" that
| wanted me to do a fully parallel trading demo -- meaning it
| receives ticks for N instruments (I successfully got to little
| less than 300) and trades with each of them depending on
| strategy. All in real-time.
|
| I got very far but the open-source libraries for Interactive
| Brokers are quite low quality in general and it was very hard and
| slow to progress (one of them couldn't even post orders, another
| used Mutex-es for "parallelism" which was of course not parallel
| at all, another one seemed to work well but only worked on
| servers of older versions compared to those I have access to,
| etc). I also had to gather code from separate places and assemble
| my own Frankenstein as I went along.
|
| Eventually I muscled through but by that time I have drained all
| my savings, my tax fund and even got a new loan. And the liaison
| + the investors of course refused to acknowledge the demo was
| basically 90% done (couldn't do full parallel trading due to
| defects of the IBKR libraries I have used and I used like 5 of
| them, and was in the process of repairing 2 of them to unlock the
| said full parallelism). They refused to send a small pre-funding
| wire (we're talking something small like $30k - $50k, not
| millions; for the work that was done, namely months of
| professional Rust programming work, that's a -75% discount if we
| look at market rates).
|
| They had all the proof and paper trail they needed to see that I
| was very close but I had to stop because I was literally about to
| be unable to pay rent and bills. They still did not concede.
| Obviously I picked a job and dumped them but I still have some
| regrets because the door is technically still open (they have not
| cut my access to the IB Gateway servers that they own), but other
| factors like now-ruined health are seriously getting in the way
| as well. Not to mention completely shattered trust.
|
| They insist "if you just finish the demo we'll give you money"
| and used every gaslighting technique I knew about (and many I
| didn't know about, so I learned a lot about gaslighting from
| them, lol) to try and coerce me to keep working for free with
| zero guarantees of funding -- but I no longer trust such rich
| investors to fulfill a promise without a binding and legally
| enforceable contract; I am in Eastern Europe, they are in the
| USA, even if they sign contract and violate it I practically
| cannot do anything to them i.e. I can't afford to travel and sue.
| Also they insist to get access to the source code but swear to
| everything that's holy that they will not run away with it and
| never give me a penny. Which is exactly what I think would
| happen.
|
| I had to draw the line at one point. To me it was red flags all
| around.
|
| ---
|
| I made many concessions and I will not make another one until
| they do. I'll be finding a new job soon again since the previous
| contract was agreed upon to be for several months, I helped a
| team accelerate bootstrapping a business-critical product (and we
| did that successfully). And then maybe, just maybe, after I
| settle a bit, I might work an hour or two on this again during
| some evenings. Maybe. And that won't be used to provide the demo
| to these toxic investors -- I'll use it to have a proven
| implementation that I can pitch to other people. These guys don't
| deserve the time of day from me.
|
| I have quite a lot of good code (mostly Rust, but also some
| Elixir and Golang -- I experimented a lot) that interfaces with
| Interactive Brokers and I have many building blocks of a good
| trading bot framework. That kind of creative and thorough work
| needs funding and peace of mind to be finished properly, of
| course. And I can't afford to invest so much energy on this
| without a stable income so I am shifting focus to that for
| however long it might take for me to feel comfortable to invest
| time and energy in this again.
|
| If any business person wants to partner up -- give me a shout.
| Mail is in profile. I also wouldn't refuse fellow techies (or
| anyone else really) stopping by and telling me how stupid and
| naive I was -- I'll agree right away and say that they are right.
| Because truth is truth.
|
| I admit I still can't get over the fact how close I got but I
| can't afford to count the stars while my livelihood is being
| endangered the longer I coast on savings. Dreams don't pay bills.
| And some dreams need more than one person to be fulfilled.
|
| Rant + story over. Thanks for reading if you stuck this far.
| dataangel wrote:
| How did you get into this kind of shady arrangement? This is
| not how it works if you go to a reputable prop firm. Also
| sometimes they already have infra setup for you to build on.
| [deleted]
| pdimitar wrote:
| A liaison of these "investors" found me somehow. I can only
| hypothesize it was through HN.
|
| And yes I was tempted. I also don't have any trading
| credentials. I'm simply a senior programmer with an eye for
| details who always wanted to try his hand at algorithmic
| trading. So yeah, I got hooked. :(
| infamousclyde wrote:
| Fascinating application of the language and a terrific write-up.
| I would presume a GC language would normally be a disqualifying
| factor in real-time trading, but I think I'm coughing up some
| premature optimization, especially with what looks like a pretty
| beefy rig. Congratulations though, this is spectacular.
| chollida1 wrote:
| Nah, if you are retail and using Interactive brokers then
| you'll never notice teh GC pause at all.
|
| Mostly because you'll never be able to respond to each tick as
| by the time the tick gets to you the market has moved.
|
| Use the language that you know and can work with the fastest.
| For retail trading GC vs non GC will never matter at all.
| WestCoastJustin wrote:
| Yeah, I never really looked a GC. Most of my trades take at
| least 500ms+ round trip. It's more in the latency to the broker
| and getting confirmations that I've been trying to tighten up.
| At least that's what I'm seeing right now.
| mhh__ wrote:
| You say this as if memory allocation in general isn't extremely
| slow.
| kasey_junk wrote:
| For trading systems that are still software based they
| absolutely do not allocate or reclaim on the hot path for
| this reason.
| dzink wrote:
| I love finding other people who enjoy the pure feedback loop of
| code, strategy, and money that trading provides for me. I've been
| working on timing and correlations between stocks and indexes as
| a source of alpha and having some success. Now trying to automate
| the process and absolutely loving the work along the way. Not
| trying to play market maker and not jumping between stocks.
| Instead honing in on an accurate model that monetizes a few
| really well. Some tools I find useful: TradingView and Pinescript
| indicators and strategies, excel models with exported data for
| backtesting, Python and Go for the backend machine learning,
| ChatGPT for faster iteration on new code. Would love to talk shop
| if you guys are interested: trading @ dianazink.com
| noname123 wrote:
| QQ for everybody since OP is using IB API which is notoriously
| bad that many wrappers have been written for it (ib-insync).
|
| This is a general question, I'm wondering is there any good
| framework/wrappers out there that one can learn from to code up a
| complex trading application?
|
| Like dealing with all the asynchronous nature of
| process/submitting trading and quotes messages.
| WestCoastJustin wrote:
| Yeah, I ended up taking https://github.com/gofinance/ib and
| rewrote my own wrapper. This took a long time but has been
| stable since. I'm basically only doing buy lmt, sell lmt,
| cancel, and updates orders though. So, the logic is pretty
| simple. Catching all the return messages and structuring them
| correctly took tons of debugging, trail, and error. Basically,
| mapping the messages into the correct orders for state
| tracking.
| yeahwhatever10 wrote:
| I use ib-insync with asyncio and it is surprisingly easy, the
| most difficult is order management. Jason Brownlee's
| SuperFastPython was the most approachable asyncio explanation I
| found.
| dataviz1000 wrote:
| I've been building a bot with Typescript because it is what I
| know and Python because of the tools available.
|
| It is an overwhelming lonely endeavor. With all my other
| projects, I've always worked on teams, although they have always
| been very small teams and most of my work was autonomous -- still
| there was the occasional meeting and stand-ups. I've been working
| on this for six months and thought about bringing a friend
| onboard for no other reason than to not be alone.
| IshKebab wrote:
| How can you hope to compete with the pros? You're not going to
| be faster than them and unless you have several PhDs in maths
| it's unlikely you'll come up with a better algorithm.
|
| The only way I can think of is to get data that they can't.
| Aken wrote:
| This article is well-timed as I was about to use options
| monitoring as an excuse to get into some other technologies.
| Probably not going to make a ton of progress as a side project,
| but I'm always up for a chat!
| dzink wrote:
| Count me in the group of solo algo trading developers. Maybe we
| should connect and chat.
| blobbers wrote:
| Regarding open sourcing: don't be afraid to show your flaws; it
| doesn't have to be lonely if it is open source.
|
| Obviously whatever trading bot you're running separate from the
| actual trading engine itself is somewhat proprietary, but it
| would be great for the community to get more of this type of
| software in the hands of other hackers.
|
| Quantopian / Robinhood tried and failed, and the numerous clones
| since then have been somewhat sub par.
| JacKTrocinskI wrote:
| You might find https://www.quantconnect.com/ interesting! They
| offer a platform for quickly developing and backtesting trading
| strategies. They have a good community and overall do a great
| job. Give it a shot!
| tdiff wrote:
| I find this article to be absolutely pointless and clickbaity. It
| can be boiled down to: "you can use poligon.io for market data
| but algotrading is difficult anyway so there is not much to share
| yet".
| WestCoastJustin wrote:
| Hey, I'm sorry you feel that way and I wrote it. Personally, I
| wanted to share the high-level structure of how you'd build
| your own system. I would have loved to have seen something like
| this when I went down this rabbit hole in that I needed to
| figure it all out myself. What do you think would have made it
| better? I'd be happy to roll that back in.
| silentwanderer wrote:
| FWIW I thought it was a really good overview of the project
| structure and I appreciated the lessons learned as well.
| giis wrote:
| did you use TWS API or Web portal API for IBKR?
| WestCoastJustin wrote:
| Go app -> TWS API (on the desktop client) -> IB
|
| What's cool about this is that you can look at the IB TWS
| client and see things happening in real-time. So, it acts as
| sort of a sanity check. I know they have that gateway too but
| personally I like to look at the client all the time too. My
| workflow is to run the bot and the TWS client side-by-side and
| watch it make trades, see how things are moving, etc.
| WestCoastJustin wrote:
| Happy to answer any questions about this. It's been a side
| project that turned into a full blown obsession. There is nothing
| too secret about the system since it's more about having a solid
| platform that you can plug your strategies into. I'd probably
| even open source it but I'd have to clean up all my hacks :)
| theLiminator wrote:
| Is your system consistently profitable enough for you to live
| off of? How long did it take you to get to this point?
| WestCoastJustin wrote:
| No way. Not even close. That would be the dream though. Taxes
| are insane in Canada. Like 50% insane. Housing and kids just
| eat money. Maybe if I was renting and didn't have a family
| but that's not happening.
| theLiminator wrote:
| Curious if you mind sharing your sharpe/annual return? I
| understand if you don't want to though!
| kirse wrote:
| Do you have an email? I've been building a similar system for
| awhile except in F#, would love to connect.
| WestCoastJustin wrote:
| justin @ sysadmincasts . com. Happy to connect.
| antiviral wrote:
| You may be interested in this: https://www.techtrader.ai/#wall
|
| Excerpt from his site:
|
| "Tech Trader is a fully autonomous trading system live with no
| human intervention or updates, now for over 10 years. It is
| unique from conventional algorithmic systems, not only because
| it actually is fully automated, but because it takes a "human"
| approach to markets. It is not quant. It is not stat-arb. It is
| not high frequency. It is a program that looks at stocks the
| same way a person does but with the cold discipline and
| infinite attention span of a machine. It is analogous to having
| a thousand independent traders each focusing on a single stock,
| as opposed to a single quant manager trying to make sense of a
| thousand datapoints. A person doesn't think through stats,
| correlations, or complex math models when trading, and neither
| does Tech Trader. Tech Trader leverages technology to do what
| human traders do at scale rather than approach markets from the
| point of view of an academic, mathematician, or scientist.
|
| Since its launch in Dec 2012, Tech Trader has been trading live
| capital completely on its own, fully automated in the truest
| sense with no human input, no tweaking, no updates. It is, for
| all intents and purposes, an autonomous hedge fund, one of the
| first to truly trade unsupervised for years on end. Whereas
| many "automated" or "AI" funds may have a hundred scientists
| providing the actual intelligence behind the curtain, the
| creator of Tech Trader consists of just one person - a self-
| taught individual going by the gaming moniker pftq, who created
| the system at age 21 and has long moved on to other interests.
| "
| WestCoastJustin wrote:
| Thanks, I'll check it out!
| dist-epoch wrote:
| No broker API change in 10 years? No bugs?
|
| I call bullshit. Reads more like a scam for trading signals.
| antiviral wrote:
| I don't run that site myself, but a quick search on sec.gov
| shows it's a registered hedge fund:
|
| https://www.sec.gov/Archives/edgar/data/1653903/00016539031
| 8...
|
| I can't say any more about its validity. Ask your financial
| planner if techtrader is right for you :)
| k9jd883jb wrote:
| Awesome write up. I have a similar project in Go myself,
| although I just use minute bar data instead of realtime ticks.
|
| Can you share your approach for plugging in various strategies?
| I quickly learned that having a pluggable strategy system is
| tricky as it could span across multiple layers of the system.
|
| Also, with backtesting, are you storing and replaying all the
| quote/tick data? or just using the historical aggregates?
| WestCoastJustin wrote:
| I wish this was better, but honestly, I'm just hard coding
| them right into the BUY loop. So, I need to restart the app
| anytime I want to change something. That's why I build this
| logic to dump and reload the state into a gob file (go memory
| dump essentially). Ideally, you'd have some type of format to
| write out your algorithm, and then have a way of hot loading
| it or something. But, I don't change these enough to really
| do that yet.
|
| For back testing, I download all raw trades and quotes, and
| put them into 1 file sorted by time (1 file per day). Then, I
| compress them using lz4. This allows me to sort of replay the
| entire market and build up all my intraday backtesting from
| the source. This took me a long time to figure out and build
| but has been so worth it. So, I have an off-line script that
| basically, loops through these files, and replays the market,
| and makes simulated trades, and then spits out what would
| have happened. There is a GUI for that too so you can go in
| an explore the trades and see what triggered the buy and
| sell. I have seen nothing that goes backtesting for intraday
| like this.
|
| This is super inefficient but I'm just building the
| aggregates on the fly. I could probably cache them somewhere
| but it takes maybe 4-5 minutes to replay a days worth of
| trades/quotes and build all this so I haven't bothered yet.
| slashdev wrote:
| Fascinating. Have you considered open sourcing this without
| your algorithms ( empty buy/ sell loops)?
|
| I'd like to play with something like this but I couldn't be
| bothered to build it all from scratch.
| WestCoastJustin wrote:
| Yeah, I can do that. Ping me at justin @ sysadmincasts .
| com and I'll send you a note when I've done it.
| dzink wrote:
| Count me in the group of solo algo trading developers (Go
| and Python and Tradingview Pinescript and other sruff).
| Planning to send you an email but maybe we can form a
| group/discord or something online.
| devgoth wrote:
| sign me up as well! I would def be interested too.
| TacticalCoder wrote:
| You connect to IB's TWS API... If you execute your trade there
| only, wouldn't it be an option to fetch the price in real-time
| from the IB API and not having to use polygon.io at all?
|
| I have no idea: does IB send the price feed in real time? (they
| certainly send the data in real-time to TWS as it constantly
| updates right? But is the order book available through their
| API?)
|
| Basically and even though I know this was published on
| polygon.io's blog, would that work by only using IB / TWS's
| API?
| WestCoastJustin wrote:
| Yeah, I initially tried that actually. They have very low
| resolution data and do not provide raw trades/quotes
| (something like a message every 250ms), you cannot watch the
| entire market so you're stuck with like 100 tickers [1] vs
| 5500 tickers, and fetching historical data is very
| cumbersome. You can basically pull down pre-aggregated
| candlestick data and I wanted to base things off the raw
| trades/quotes. I actually probably spend a month or two
| trying to make this work. So, that's why I'm using polygon.io
| in that you can stream trades/quotes for the entire stock
| market in real-time, without any caps, which is amazing!
|
| [1] https://www.interactivebrokers.com/en/?f=%2Fen%2Fgeneral%
| 2Fe...
| czbond wrote:
| @WestCoastJustin
|
| I've been really wanting to use Go, but as you say, much of the
| community is Python due to the data analysis strengths. To the
| detriment of the other things Python does do poorly.
|
| Can you give some thoughts with your experimentation on the
| following from a Go perspective.
|
| 1. Supported TA libraries in Go. I'm familiar with TAlib
| (python), bloom, etc. - certain forks tailored to real time
| rather than historical (eg: no re-compute on ticks)
|
| 2. Data storage (article mentioned you're all in memory). I've
| been using S3 & ArticDB
|
| 3. If your in-data memory is treating you well for multiple TA
| calculations (example: in Python, you can compute & save
| pickled dataframes - and re-read those over longer time
| periods)
| eshnil wrote:
| Why not switch to Mojo lang for this? It's Python-compatible
| with Golang like performance from what I hear.
| czbond wrote:
| @eshnil (& others familiar with Mojo)
|
| I had not looked at Mojo... thanks for the pointer. In the
| past, I've had issues with library compatibility with
| compiled derivative languages of interpreted ones (eg:
| Crystal of Ruby, etc, etc).
|
| Know if Mojo directly uses existing Python ecosystem?
|
| I've been using Polars, etc.
| silisili wrote:
| I'm excited for Mojo, but it'll be years before it's ready
| for prime time as a general purpose language. It's not even
| available to download yet AFAIK...
| jjtheblunt wrote:
| It's not available yet outside of hosted environments
| provided by Modular.
| WestCoastJustin wrote:
| Haven't looked. Honestly, I'm so invested in this now it
| would be a pain to change anything.
| pdimitar wrote:
| Not a complete answer but I quite liked
| https://github.com/markcheno/go-talib for technical
| indicators.
| WestCoastJustin wrote:
| > Supported TA libraries in Go. I'm familiar with TAlib
| (python), bloom, etc. - certain forks tailored to real time
| rather than historical (eg: no re-compute on ticks)
|
| I've been basically, just manually coding the algorithms from
| python into Go. ChatGPT is amazing at this. I really only
| just about 4 so it was a one time thing.
|
| > Data storage (article mentioned you're all in memory). I've
| been using S3 & ArticDB
|
| Yeah, I ran into issues and then was like what would be the
| fastest, then just went in-memory. I download all raw
| trades/quotes each night and store then into gob+lz4
| compressed files. Then for backtesting and stuff I can load
| these in and build the aggrogate bars on the fly.
|
| > If your in-data memory is treating you well for multiple TA
| calculations (example: in Python, you can compute & save
| pickled dataframes - and re-read those over longer time
| periods)
|
| Yeah, I have a historical lookup table that I build nightly
| too. This gives me a reference point when I'm doing % change
| calculations and stuff. I should probably have mentioned
| that.
___________________________________________________________________
(page generated 2023-06-30 23:00 UTC)