[HN Gopher] Types of events in event-driven systems
___________________________________________________________________
Types of events in event-driven systems
Author : frankdejonge
Score : 90 points
Date : 2022-02-19 13:07 UTC (9 hours ago)
(HTM) web link (blog.frankdejonge.nl)
(TXT) w3m dump (blog.frankdejonge.nl)
| adamkl wrote:
| I like the idea of using "fat" events as a means of data
| integration between systems.
|
| Calling them RESTful events is a good term as I've pitched in
| much the same fashion. Every time a system responds to a
| create/update/execute request, drop an equivalent message into an
| event stream. Initially these can form the basis of a streaming
| ETL to whatever systems you use for analytics, but as new
| internal systems are brought online (e.g a new CRM system to help
| sales manage all the leads signing up for your successful
| product) the integration is already there, waiting. Just add a
| new consumer to the existing stream.
|
| Overtime, you end up with a "data mesh". If you look past the
| marketing of the term, it's basically just means each application
| in your company should publish both sync (for real-time usage)
| and async (for data sharing between systems of record)
| interfaces.
| hashimotonomora wrote:
| Couldn't that bring inconsistencies eventually? If one commits
| and the other doesn't.
| themoop wrote:
| This is where you might want to use the outbox pattern. If
| you depend on dual writes you will definitly have consistency
| issues at some point
| Gwypaas wrote:
| The question at hand will always reduce down to the "Two
| Generals problem" [0]. The outbox pattern is a nice
| separation of concerns and gives at least once consistency
| as long as the forwarding happen before writing the event
| as processed in the outbox. Both side effects happening
| atomically through all failure cases is impossible.
|
| To solve that issue you need a cooperating destination for
| your writes which handles de-duplication. Then you get into
| the weeds of causality, ordering and idempotence. Ugh.
|
| For some real world examples see the AWS Kinesis
| documentation which says that any application using Kinesis
| must be able to handle duplicate records [1].
|
| > There are two primary reasons why records may be
| delivered more than one time to your Amazon Kinesis Data
| Streams application: producer retries and consumer retries.
| Your application must anticipate and appropriately handle
| processing individual records multiple times.
|
| [0]: https://en.wikipedia.org/wiki/Two_Generals%27_Problem
|
| [1]:
| https://docs.aws.amazon.com/streams/latest/dev/kinesis-
| recor...
| adamkl wrote:
| It is possibly that you could get inconsistencies between
| systems, but you mitigate that with durable event streams
| (e.g. Kafka) and by ensuring that each data entity in your
| enterprise has a system of record that can be used to resolve
| conflicts.
|
| This is really not that different than using ETL jobs to move
| data about, but taking a streaming approach vs a batch
| approach.
| majormajor wrote:
| > It is possibly that you could get inconsistencies between
| systems, but you mitigate that with durable event streams
| (e.g. Kafka) and by ensuring that each data entity in your
| enterprise has a system of record that can be used to
| resolve conflicts.
|
| The event stream layer isn't where the sync problems have
| arised in the systems I've worked on. It's the "commit
| transactionally both to your database and the event stream"
| part. Not a lot of systems are built to be ready to roll
| back the database change if the event publish fails. Or to
| be able to handle duplicate events if you error the other
| way.
| northstar702 wrote:
| Would it make sense to always write to a transactional
| queue first and then write to the database by reading
| sequentially from the queue?
| morelisp wrote:
| If something is relying on the data in the queue to
| trigger a request to a service using the DB, the
| requestee may not have the full data yet.
|
| The pattern I usually see is wiring this the other way -
| use an initial event as the trigger to write to the DB,
| while also streaming the DB's binlog or equivalent back
| as events. Now the risk is that another service gets a
| "too fresh" view but this is usually less harmful than a
| stale view. Things listening to the binlog events need to
| process them idempotently, but this is usually not a
| major complication since most queue designs will require
| you to be prepared for that anyway.
| northstar702 wrote:
| So always write to a transactional queue first and then
| write to the database by reading sequentially from the
| queue?
| Gwypaas wrote:
| That would work. The consistency will be at least once
| and you do de-duplication to handle fault tolerance
| through the database then.
|
| Just need to make sure to not mess up causal ordering
| between events because of out of order retries, if such
| things are important for your application.
| adamkl wrote:
| I guess if the concern is around atomically committing to
| the DB and event stream at the same time, you could hang
| the event stream off the DB using change-data-capture to
| populate the events.
|
| Ultimately, whenever you are pushing data between systems
| you can end up with inconsistencies which why it's
| important to clearly define systems of record.
| morelisp wrote:
| I like this ontology; we've divided our events into roughly
| similar categories but these are better names than we use.
|
| I'd further advance that
|
| - Trigger events and RESTful events are roughly the same thing,
| just a question of what you choose in latency vs. size vs. schema
| flexibility space. We even have events in our system whose schema
| inlines data below a certain size but links above that.
|
| - There is a fourth type: windowed reductions over domain events,
| e.g. the "changelog" in Kafka Streams. This bears a similar
| relation to domain events as the RESTful event does to trigger
| events.
| hashimotonomora wrote:
| Could anyone recommend more resources on this for studying?
| TobbenTM wrote:
| Martin Fowler talks about the same kind of events, using
| terminology I've heard much more often:
| https://martinfowler.com/articles/201701-event-driven.html
___________________________________________________________________
(page generated 2022-02-19 23:01 UTC)