Let’s say we have a topic where we publish updates for invoices. All the messages are cumulative and need to be received in order by the consumer.
If a particular message fails (after retries, and exponential backoff), can PubSub ONLY stop the messages for that particular OrderingKey (the InvoiceID) and allow all the other messages for other Invoices to keep on trucking?
The goal here is that if a particular Invoice cannot be processed, all the other Invoices should NOT be affected. Regardless of the partition they are part of.
Does the behavior change if we are using Push or Pull?
By looking at this old document, it appears that all messages for that partition (not only the Ordering Key) will get stuck. Which seems to be a problem for anyone using PubSub that needs ordering.
Only within a single partition in Pub/Sub, which is a subset of the topic’s message stream, is message ordering guaranteed. If your use case depends on ordering, you should divide your topic using a key that embodies the ordering requirement. It appears that you are dividing the subject by the InvoiceID, which is a wise strategy. In response to your query, Pub/Sub will only stop delivering messages for that particular message ordering group if a given message fails after multiple tries and exponential backoff, rather than stopping all messages for that OrderingKey (InvoiceID). This means that the unsuccessful message won’t have an impact on the way that messages for other OrderingKeys are delivered. Whether you use Push or Pull, Pub/Sub behaves in the same way. But if you use Pull, you have more control over the message flow and can decide to halt pulling messages for a particular subscription or OrderingKey if necessary.
Regarding the document you cited, it’s critical to go to the most recent documentation because it’s likely that Pub/Sub’s behavior has changed since that article was created.