Skip to content

[Transaction] Handle Acknowledge In The Transaction #7856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Sep 4, 2020

Conversation

gaoran10
Copy link
Contributor

Fix https://github.com/streamnative/pulsar/issues/1307

Master Issue: #2664

Motivation

Handle acknowledge in the transaction.

Modifications

  1. Register subscription to the transaction metadata store.
  2. Make the ack command carry the transaction information.
  3. When committing a transaction, commit every subscription in the transaction.

Verifying this change

This change added tests and can be verified as follows:

  • org.apache.pulsar.broker.transaction.TransactionProduceTest.ackCommitTest

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (yes)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

@gaoran10
Copy link
Contributor Author

/pulsarbot run-failure-checks

@sijie sijie requested review from codelipenghui and jiazhai August 24, 2020 22:13
@sijie sijie added this to the 2.7.0 milestone Aug 24, 2020
@sijie sijie added area/transaction type/feature The PR added a new feature or issue requested a new feature labels Aug 24, 2020
@gaoran10
Copy link
Contributor Author

gaoran10 commented Sep 1, 2020

/pulsarbot run-failure-checks

public void addSubscriptionToTxn(TxnID txnID, String topic, String subscription)
throws TransactionCoordinatorClientException {
try {
addSubscriptionToTxnAsync(txnID, topic, subscription);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method will return before processing the add subscription to the txn, is this an expected behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I miss calling the get().

.setTopic(topic)
.setSubscription(subscription)
.build();
return handler.addSubscriptionToTxn(txnID, Lists.newArrayList(sub));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Singleton list is more suitable here. Or maybe you add a thread-local list to avoid introduce many list instance

Comment on lines 222 to 224
} else {
// Unsupported txnStatus
commitFuture.completeExceptionally(new Throwable("Unsupported txnStatus."));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this branch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll revert this.

Comment on lines 1210 to 1211
// dispatcher.redeliverUnacknowledgedMessages(consumer, (List<PositionImpl>)
// (List<?>)pendingAckMessageForCurrentTxn.values());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a useful change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I'll revert this.

*/
@Data
@Builder
public class TransactionSubscription implements Comparable<TransactionSubscription> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is the same struct in PulsarApi.proto named Subscription, maybe you can use that one directly.

Copy link
Contributor Author

@gaoran10 gaoran10 Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a problem, I can't override the hashCode method of the Subscription and it can't be used in a Set collection.

@codelipenghui
Copy link
Contributor

/pulsarbot run-failure-checks

@codelipenghui codelipenghui merged commit 1fdffe7 into apache:master Sep 4, 2020
@gaoran10 gaoran10 deleted the handle-ack-in-txn branch September 4, 2020 08:34
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
Fix https://github.com/streamnative/pulsar/issues/1307

Master Issue: apache#2664 

### Motivation

Handle acknowledge in the transaction.

### Modifications

1. Register subscription to the transaction metadata store.
2. Make the ack command carry the transaction information.
3. When committing a transaction, commit every subscription in the transaction.
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
Fix https://github.com/streamnative/pulsar/issues/1307

Master Issue: apache#2664 

### Motivation

Handle acknowledge in the transaction.

### Modifications

1. Register subscription to the transaction metadata store.
2. Make the ack command carry the transaction information.
3. When committing a transaction, commit every subscription in the transaction.
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
Fix https://github.com/streamnative/pulsar/issues/1307

Master Issue: apache#2664 

### Motivation

Handle acknowledge in the transaction.

### Modifications

1. Register subscription to the transaction metadata store.
2. Make the ack command carry the transaction information.
3. When committing a transaction, commit every subscription in the transaction.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Transaction] Commit on the subscription.
3 participants