[pulsar-admin] Support delete schema ledgers when delete topics #8167
+179
−38
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jiazhai
approved these changes
Sep 30, 2020
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
3 tasks
jiazhai
added
area/admin
doc-required
BewareMyPower
added a commit
to streamnative/pulsar-archived
that referenced
this pull request
Oct 1, 2020
…he#8167) Master Issue: apache#8134 ### Motivation The REST APIs of `PersistentTopics#deleteTopic` and `PersistentTopics#deletePartitionedTopic` didn't support delete schema of topics, which may cause after topics were deleted, the schema ledgers still existed. And current implementation of delete schema is just add a empty schema ledger but not delete existed schema ledgers. ### Modifications - Add a `deleteSchema` query param to REST APIs of deleting topics/partitioned topics; - Add a map to record the created ledgers in `BookkeeperSchemaStorage`; - Expose `deleteSchema` param in pulsar-admin APIs; - Delete schema ledgers when delete the cluster with `-a` option. ### Verifying this change It has been tested in local environments: 1. Check the current count of ledgers; 2. Create a new partitioned topic or non-partitioned topic; 3. Create producers/consumers with schema to send and receive messages with the created topic; 4. Delete the topic with `deleteSchema=true` query param or call admin API that set `deleteSchema` to false; 5. Recheck the count of ledgers, make sure it's the same with step 1.
BewareMyPower
added a commit
to streamnative/pulsar-archived
that referenced
this pull request
Oct 1, 2020
…he#8167) Master Issue: apache#8134 ### Motivation The REST APIs of `PersistentTopics#deleteTopic` and `PersistentTopics#deletePartitionedTopic` didn't support delete schema of topics, which may cause after topics were deleted, the schema ledgers still existed. And current implementation of delete schema is just add a empty schema ledger but not delete existed schema ledgers. ### Modifications - Add a `deleteSchema` query param to REST APIs of deleting topics/partitioned topics; - Add a map to record the created ledgers in `BookkeeperSchemaStorage`; - Expose `deleteSchema` param in pulsar-admin APIs; - Delete schema ledgers when delete the cluster with `-a` option. ### Verifying this change It has been tested in local environments: 1. Check the current count of ledgers; 2. Create a new partitioned topic or non-partitioned topic; 3. Create producers/consumers with schema to send and receive messages with the created topic; 4. Delete the topic with `deleteSchema=true` query param or call admin API that set `deleteSchema` to false; 5. Recheck the count of ledgers, make sure it's the same with step 1.
lbenc135
pushed a commit
to lbenc135/pulsar
that referenced
this pull request
Oct 3, 2020
…he#8167) Master Issue: apache#8134 ### Motivation The REST APIs of `PersistentTopics#deleteTopic` and `PersistentTopics#deletePartitionedTopic` didn't support delete schema of topics, which may cause after topics were deleted, the schema ledgers still existed. And current implementation of delete schema is just add a empty schema ledger but not delete existed schema ledgers. ### Modifications - Add a `deleteSchema` query param to REST APIs of deleting topics/partitioned topics; - Add a map to record the created ledgers in `BookkeeperSchemaStorage`; - Expose `deleteSchema` param in pulsar-admin APIs; - Delete schema ledgers when delete the cluster with `-a` option. ### Verifying this change It has been tested in local environments: 1. Check the current count of ledgers; 2. Create a new partitioned topic or non-partitioned topic; 3. Create producers/consumers with schema to send and receive messages with the created topic; 4. Delete the topic with `deleteSchema=true` query param or call admin API that set `deleteSchema` to false; 5. Recheck the count of ledgers, make sure it's the same with step 1.
|
need to cherry-pick to 2.6.2. |
wolfstudy
pushed a commit
that referenced
this pull request
Oct 30, 2020
Master Issue: #8134 ### Motivation The REST APIs of `PersistentTopics#deleteTopic` and `PersistentTopics#deletePartitionedTopic` didn't support delete schema of topics, which may cause after topics were deleted, the schema ledgers still existed. And current implementation of delete schema is just add a empty schema ledger but not delete existed schema ledgers. ### Modifications - Add a `deleteSchema` query param to REST APIs of deleting topics/partitioned topics; - Add a map to record the created ledgers in `BookkeeperSchemaStorage`; - Expose `deleteSchema` param in pulsar-admin APIs; - Delete schema ledgers when delete the cluster with `-a` option. ### Verifying this change It has been tested in local environments: 1. Check the current count of ledgers; 2. Create a new partitioned topic or non-partitioned topic; 3. Create producers/consumers with schema to send and receive messages with the created topic; 4. Delete the topic with `deleteSchema=true` query param or call admin API that set `deleteSchema` to false; 5. Recheck the count of ledgers, make sure it's the same with step 1. (cherry picked from commit 983266d)
huangdx0726
pushed a commit
to huangdx0726/pulsar
that referenced
this pull request
Nov 13, 2020
…he#8167) Master Issue: apache#8134 ### Motivation The REST APIs of `PersistentTopics#deleteTopic` and `PersistentTopics#deletePartitionedTopic` didn't support delete schema of topics, which may cause after topics were deleted, the schema ledgers still existed. And current implementation of delete schema is just add a empty schema ledger but not delete existed schema ledgers. ### Modifications - Add a `deleteSchema` query param to REST APIs of deleting topics/partitioned topics; - Add a map to record the created ledgers in `BookkeeperSchemaStorage`; - Expose `deleteSchema` param in pulsar-admin APIs; - Delete schema ledgers when delete the cluster with `-a` option. ### Verifying this change It has been tested in local environments: 1. Check the current count of ledgers; 2. Create a new partitioned topic or non-partitioned topic; 3. Create producers/consumers with schema to send and receive messages with the created topic; 4. Delete the topic with `deleteSchema=true` query param or call admin API that set `deleteSchema` to false; 5. Recheck the count of ledgers, make sure it's the same with step 1.
Anonymitaet
removed
the
doc-required
Anonymitaet
added
the
doc
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Master Issue: #8134
Motivation
The REST APIs of
PersistentTopics#deleteTopicandPersistentTopics#deletePartitionedTopicdidn't support delete schema of topics, which may cause after topics were deleted, the schema ledgers still existed. And current implementation of delete schema is just add a empty schema ledger but not delete existed schema ledgers.Modifications
deleteSchemaquery param to REST APIs of deleting topics/partitioned topics;BookkeeperSchemaStorage;deleteSchemaparam in pulsar-admin APIs;-aoption.Verifying this change
It has been tested in local environments:
deleteSchema=truequery param or call admin API that setdeleteSchemato false;Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation