Add deleted schema judgment when adding schema #4731
Merged
+102
−42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
congbobo184
changed the title
codelipenghui
requested changes
Jul 15, 2019
| ) { | ||
|
|
||
| if (index.isEmpty()) { | ||
| return completedFuture(null); | ||
| } | ||
|
|
||
| for (SchemaStorageFormat.IndexEntry entry : index) { | ||
| if (Arrays.equals(entry.getHash().toByteArray(), hash)) { | ||
| if (Arrays.equals(entry.getHash().toByteArray(), hash) && entry.getVersion() > maxDeletedVersion) { |
There was a problem hiding this comment.
Suggested change
| if (Arrays.equals(entry.getHash().toByteArray(), hash) && entry.getVersion() > maxDeletedVersion) { | |
| if (Arrays.equals(entry.getHash().toByteArray(), hash)) { | |
| if (entry.getVersion() > maxDeletedVersion) { | |
| return completedFuture(entry.getVersion()); | |
| } else { | |
| return completedFuture(null); | |
| } | |
| } |
| @@ -129,7 +145,12 @@ | |||
| .setTimestamp(clock.millis()) | |||
| .addAllProps(toPairs(schema.getProps())) | |||
| .build(); | |||
| return schemaStorage.put(schemaId, info.toByteArray(), context); | |||
| try { | |||
There was a problem hiding this comment.
Suggested change
| try { | |
| return getMaxDeletedVersion(schemaId).thenCompose(deleteVersion -> { | |
| return schemaStorage.put(schemaId, info.toByteArray(), context, deleteVersion); | |
| }) |
sijie
requested changes
Jul 16, 2019
| @@ -312,7 +312,7 @@ public void close() throws Exception { | |||
| // There was a race condition on the schema creation. Since it has now been created, | |||
| // retry the whole operation so that we have a chance to recover without bubbling error | |||
| // back to producer/consumer | |||
| putSchemaIfAbsent(schemaId, data, hash) | |||
| putSchemaIfAbsent(schemaId, data, hash, -1) | |||
There was a problem hiding this comment.
define a constant for -1 and use the constant here.
| @@ -129,7 +145,10 @@ | |||
| .setTimestamp(clock.millis()) | |||
| .addAllProps(toPairs(schema.getProps())) | |||
| .build(); | |||
| return schemaStorage.put(schemaId, info.toByteArray(), context); | |||
|
|
|||
| return getMaxDeletedVersion(schemaId).thenCompose(deleteVersion -> | |||
There was a problem hiding this comment.
I think we should return the maxDeletedVersion when we are doing compatibility check above. This would reduce the number of times we are reading schema info. So the logic would be something like this:
getSchema(schemaId).thenCompose(existingSchema -> {
if (existingSchema == null) {
return completedFuture(-1L, true)
} else if (existingSchema.schema.isDeleted()) {
return completedFuture(existingSchema.schema.version, true);
} else {
if (isTransitiveStrategy) {
return ...;
} else {
// find the maxDeletedVersion in backward sequence.
return ...;
}
}
}).thenCompose((maxDeletedVersion, isCompatible) -> {
})
sijie
added
component/schemaregistry
type/bug
|
retest this please |
sijie
approved these changes
Jul 18, 2019
codelipenghui
approved these changes
Jul 19, 2019
| @@ -139,7 +140,7 @@ public void start() throws IOException { | |||
| .thenApply(entry -> new StoredSchema | |||
| ( | |||
| entry.getSchemaData().toByteArray(), | |||
| new LongSchemaVersion(schemaLocator.getInfo().getVersion()) | |||
| new LongSchemaVersion(indexEntry.getVersion()) | |||
There was a problem hiding this comment.
Please help create an issue that this change fixed.
|
retest this please |
|
run integration tests |
|
This PR fixed #4763 |
|
run Integration Tests |
|
retest this please |
…congbobo184/pulsar into add_schema_if_delete_the_schema
|
run java8 tests |
2 similar comments
|
run java8 tests |
|
run java8 tests |
easyfan
pushed a commit
to easyfan/pulsar
that referenced
this pull request
Jul 26, 2019
### Motivation to fix apache#4724 ### Verifying this change Add the tests for it
jiazhai
pushed a commit
that referenced
this pull request
Aug 28, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
to fix #4724
Verifying this change
Add the tests for it
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes
Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (yes)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)
Documentation
Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation