[FLINK-29922] Support create external table for hive catalog #357
+96
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JingsongLi
requested changes
Nov 8, 2022
SteNicholas
reviewed
Nov 8, 2022
| @@ -49,6 +49,12 @@ private CatalogOptions() {} | |||
| .noDefaultValue() | |||
| .withDescription("Uri of metastore server."); | |||
|
|
|||
| public static final ConfigOption<Boolean> HMS_EXTERNAL = | |||
| ConfigOptions.key("hms.external") | |||
There was a problem hiding this comment.
@wxplovecc，could this introduce the TableType enum and table.type config option which default value is MANAGED_TABLE? In the CatalogOptions, the hms.external is specific for core module.
cc @JingsongLi
There was a problem hiding this comment.
@SteNicholas Thanks for review, updated
SteNicholas
reviewed
Nov 11, 2022
| ConfigOptions.key("table.type") | ||
| .enumType(TableType.class) | ||
| .defaultValue(TableType.MANAGED_TABLE) | ||
| .withDescription("Table type of hms table, MANAGED_TABLE/EXTERNAL_TABLE."); |
There was a problem hiding this comment.
Suggested change
| .withDescription("Table type of hms table, MANAGED_TABLE/EXTERNAL_TABLE."); | |
| .withDescription("Type of table."); |
SteNicholas
reviewed
Nov 11, 2022
|
|
||
| package org.apache.flink.table.store.table; | ||
|
|
||
| /** TableType enum of catalog table. */ |
There was a problem hiding this comment.
Suggested change
| /** TableType enum of catalog table. */ | |
| /** Enum of catalog table type. */ |
JingsongLi
reviewed
Nov 11, 2022
|
|
||
| /** Enum of catalog table type. */ | ||
| public enum TableType { | ||
| MANAGED_TABLE, |
There was a problem hiding this comment.
MANAGED and EXTERNAL is OK.
Can you let it implements DescribedEnum just like ChangelogProducer?
SteNicholas
reviewed
Nov 12, 2022
| @@ -226,6 +227,13 @@ public void createTable(ObjectPath tablePath, UpdateSchema updateSchema, boolean | |||
| e); | |||
| } | |||
| Table table = newHmsTable(tablePath); | |||
|
|
|||
| if (hiveConf.getEnum(TABLE_TYPE.key(), TableType.MANAGED_TABLE) | |||
There was a problem hiding this comment.
It's better to move this logic to newHmsTable which initialzes the Table and sets the parameters of the Table.
SteNicholas
reviewed
Nov 14, 2022
|
|
||
| /** Enum of catalog table type. */ | ||
| public enum TableType implements DescribedEnum { | ||
| MANAGED("MANAGED_TABLE", "Hive manage the lifecycle of the table."), |
There was a problem hiding this comment.
Suggested change
| MANAGED("MANAGED_TABLE", "Hive manage the lifecycle of the table."), | |
| MANAGED("MANAGED_TABLE", "Table Store owned table where the entire lifecycle of the table data is managed."), |
SteNicholas
reviewed
Nov 14, 2022
| /** Enum of catalog table type. */ | ||
| public enum TableType implements DescribedEnum { | ||
| MANAGED("MANAGED_TABLE", "Hive manage the lifecycle of the table."), | ||
| EXTERNAL("EXTERNAL_TABLE", "Files are already present or in remote locations."); |
There was a problem hiding this comment.
Suggested change
| EXTERNAL("EXTERNAL_TABLE", "Files are already present or in remote locations."); | |
| EXTERNAL("EXTERNAL_TABLE", "The table where Table Store has loose coupling with the data stored in external locations."); |
There was a problem hiding this comment.
updated @SteNicholas
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
support hive catalog to create external table