Skip to content

Multiple pulsar clusters use the same bookkeeper cluster ? #5935

Closed
@SunDapeng1

Description

@SunDapeng1

Can multiple pulsar clusters use the same bookkeeper cluster? If I had deployed the bk separately in advance and the metadata was stored under /ledgers, how would I configure the pulsar so that every pulsar recognizes /ledgers ?
(According to my test: pulsar initializes metadata to /pulsar, then the pulasr cluster can only recognize /pulsar/ledger to store metadata of bk)

Activity

added
type/featureThe PR added a new feature or issue requested a new feature
on Dec 25, 2019
jiazhai

jiazhai commented on Dec 26, 2019

@jiazhai
Member

Thanks @SunDapeng1 for open this issue. currently class PulsarClusterMetadataSetup, which is called by command bin/pulsar initialize-cluster-metadata, will do all the metadata setup for Pulsar.
As discussed here with @murong00 ,
#5879 (comment)
we should make this configuration more easy.

But, Seems in current code, of PulsarClusterMetadataSetup, it is using /ledgers for bookies by default? In your deployment, are you using the localzk as: zkhost:2181/pulsar ? maybe just change it back to zkhost:2181 would solve this issue?

SunDapeng1

SunDapeng1 commented on Dec 26, 2019

@SunDapeng1
Author

Thank you for your reply. Maybe I didn't describe it clearly. What I want to express is that if the bk metadata is in /ledgers, then when the first pulsar is deployed, the metadata can be initialized to /. At this time, the first pulsar cluster can normally identify /ledgers Bk cluster, but how can I deploy a second pulsar cluster, so that the second pulsar cluster also uses the same bk cluster /ledgers?

jiazhai

jiazhai commented on Dec 26, 2019

@jiazhai
Member

@SunDapeng1 , as we discussed, the main issue is make bookie service (/ledgers) configurable. This may involves some change like PulsarClusterMetadataSetup InternalConfigurationData.
at present, ZkLedgersRootPath that passed into InternalConfigurationData is some kind of hard coded.

SunDapeng1

SunDapeng1 commented on Feb 10, 2020

@SunDapeng1
Author

@jiazhai How is this issue going?

added a commit that references this issue on Apr 21, 2020
188d896
added a commit that references this issue on Aug 24, 2020
b6dcfc5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/featureThe PR added a new feature or issue requested a new feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @sijie@jiazhai@SunDapeng1

      Issue actions

        Multiple pulsar clusters use the same bookkeeper cluster ? · Issue #5935 · apache/pulsar