Cluster Metadata #
Persistent Metadata Repository #
Background Information #
DBPlusEngine provides different ways to persist metadata, and users can choose the appropriate way to store metadata.
Parameter Explanation #
ZooKeeper persistence
Type: ZooKeeper
Configurable properties：
| Name | Data Type | Description | Default Value |
|---|---|---|---|
| retryIntervalMilliseconds | int | Number of milliseconds between retries | 500 |
| maxRetries | int | Maximum number of retries for client connections | 3 |
| timeToLiveSeconds | int | Number of seconds of temporary data expiration | 60 |
| operationTimeoutMilliseconds | int | Number of milliseconds for client operation timeout | 500 |
| digest | String | Login authentication password |
Etcd persistence
Type: Etcd
Configurable properties:
| Name | Data Type | Description | Default Value |
|---|---|---|---|
| timeToLiveSeconds | long | Number of seconds of temporary data expiration | 30 |
| connectionTimeout | long | Connection timeout seconds | 30 |
SphereEx-DBPlusEngine-Mate persistence
Type: SphereEx:MATE
Configurable Properties:
Currently none.
Operation steps #
Configure the metadata persistence repository type.
Configuration Example #
mode:
type: Cluster
repository:
type: SphereEx:MATE
props:
namespace: governance
server-lists: localhost:21506
Clustered metadata storage (ZooKeeper) #
Under the defined namespace, rules , props and metadata store configuration in YAML format, which can be modified to dynamically manage configuration.
Nodes store database access object runtime nodes, which are used to distinguish between different database access instances.
namespace
├──rules # Global rule configuration
├──props # Property Configuration
├──metadata # Metadata Configuration
├ ├──${databaseName} # Logical database name
├ ├ ├──schemas # Schema List
├ ├ ├ ├──${schemaName} # Logic Schema Name
├ ├ ├ ├ ├──tables # Table Structure Configuration
├ ├ ├ ├ ├ ├──${tableName}
├ ├ ├ ├ ├ ├──...
├ ├ ├ ├ ├──views # View Structure Configuration
├ ├ ├ ├ ├ ├──${viewName}
├ ├ ├ ├ ├ ├──...
├ ├ ├ ├──...
├ ├ ├──versions # Metadata Version List
├ ├ ├ ├──${versionNumber} # Metadata version number
├ ├ ├ ├ ├──data_sources # Data source configuration
├ ├ ├ ├ ├──rules # Rule Configuration
├ ├ ├ ├──...
├ ├ ├──active_version # Activated metadata version number
├ ├──...
├──nodes
├ ├──compute_nodes
├ ├ ├──online
├ ├ ├ ├──proxy
├ ├ ├ ├ ├──UUID # Proxy Instance Unique Identification
├ ├ ├ ├ ├──....
├ ├ ├ ├──jdbc
├ ├ ├ ├ ├──UUID # JDBC Instance Unique Identification
├ ├ ├ ├ ├──....
├ ├ ├──status
├ ├ ├ ├──UUID
├ ├ ├ ├──....
├ ├ ├──worker_id
├ ├ ├ ├──UUID
├ ├ ├ ├──....
├ ├ ├──process_trigger
├ ├ ├ ├──process_list_id:UUID
├ ├ ├ ├──....
├ ├ ├──labels
├ ├ ├ ├──UUID
├ ├ ├ ├──....
├ ├ ├──xa_recovery_id
├ ├ ├ ├──UUID
├ ├ ├ ├ ├──├──UUID
├ ├──storage_nodes
├ ├ ├──${databaseName.groupName.ds}
├ ├ ├──${databaseName.groupName.ds}
├──sys_data
├ ├──shardingsphere
├ ├ ├──schemas
├ ├ ├ ├──shardingsphere
├ ├ ├ ├ ├──tables # System Table
├ ├ ├ ├ ├ ├──sharding_table_statistics # Sharding statistics table data
├ ├ ├ ├ ├ ├ ├──8a2dcb0d97c3d86ef77b3d4651a1d7d0 # md5
├ ├ ├ ├ ├ ├──cluster_information # Cluster Information Sheet
/rules
Global rule configuration, which can include access to DBPlusEngine-Proxy username and password permissions configuration.
Global rule configuration, which can include transaction configuration, SQL parsing configuration, etc.
- !TRANSACTION
defaultType: XA
providerType: Atomikos
- !SQL_PARSER
sqlCommentParseEnabled: true
/props
Property Configuration
kernel-executor-size: 20
sql-show: true
/metadata/${schemaName}/dataSources
A collection of multiple database connection pools with self-adapting properties for different database connection pools(e.g. DBCP，C3P0，Druid, HikariCP).
ds_0:
initializationFailTimeout: 1
validationTimeout: 5000
maxLifetime: 1800000
leakDetectionThreshold: 0
minimumIdle: 1
password: root
idleTimeout: 60000
jdbcUrl: jdbc:mysql://127.0.0.1:3306/ds_0?serverTimezone=UTC&useSSL=false
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
maximumPoolSize: 50
connectionTimeout: 30000
username: root
poolName: HikariPool-1
ds_1:
initializationFailTimeout: 1
validationTimeout: 5000
maxLifetime: 1800000
leakDetectionThreshold: 0
minimumIdle: 1
password: root
idleTimeout: 60000
jdbcUrl: jdbc:mysql://127.0.0.1:3306/ds_1?serverTimezone=UTC&useSSL=false
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
maximumPoolSize: 50
connectionTimeout: 30000
username: root
poolName: HikariPool-2
/metadata/${schemaName}/rules
Rule configuration can include data sharding, read/write separation, data encryption, shadow library pressure testing and other configurations.
- !SHARDING
xxx
- !READWRITE_SPLITTING
xxx
- !ENCRYPT
xxx
/metadata/${schemaName}/tables
Table structure configuration, in which each table uses a separate node storage, does not support dynamic modification for the time being.
name: t_order # Table name
columns: # Column
id: # Column name
caseSensitive: false
dataType: 0
generated: false
name: id
primaryKey: trues
order_id:
caseSensitive: false
dataType: 0
generated: false
name: order_id
primaryKey: false
indexs: # Index
t_user_order_id_index: # Index name
name: t_user_order_id_index
/nodes/compute_nodes
The database access object running instance information, and the child node is the currently running instance’s identifier. The running instance identifier is composed of the IP address and PORT of the running server.
Running instance identification are temporary nodes, when the instance is registered online, automatically cleaned up when offline. The registration center monitors the changes of these nodes to govern the running instance’s access to the database, etc.
/nodes/storage_nodes
Read/write slave libraries can be managed. It can be dynamically added and removed, and disabled.