Logo
Technical Architecture

Technical Architecture #

Overall Architecture #

The pluggable architecture of DBPlusEngine is divided into three layers: L1 kernel layer, L2 feature layer and L3 ecosystem layer.

The overall architecture of DBPlusEngine is shown in the following figure.

L1 Kernel Layer #

An abstraction of basic capabilities of database. All components are required and the specific implementation can be replaced by pluggable way. It includes query optimizer, distributed transaction engine, distributed execution engine, authority engine and scheduling engine.

L2 Feature Layer #

Used to provide enhanced capability. All components are optional and can contain zero or multiple components. Components isolate each other and multiple components can be used together superimposed. It includes data sharding, readwrite-splitting, database highly availability, data encryption, shadow database and so on. The user-defined feature can be fully customized and extended for the top-level interface defined by Apache ShardingSphere without changing kernel codes.

L3 Ecosystem Layer #

Used to integrate into the current database ecosystem. It includes database protocol, SQL parser and storage adapter.

The access modes of DBPlusEngine include driver and proxy, namely DBPlusEngine-Driver and DBPlusEngine-Proxy.

DBPlusEngine-Driver #

The lightweight Java framework provides additional services in the JDBC layer of Java. It uses the client to connect directly to the database and provides services in the form of jar packages without additional deployment and dependency. It can be understood as an enhanced JDBC driver and is fully compatible with JDBC and various ORM frameworks.

  • It is applicable to any JDBC based ORM framework, such as JPA, Hibernate, Mybatis, Spring JDBC template or direct use of JDBC;
  • Support any third-party database connection pool, such as DBCP, C3P0, BoneCP, HikariCP, etc;
  • It supports any database that implements the JDBC specification. Currently, it supports MySQL, PostgreSQL, Oracle, SQLServer and any database that can be accessed using JDBC.

DBPlusEngine-Proxy #

The transparent database agent provides a server version that encapsulates the database binary protocol to support heterogeneous languages. At present, MySQL and PostgreSQL (compatible with openGauss and other PostgreSQL based databases) are available. It can use any access client compatible with MySQL/PostgreSQL protocol (such as MySQL Command Client, MySQL Workbench, Navicat and DBeaver) to operate data, which is more friendly to DBAs.

  • It is completely transparent to applications and can be directly used as MySQL/PostgreSQL;
  • Applicable to any client compatible with MySQL/PostgreSQL protocol.

Security System #

DBPlusEngine-Proxy (hereinafter referred to as Proxy) provides a complete security system, taking into account engine security and data security.

Login Authentication #

Proxy has a strict login authentication mechanism. Only authenticated users can successfully establish a connection.

Password Authentication

By default, Proxy uses password authentication. The login user must provide the correct user name and password.

In particular, because Proxy supports a variety of database protocols (such as MySQL, PostgreSQL, etc.), when users apply different database clients, Proxy can automatically adapt the password communication protocol to provide users with a consistent security experience in complex scenarios.

Host Restrictions

The administrator can restrict the login host address for Proxy users to improve the security level. For example:

authority:
  users:
    - user: root@127.0.0.1
      password: root

The above configuration specifies that the root user can only access the proxy from the address 127.0.0.1. When logging in from another address, even if the password is correct, it will be rejected.

LDAP Authentication

To facilitate unified authentication management for enterprise users, proxy also provides LDAP (Lightweight Directory Access Protocol) authentication. LDAP authentication now supports MySQL and PostgreSQL clients.

At the same time, Proxy allows users to access LDAP in a very flexible way, such as:

  1. It can be configured to use LDAP by default, so that all users can pass LDAP authentication;
  2. It supports configuring auth attribute for users, specifying that users use password or LDAP authentication. Each user can use different methods;
  3. Each user can use different LDAP authenticators, that is, connect to different LDAP services;
  4. It supports specifying DN templates for users to meet the needs of complex scenarios;
  5. Support LDAPS protocol, which can further improve the security level.

Security of Management #

In DBPlusEngine-Proxy, users can perform multiple dimension management operations through DistSQL, including but not limited to:

  1. Proxy configuration management, such as transaction type, log switch, etc;
  2. Logical database management;
  3. Storage resource management;
  4. Data sharding rule management;
  5. Read write splitting rule management;
  6. Encryption and decryption rule management;
  7. Database discovery rule management;
  8. Shadow rule management;
  9. Metadata viewing, etc.

Due to the powerful DistSQL function, the database administrator can assign different DistSQL authorities to different users to achieve proxy management security. For example:

GRANT DIST SHOW SHARDING ON sharding_db.* TO 'sharding'@'%';

Through the above authorization statement, grant ‘sharding@%’ the authorization of ‘view sharding rules’ in the logical database sharding_db, then the user can execute ‘SHOW SHARDING TABLE RULES’, ‘SHOW SHARDING BINDING TABLE RULES’ and other sharding related RQLs in sharding_db, but cannot execute other unauthorized DistSQL.

For example, if the’sharding@%‘user executes the CREATE SHARDING TABLE RULE statement at this time, he will get an exception prompt:

Access denied for operation [CREATE] of subject sharding_db.table_name:SHARDING.]

To grant all DistSQL authorization to the ‘sharding@%’ user, do the following:

GRANT DIST RDL,RQL,RAL ON sharding_db.* TO 'sharding'@'%';

Access Security #

Data access security is one of the necessary capabilities of enterprise database. As the portal of the distributed database cluster, DBPlusEngine-Proxy provides users with comprehensive access control capabilities.

Unlike traditional centralized databases or single protocol databases, DBPlusEngine has the ability to manage multi type underlying databases and connect multi protocol clients. It will face many challenges in access control:

  • Different database types have different logical concepts;
  • Different database types use different dialects;
  • Different databases provide different storage structures.

In order to provide users with a consistent security experience, Proxy shields the differences of underlying databases and provides a unified and easy-to-use security system, which has the following characteristics:

  • Fine granularity authorization management: support database level, table level and column level access control;
  • Unified interactive language: use DistSQL to manage users and authorizations, which is applicable to different database protocols;
  • Independent storage: the authorization information is stored in the governance center of DBPlusEngine and does not depend on the underlying database;
  • Real time effective: the control of users and authorizations takes effect in real time without restart or manual refresh.

For example, if the administrator grants the ‘sharding@%’ authorization to query and write to the t_order table in sharding_db, you can execute distsql as follows:

GRANT DIST SELECT, INSERT ON sharding_db.t_order TO ‘sharding’@'%';

After the operation is completed, the ‘sharding@%’ user will get the corresponding authorization immediately. If the user performs unauthorized operations, such as DELETE, he will receive a rejection prompt:

=> DELETE FROM sharding_db.t_order WHERE id = 1;
Access denied for operation [DELETE] of subject sharding_db.t_order]

Storage Security #

In recent years, more and more attention has been paid to data security and privacy protection. Dealing with data encryption and data desensitization has also become an important task for many enterprises. Under such a trend, the technical team faces new challenges:

  • Is the encryption process on the application side or in the database?
  • In the case of the application side, each project team will have coding tasks, and the later modification of the algorithm will also have a great impact;
  • If different databases have different encryption methods in the database, there will be a variety of encryption methods in the enterprise, which can not be reused well.

To this end, the DBPlusEngine provides a new idea. By supporting configurable encryption rules in the data engine, it not only simplifies the workload of the development team, but also provides more general data security services. The data encryption function in DBPlusEngine has many advantages:

  • No invasion to the application, no need to modify the source code;
  • Widely available, applicable to any database accessed through DBPlusEngine;
  • A variety of encryption algorithms are available, including SHA, SM3, SM4 and other built-in algorithm support;
  • Flexible customization: users can customize the encryption algorithm according to SPI to meet personalized needs.

Take the user’s mobile number encryption scenario as an example. After the application accesses the DBPlusEngine, it only needs to define an ENCRYPT RULE through YAML or DistSQL, such as:

CREATE ENCRYPT RULE t_user (
COLUMNS(
(NAME=mobile,CIPHER=mobile,TYPE(NAME="AES",PROPERTIES("aes-key-value"="123456abc"'")))));

In this way, when the application writes a new record to the t_user table, the DBPlusEngine will automatically encrypt the mobile field and store the ciphertext content. On the contrary, the ciphertext can also be decrypted automatically during query, and the query results obtained by the application will be restored to plaintext.

In this way, it not only ensures that the application is insensitive to the data encryption process, but also ensures the security of the stored data. If a hacker event such as “off database” occurs, only the ciphertext content will be leaked, and the security risk will be greatly reduced.

Architecture Advantages #

  • High performance

The driver side has been polished for many years, and its efficiency is close to that of native JDBC, with extreme performance. Among the mainstream competitive products, only ShardingSphere currently provides the driver access form.

  • High compatibility

The agent side is applicable to any client compatible with MySQL/PostgreSQL protocol, and the driver side supports any database that implements JDBC specification.

  • High expansion

In the database replacement scenario, the DBPlusEngine can meet the requirements of smooth business migration and zero intrusion into business.

  • Low cost

It retains the original database technology stack, is friendly to DBA, and has low learning and management costs.

  • Safe and stable

It does not interfere with the database kernel, and provides increased capacity based on the mature database base, taking into account security and stability.

  • Elastic extension

With elastic computing and storage capabilities, it can meet the changing needs of computing and storage layers, and complete database splitting and migration online;

  • Open ecology

The pluggable model enables the kernel, functional components and ecological docking to be pluggable and expanded in a flexible way. Users can customize unique systems suitable for business like building blocks.