Logo
Traffic Governance

Traffic Governance #

As data scale continues to expand, the distributed approach of using multi-node clusters is becoming a trend. The ability to unify the management of the overall view of the cluster, and the ability to control at a fine-grained level for individual components, is an indispensable feature in architecture based on the separation of storage and computation.

Overview #

SphereEx-DBPlusEngine provides users with an entry point for traffic governance, unified management capabilities for centralized management of clusters, and granular operational capabilities in the event of a single point of failure.

The challenges of centralized management include unified management of the status of database storage nodes and middleware compute nodes, and the ability to detect the latest changes in a distributed environment in real-time, further providing a basis for cluster control and scheduling.

Basic Concepts #

  • Fusing

Blocking the connection entry of SphereEx-DBPlusEngine. When a SphereEx-DBPlusEngine node exceeds the load, access to the database from that node is stopped so that the database can ensure enough resources to serve other nodes.

Applicable Scenarios #

When a compute node in the SphereEx-DBPlusEngine cluster exceeds the load, the fuse function blocks the traffic applied to that compute node to ensure that the whole cluster continues to provide stable services.

Usage Prerequisites #

  • SphereEx-DBPlusEngine and database cluster have been installed and deployed, and the service is running normally.
  • SphereEx-Console has been installed and deployed, and the service is running normally. (Optional)

Usage Restrictions #

None

Caution #

After the fusion operation is executed, the request in progress is interrupted and the service is aware of it.

Principle Introduction #

In the face of overloaded traffic, SphereEx-DBPlusEngine can fuse a node to keep the entire database cluster running.

Usage Guidelines #

  1. Confirm the current cluster traffic

Confirm the overall cluster traffic through SphereEx-Console or database to determine if there are any compute nodes that are about to be overloaded.

  1. Melt down a compute node

After confirming the session status of the node to be fused and confirming that it can be fused, fuse the node on the command line in another Proxy and confirm the result. The request of the fused node will be interrupted.

  1. Confirming the fusing result

Observe the traffic situation again through SphereEx-Console.

Operation Guide #

  1. Confirm the current cluster and each node traffic via Console

  2. Confirm the cluster environment

  3. Confirm the current requests in the Proxy

  4. fuse the specified compute node

  5. Confirm the result of the fusion

Configuration Example #

  1. Confirm the current cluster and node traffic via Console

  2. Confirm the cluster environment

mysql> SHOW COMPUTE NODES;
+--------------------------------------+----------------+------+--------+-----------+-----------+--------+---------+
| instance_id                          | host           | port | status | mode_type | worker_id | labels | version |
+--------------------------------------+----------------+------+--------+-----------+-----------+--------+---------+
| 01faadb5-3b09-4449-ad1a-252ce0c50117 | 192.168.xx.102 | 3307 | OK     | Cluster   | -1        |        | 1.3.0   |
| 9fca0beb-edc9-4b0f-8e3c-1a3187461565 | 192.168.xx.103 | 3307 | OK     | Cluster   | -1        |        | 1.3.0   |
+--------------------------------------+----------------+------+--------+-----------+-----------+--------+---------+
2 rows in set (0.01 sec)
  1. Confirm the request currently being made in the Proxy
mysql> SHOW PROCESSLIST;
  1. Fuse the specified calculation node
mysql> DISABLE INSTANCE '9fca0beb-edc9-4b0f-8e3c-1a3187461565';
Query OK, 0 rows affected (0.06 sec)

mysql> SHOW COMPUTE NODES;
+--------------------------------------+----------------+------+---------------+-----------+-----------+--------+---------+
| instance_id                          | host           | port | status        | mode_type | worker_id | labels | version |
+--------------------------------------+----------------+------+---------------+-----------+-----------+--------+---------+
| 01faadb5-3b09-4449-ad1a-252ce0c50117 | 192.168.xx.102 | 3307 | OK            | Cluster   | -1        |        | 1.3.0   |
| 9fca0beb-edc9-4b0f-8e3c-1a3187461565 | 192.168.xx.103 | 3307 | CIRCUIT_BREAK | Cluster   | -1        |        | 1.3.0   |
+--------------------------------------+----------------+------+---------------+-----------+-----------+--------+---------+
2 rows in set (0.02 sec)
  1. Confirm the results of the fuse

The traffic can be observed again through the Console.

If the operation is performed manually on the fused node, the operation will not be executed.

mysql> SELECT * FROM WHERE t_user WHERE ID=1;
ERROR 13010 (01000): Circuit break open, the request has been ignored.

FAQ #

  1. Is it possible to fuse all calculation nodes?

    No, you cannot. The fuse operation can’t fuse the current node, so one compute node will remain in the compute cluster.

  2. Does fusing a given compute node affect the outstanding requests on that node?

    Yes, it does. Requests that are being executed are interrupted and the business is aware of it.

  3. Does the meltdown operation have any impact on the monitoring logs and the governance center?

    No. There is no impact.

  4. Does fusing the Proxy affect the Traffic Rule policy?

    No. There is no impact. A fused Proxy will not receive Traffic Rule traffic.