从 node3 的角度来看,此时 node1,node2 处于 UNREACHABLE 状态 。
三个节点,只有一个节点处于 ONLINE 状态,不满足组复制的多数派原则 。此时,node3 只能查询 , 写操作会被阻塞 。
mysql> select * from slowtech.t1 where id=1;+----+------+| id | c1 |+----+------+| 1 | a |+----+------+1 row in set (0.00 sec)mysql> delete from slowtech.t1 where id=1;阻塞中 。。。
又过了 16s(这里的 16s,实际上与 group_replication_member_expel_timeout 参数有关),node1、node2 会将 node3 驱逐出(expel)集群 。此时,集群只有两个节点组成 。
看看 node1 的日志及集群状态 。
2022-07-31T13:03:23.576960-00:00 0 [Warning] [MY-011499] [Repl] Plugin group_replication reported: 'Members removed from the group: 192.168.244.30:3306'2022-07-31T13:03:23.577091-00:00 0 [System] [MY-011503] [Repl] Plugin group_replication reported: 'Group membership changed to 192.168.244.10:3306, 192.168.244.20:3306 on view 16592724636525403:3.'
mysql> select member_id,member_host,member_port,member_state,member_role from performance_schema.replication_group_members;+--------------------------------------+----------------+-------------+--------------+-------------+| member_id | member_host | member_port | member_state | member_role |+--------------------------------------+----------------+-------------+--------------+-------------+| 207db264-0192-11ed-92c9-02001700754e | 192.168.244.10 | 3306 | ONLINE | PRIMARY || 2cee229d-0192-11ed-8eff-02001700f110 | 192.168.244.20 | 3306 | ONLINE | PRIMARY |+--------------------------------------+----------------+-------------+--------------+-------------+2 rows in set (0.00 sec)
再来看看 node3 的 , 日志没有新的输出,节点状态也没变化 。
mysql> select member_id,member_host,member_port,member_state,member_role from performance_schema.replication_group_members;+--------------------------------------+----------------+-------------+--------------+-------------+| member_id | member_host | member_port | member_state | member_role |+--------------------------------------+----------------+-------------+--------------+-------------+| 207db264-0192-11ed-92c9-02001700754e | 192.168.244.10 | 3306 | UNREACHABLE | PRIMARY || 2cee229d-0192-11ed-8eff-02001700f110 | 192.168.244.20 | 3306 | UNREACHABLE | PRIMARY || 4cbfdc79-0192-11ed-8b01-02001701bd0a | 192.168.244.30 | 3306 | ONLINE | PRIMARY |+--------------------------------------+----------------+-------------+--------------+-------------+3 rows in set (0.00 sec)
恢复网络连接接下来我们恢复 node3 与 node1、node2 之间的网络连接 。
# iptables -F# date "+%Y-%m-%d %H:%M:%S"2022-07-31 13:07:30
首先看看 node3 的日志
2022-07-31T13:07:30.464179-00:00 0 [Warning] [MY-011494] [Repl] Plugin group_replication reported: 'Member with address 192.168.244.10:3306 is reachable again.'2022-07-31T13:07:30.464226-00:00 0 [Warning] [MY-011494] [Repl] Plugin group_replication reported: 'Member with address 192.168.244.20:3306 is reachable again.'2022-07-31T13:07:30.464239-00:00 0 [Warning] [MY-011498] [Repl] Plugin group_replication reported: 'The member has resumed contact with a majority of the members in the group. Regular operation is restored and transactions are unblocked.'2022-07-31T13:07:37.458761-00:00 0 [ERROR] [MY-011505] [Repl] Plugin group_replication reported: 'Member was expelled from the group due to network failures, changing member status to ERROR.'2022-07-31T13:07:37.459011-00:00 0 [Warning] [MY-011630] [Repl] Plugin group_replication reported: 'Due to a plugin error, some transactions were unable to be certified and will now rollback.'2022-07-31T13:07:37.459037-00:00 0 [ERROR] [MY-011712] [Repl] Plugin group_replication reported: 'The server was automatically set into read only mode after an error was detected.'2022-07-31T13:07:37.459431-00:00 31 [ERROR] [MY-011615] [Repl] Plugin group_replication reported: 'Error while waiting for conflict detection procedure to finish on session 31'2022-07-31T13:07:37.459478-00:00 31 [ERROR] [MY-010207] [Repl] Run function 'before_commit' in plugin 'group_replication' failed2022-07-31T13:07:37.459811-00:00 33 [System] [MY-011565] [Repl] Plugin group_replication reported: 'Setting super_read_only=ON.'2022-07-31T13:07:37.465738-00:00 34 [System] [MY-013373] [Repl] Plugin group_replication reported: 'Started auto-rejoin procedure attempt 1 of 3'2022-07-31T13:07:37.496466-00:00 0 [System] [MY-011504] [Repl] Plugin group_replication reported: 'Group membership changed: This member has left the group.'2022-07-31T13:07:37.498813-00:00 36 [System] [MY-010597] [Repl] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_applier' executed'. Previous state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 351, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.2022-07-31T13:07:39.653028-00:00 34 [System] [MY-013375] [Repl] Plugin group_replication reported: 'Auto-rejoin procedure attempt 1 of 3 finished. Member was able to join the group.'2022-07-31T13:07:40.653484-00:00 0 [System] [MY-013471] [Repl] Plugin group_replication reported: 'Distributed recovery will transfer data using: Incremental recovery from a group donor'2022-07-31T13:07:40.653822-00:00 0 [System] [MY-011503] [Repl] Plugin group_replication reported: 'Group membership changed to 192.168.244.10:3306, 192.168.244.20:3306, 192.168.244.30:3306 on view 16592724636525403:4.'2022-07-31T13:07:40.670530-00:00 46 [System] [MY-010597] [Repl] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='192.168.244.20', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''.2022-07-31T13:07:40.682990-00:00 47 [Warning] [MY-010897] [Repl] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.2022-07-31T13:07:40.687566-00:00 47 [System] [MY-010562] [Repl] Slave I/O thread for channel 'group_replication_recovery': connected to master 'repl@192.168.244.20:3306',replication started in log 'FIRST' at position 42022-07-31T13:07:40.717851-00:00 46 [System] [MY-010597] [Repl] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='192.168.244.20', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.2022-07-31T13:07:40.732297-00:00 0 [System] [MY-011490] [Repl] Plugin group_replication reported: 'This server was declared online within the replication group.'2022-07-31T13:07:40.732511-00:00 53 [System] [MY-011566] [Repl] Plugin group_replication reported: 'Setting super_read_only=OFF.'
推荐阅读
- 云原生之旅 - 9)云原生时代网关的后起之秀Envoy Proxy 和基于Envoy 的 Emissary Ingress
- 华为云 MRS 基于 Apache Hudi 极致查询优化的探索实践
- 基于 .NET 7 的 QUIC 实现 Echo 服务
- Mysql InnoDB Redo log
- 基于docker和cri-dockerd部署kubernetes v1.25.3
- 基于 Docker 构建轻量级 CI 系统:Gitea 与 Woodpecker CI 集成
- 使用LabVIEW实现基于pytorch的DeepLabv3图像语义分割
- 之七 2流高手速成记:基于Dubbo&Nacos的微服务简要实现
- MySQL的下载、安装、配置
- 我的Vue之旅 09 数据数据库表的存储与获取实现 Mysql + Golang