运维人员直接把测试的机器重启了，/usr目录不存在，重新安装新的MySQL，接着测试库重启myshard，有些端口能起来，有些不能起来
创新互联建站是一家专业从事成都网站设计、成都做网站、外贸网站建设的网络公司。作为专业网站建设公司,创新互联建站依托的技术实力、以及多年的网站运营经验,为您提供专业的成都网站建设、营销型网站建设及网站设计开发服务！
root@ubuntu:/data/rds/myshard/bin# ./shard_ctrl status shard_d 0 is down shard_d 11 is working shard_d 12 is working shard_d 13 is working shard_d 14 is down shard_d 15 is down shard_d 16 is down shard_d 17 is down
查看不能起来端口的日志
May 18 11:25:54 info shard_live_test_111_d[17763]: HAVE_EPOLL May 18 11:25:54 debug shard_live_test_111_d[17763]: output queue ok May 18 11:25:54 debug shard_live_test_111_d[17763]: output queue ok May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 1, min: 20, max: 20, idle: 0 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 2, min: 20, max: 20, idle: 1 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 3, min: 20, max: 20, idle: 2 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 4, min: 20, max: 20, idle: 3 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 5, min: 20, max: 20, idle: 4 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 6, min: 20, max: 20, idle: 5 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 7, min: 20, max: 20, idle: 6 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 8, min: 20, max: 20, idle: 7 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 9, min: 20, max: 20, idle: 8 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 10, min: 20, max: 20, idle: 9 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 11, min: 20, max: 20, idle: 10 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 12, min: 20, max: 20, idle: 11 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 13, min: 20, max: 20, idle: 12 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 14, min: 20, max: 20, idle: 13 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 15, min: 20, max: 20, idle: 14 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 16, min: 20, max: 20, idle: 15 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 17, min: 20, max: 20, idle: 16 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 18, min: 20, max: 20, idle: 17 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 19, min: 20, max: 20, idle: 18 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1, cur: 20, min: 20, max: 20, idle: 19 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1_nobinlog, cur: 1, min: 20, max: 20, idle: 0 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1_nobinlog, cur: 2, min: 20, max: 20, idle: 1 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 notice shard_live_test_111_d[17763]: inc pool, type: MySQL, name: server1_nobinlog, cur: 3, min: 20, max: 20, idle: 2 [PartitionConnectionPool.cpp, #76] May 18 11:25:54 err shard_live_test_111_d[17763]: Failed to connect to database: server1_nobinlog, Error: Too many connections May 18 11:25:54 err shard_live_test_111_d[17763]: build pool error! [PartitionConnectionPool.cpp, #69] May 18 11:25:54 err shard_live_test_111_d[17763]: build pool [server1_nobinlog] error [PartitionServerPools.cpp, #141] May 18 11:25:54 err shard_live_test_111_d[17763]: Build Pools failed! [db_proxy.cpp, #153]
发现myshard报mysql的连接数已经满了，加大连接数
mysql> show variables like 'max_connections'; +-----------------+-------+ | Variable_name | Value | +-----------------+-------+ | max_connections | 151 | +-----------------+-------+ 1 row in set (0.00 sec) mysql> mysql> mysql> set global max_connections = 1000; Query OK, 0 rows affected (0.00 sec)
再次重启myshard的端口，就可以顺利启动了
root@ubuntu:/data/rds/myshard/bin# ./shard_ctrl status shard_d 0 is down shard_d 11 is working shard_d 12 is working shard_d 13 is working shard_d 14 is working shard_d 15 is working shard_d 16 is working shard_d 17 is working
这时发现810端口还是起不来，看日志报一个binlog_sync.properties文件不存在
May 18 11:39:08 notice shard_live_test_100_d[24364]: inc pool, type: MySQL, name: server1_nobinlog, cur: 17, min: 20, max: 20, idle: 16 [PartitionConnectionPool.cpp, #76] May 18 11:39:08 notice shard_live_test_100_d[24364]: inc pool, type: MySQL, name: server1_nobinlog, cur: 18, min: 20, max: 20, idle: 17 [PartitionConnectionPool.cpp, #76] May 18 11:39:08 notice shard_live_test_100_d[24364]: inc pool, type: MySQL, name: server1_nobinlog, cur: 19, min: 20, max: 20, idle: 18 [PartitionConnectionPool.cpp, #76] May 18 11:39:08 notice shard_live_test_100_d[24364]: inc pool, type: MySQL, name: server1_nobinlog, cur: 20, min: 20, max: 20, idle: 19 [PartitionConnectionPool.cpp, #76] May 18 11:39:08 err shard_live_test_100_d[24364]: open binlog sync status file [/data/myshard/live_test/shard_0/binlog_sync.properties] error:[2-No such file or directory] [BinlogSynchronizer.cpp, #77] May 18 11:39:08 notice shard_live_test_100_d[24364]: dec pool, type: MySQL, name: server1, cur: 19, min: 20, max: 20, idle: 19 [PartitionConnectionPool.cpp, #165] May 18 11:39:08 notice shard_live_test_100_d[24364]: dec pool, type: MySQL, name: server1, cur: 18, min: 20, max: 20, idle: 18 [PartitionConnectionPool.cpp, #165] May 18 11:39:08 notice shard_live_test_100_d[24364]: dec pool, type: MySQL, name: server1, cur: 17, min: 20, max: 20, idle: 17 [PartitionConnectionPool.cpp, #165] May 18 11:39:08 notice shard_live_test_100_d[24364]: dec pool, type: MySQL, name: server1, cur: 16, min: 20, max: 20, idle: 16 [PartitionConnectionPool.cpp, #165]
这个文件是记录myshard下面MySQL生成binlog的位置点，相当于mysql-bin.index的，但大小是0，
root@ubuntu:/data/myshard/live_test/shard_0# ls -l total 12 -rw-r--r-- 1 root root 0 May 17 18:52 binlog_sync.properties drwxr-xr-x 2 root root 4096 May 17 00:00 deleted drwxr-xr-x 2 root root 4096 May 17 00:00 local drwxr-xr-x 2 root root 4096 May 17 00:00 remote -rw------- 1 root root 0 May 18 11:39 shard.lock
因为更换了MySQL，所以可以把它删除掉，让它重新生成新的位置点，重启8810成功了
May 18 11:44:23 info shard_live_test_100_d[25977]: HAVE_EPOLL May 18 11:44:23 debug shard_live_test_100_d[25977]: output queue ok May 18 11:44:23 debug shard_live_test_100_d[25977]: output queue ok May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 1, min: 20, max: 20, idle: 0 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 2, min: 20, max: 20, idle: 1 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 3, min: 20, max: 20, idle: 2 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 4, min: 20, max: 20, idle: 3 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 5, min: 20, max: 20, idle: 4 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 6, min: 20, max: 20, idle: 5 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 7, min: 20, max: 20, idle: 6 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 8, min: 20, max: 20, idle: 7 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 9, min: 20, max: 20, idle: 8 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 10, min: 20, max: 20, idle: 9 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 11, min: 20, max: 20, idle: 10 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 12, min: 20, max: 20, idle: 11 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 13, min: 20, max: 20, idle: 12 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 14, min: 20, max: 20, idle: 13 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 15, min: 20, max: 20, idle: 14 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 16, min: 20, max: 20, idle: 15 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 17, min: 20, max: 20, idle: 16 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 18, min: 20, max: 20, idle: 17 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 19, min: 20, max: 20, idle: 18 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1, cur: 20, min: 20, max: 20, idle: 19 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 1, min: 20, max: 20, idle: 0 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 2, min: 20, max: 20, idle: 1 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 3, min: 20, max: 20, idle: 2 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 4, min: 20, max: 20, idle: 3 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 5, min: 20, max: 20, idle: 4 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 6, min: 20, max: 20, idle: 5 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 7, min: 20, max: 20, idle: 6 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 8, min: 20, max: 20, idle: 7 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 9, min: 20, max: 20, idle: 8 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 10, min: 20, max: 20, idle: 9 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 11, min: 20, max: 20, idle: 10 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 12, min: 20, max: 20, idle: 11 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 13, min: 20, max: 20, idle: 12 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 14, min: 20, max: 20, idle: 13 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 15, min: 20, max: 20, idle: 14 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 16, min: 20, max: 20, idle: 15 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 17, min: 20, max: 20, idle: 16 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 18, min: 20, max: 20, idle: 17 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 19, min: 20, max: 20, idle: 18 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: inc pool, type: MySQL, name: server1_nobinlog, cur: 20, min: 20, max: 20, idle: 19 [PartitionConnectionPool.cpp, #76] May 18 11:44:23 notice shard_live_test_100_d[25977]: trying connect to [127.0.0.1:12505,local]...... [BinlogScratcher.cpp, #64] May 18 11:44:23 notice shard_live_test_100_d[25977]: trying connect to [127.0.0.1:12505,remote]...... [BinlogScratcher.cpp, #64] May 18 11:44:23 err shard_live_test_100_d[25977]: open connect [127.0.0.1:12505,local] ,error:[111:Connection refused] [BinlogScratcher.cpp, #71] May 18 11:44:23 err shard_live_test_100_d[25977]: open connect [127.0.0.1:12505,remote] ,error:[111:Connection refused] [BinlogScratcher.cpp, #71] May 18 11:44:23 info shard_live_test_100_d[25977]: create listen port 8810 success May 18 11:44:23 info shard_live_test_100_d[25977]: server start, epoll_prt=0x1d9d630 May 18 11:44:33 notice shard_live_test_100_d[25977]: request:0 proc:0 droped:0 active:0
重新生成位置文件了
