zabbix-server.service: control process exited, code=exited status=1

今天zabbix服务器mysql突然占用了大量内存,导致zabbix server服务器本身出现了内存占用大于95%的报警。占用大量内存,本着重启解决一切的原则,重启了该服务器。重启后发现zabbix server服务出现了无法启动的问题。

故障现象

一,检查服务是否启动


[root@shengbao.org ~]# ps aux|grep zabbix
zabbix 1063 0.0 0.0 78800 1300 ? S 14:00 0:00 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix 1085 0.0 0.0 78800 1412 ? S 14:00 0:00 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
zabbix 1086 0.0 0.0 78800 2136 ? S 14:00 0:00 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
zabbix 1087 0.0 0.0 78800 2136 ? S 14:00 0:00 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
zabbix 1088 0.0 0.0 78800 2404 ? S 14:00 0:00 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
zabbix 1089 0.0 0.0 78800 2208 ? S 14:00 0:00 /usr/sbin/zabbix_agentd: active checks #1 [idle 1 sec]

二,重启zabbix_server服务


[root@shengbao.org ~]# systemctl restart zabbix_server

三,重启后检查该服务状态。


[root@shengbao.org ~]# systemctl status zabbix-server
● zabbix-server.service - Zabbix Server
Loaded: loaded (/usr/lib/systemd/system/zabbix-server.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Mon 2020-11-30 14:06:18 CST; 12s ago
Main PID: 11458 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/zabbix-server.service
Nov 30 14:06:18 shengbao.org systemd[1]: zabbix-server.service: control process exited, code=exited status=1
Nov 30 14:06:18 shengbao.org systemd[1]: Unit zabbix-server.service entered failed state.
Nov 30 14:06:18 shengbao.org systemd[1]: zabbix-server.service failed.

四,出现失败后,检查/var/log/message检查系统日志


[root@shengbao.org ~]# cat /var/log/messages
Nov 30 14:01:47 shengbao.org systemd: zabbix-server.service holdoff time over, scheduling restart.
Nov 30 14:01:47 shengbao.org systemd: Starting Zabbix Server...
Nov 30 14:01:47 shengbao.org systemd: PID file /run/zabbix/zabbix_server.pid not readable (yet?) after start.
Nov 30 14:01:47 shengbao.org systemd: Started Zabbix Server.
Nov 30 14:01:50 shengbao.org kill: Usage:
Nov 30 14:01:50 shengbao.org kill: kill [options] <pid|name> [...]
Nov 30 14:01:50 shengbao.org kill: Options:
Nov 30 14:01:50 shengbao.org kill: -a, --all do not restrict the name-to-pid conversion to processes
Nov 30 14:01:50 shengbao.org kill: with the same uid as the present process
Nov 30 14:01:50 shengbao.org kill: -s, --signal <sig> send specified signal
Nov 30 14:01:50 shengbao.org kill: -q, --queue <sig> use sigqueue(2) rather than kill(2)
Nov 30 14:01:50 shengbao.org kill: -p, --pid print pids without signaling them
Nov 30 14:01:50 shengbao.org kill: -l, --list [=<signal>] list signal names, or convert one to a name
Nov 30 14:01:50 shengbao.org kill: -L, --table list signal names and numbers
Nov 30 14:01:50 shengbao.org kill: -h, --help display this help and exit
Nov 30 14:01:50 shengbao.org kill: -V, --version output version information and exit
Nov 30 14:01:50 shengbao.org kill: For more details see kill(1).
Nov 30 14:01:50 shengbao.org systemd: zabbix-server.service: control process exited, code=exited status=1
Nov 30 14:01:50 shengbao.org systemd: Unit zabbix-server.service entered failed state.
Nov 30 14:01:50 shengbao.org systemd: zabbix-server.service failed.

五,通过系统日志仅仅能查看到服务启动失败,造成失败的愿意并没有直观的显示出来。查看zabbix应用日志。


[root@shengbao.org ~]# cat /var/log/zabbix/zabbix-server.log
3520:20201130:144346.442 [Z3001] connection to database 'zabbix' failed: [1040] Too many connections
3520:20201130:144346.442 Cannot connect to the database. Exiting...
3521:20201130:144346.444 [Z3001] connection to database 'zabbix' failed: [1040] Too many connections
3521:20201130:144346.444 Cannot connect to the database. Exiting...

六,通过观察zabbix应用日志,发现报错是数据库Too many connections,修改数据库连接数解决该问题


[root@shengbao.org ~]# mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 1247
Server version: 5.5.65-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
#查看当前最大连接数
MariaDB [(none)]> show variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
1 row in set (0.00 sec)
#设置全局最大连接数为1000.
MariaDB [(none)]> set global max_connections=1000;
Query OK, 0 rows affected (0.00 sec)
#检查是否设置成功。
MariaDB [(none)]> show variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 1000 |
+-----------------+-------+
1 row in set (0.00 sec)
MariaDB [(none)]> exit

七,重启服务故障解决


#systemctl restart zabbix_server
#systemctl status zabbix_server

给TA买糖
共{{data.count}}人
人已赞赏
笔记

如何将操作系统从 CentOS 或 Oracle Linux 转换为 RHEL

2021-3-18 15:06:36

笔记

Device /dev/sdb excluded by a filter

2021-3-18 15:25:47

此心远送浑河岸,斟别酒,唱阳关,临别无语空长叹,酒已阑,曲未残,人初散,心长怀去后,杳鱼雁,对遥山,当时无计锁雕鞍,去后思量悔应晚,别时容易见时难!
17 条回复 A文章作者 M管理员
  1. 飞机动人

    哈哈哈哈哈

  2. 水壶留胡子

    超喜欢这里的

  3. 满意扯美女

    点赞是不可能点赞的

  4. 小松鼠秀丽

    百度来的,网站挺好看的

  5. 可靠迎铃铛

    百度来的,网站挺好看的

  6. 天空舒心

    我只是随便看看

  7. 万宝路刻苦

    我只是随便看看

  8. 电源乐观

    哈哈

个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索