CheatSheet_Linux
显示文件中所有未注释的行grep -v "#" /usr/local/nginx/conf/nginx.conf |grep -v "^$"
公网IP地址质量测试IP质量体检报告 https://github.com/xykt/IPQuality
双栈
bash <(curl -sL IP.Check.Place)
v4
bash <(curl -Ls IP.Check.Place) -4
查看时间范围内的 atop 记录atop -r /var/log/atop_20240704 -b 03:53 -e 03:59
查找 Grafana 日志中remote_addr字段出现次数awk '{ match($0, /remote_addr=([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/, ip); if(ip[1]) print ip[1] }' grafana.log | sort | uniq -c
使用 Ssh-keygen 从 Pem 文件中导出公钥ssh-keygen -y -f awesome.pem > public.pub
Rsync 命令传输数据保留文件的原本的信息, 归档模式, 传输前压缩, 以及展示进度 和 暂存发送文件进度.
rsync -azP /source/path/ /destination/path/
rsync -aP /source/path/ /destination/path/
查看磁盘设备的详细信息包括 EBS Volume ID 和 设备驱动类型
lsblk -f -o +SERIAL,SCHED,TRAN,STATE
强制退出程序并生成 Corefile组合键用于发送 SIGQUIT 信号,用于终止正在运行的进程。与Ctrl+C不同的是,Ctrl+\会生成一个core文件,用于调试程序。同时在某些特定的场景下, 这个指令并Ctrl+c停止进程的速度快。
Ctrl+\
查看当前系统中 CPU 使用率最高的前 20 个进程ps auxwww --sort -%cpu | head -20
列出当前系统中内存使用最高的前 20 个进程ps auxwww --sort -rss | head -20
列出系统中所有进程的详细信息ps auxf --width=200
以完整格式列出系统中所有进程的详细信息ps auxwwwf
Curl 输出各个阶段所花费的时间Network_tc控制流量-update
curl -s -w "time_namelookup:%{time_namelookup}\ntime_connect: %{time_connect}\ntime_appconnect: %{time_appconnect}\ntime_redirect: %{time_redirect}\ntime_pretransfer: %{time_pretransfer}\ntime_starttransfer: %{time_starttransfer}\ntime_total: %{time_total}\n" -o /dev/null $HTTP_URL
对于命令输出内容的解释:
curl -o /dev/null \ # 将下载的文件输出到 /dev/null (即丢弃)
-s \ # 静默模式,不显示进度条等信息
-w "time_namelookup:%{time_namelookup}\n \ # 输出 DNS 解析时间
time_connect: %{time_connect}\n \ # 输出建立 TCP 连接所花费的时间
time_appconnect: %{time_appconnect}\n \ # 输出建立 SSL/TLS 连接所花费的时间
time_redirect: %{time_redirect}\n \ # 输出重定向所花费的时间
time_pretransfer: %{time_pretransfer}\n \ # 输出从开始到准备好传输所花费的时间
time_starttransfer: %{time_starttransfer}\n \ # 输出从开始到第一个字节被传输所花费的时间
time_total: %{time_total}\n" \ # 输出整个传输所花费的总时间
$HTTP_URL # 要下载的文件 URL
查看与 TCP 内存管理和拥塞控制相关的内核参数通常这些配置会出现在 [[Linux_Sysctl 参数记录]] /etc/sysctl.conf 文件中。
sysctl -a | egrep "rmem|wmem|tcp_mem|adv_win|moderate|slow_start"
列出系统中所有用户的 Crontab 任务cat /etc/passwd | cut -f 1 -d : | xargs -I {} crontab -l -u {}
查看 AWS EC2 的 Billing CodeRedhat / SUSE 的 Enterprise 版本在 AWS EC2 上面运行实例会带有 Billing Code 的, 这标记 OS 是否是付过费用的,会不会收到安全补丁, 不会使用系统内的订阅管理器。Finding AMI billing and usage detailsAMI billing information fields
# IDMSv1
curl http://169.254.169.254/latest/dynamic/instance-identity/document
# IDMSv2
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` && curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/dynamic/instance-identity/document
查看 SUSE 的注册情况,以及是否启用 LTSSsudo SUSEConnect --list-extensions
查看网络可能存在的丢包和异常ip 命令查看链路层丢包指标.
ip -s link show eth0
ethtool 命令查看网卡硬件队列的丢包
ethtool -S eth0 | grep drop
OpenSSL 测试指定域名并输出调试信息openssl s_client -debug --connect $URL:443
指定算法和协议版本
openssl s_client -debug --connect $URL:443 -tls1_2 -cipher RC4
查看证书的有效期:
echo | openssl s_client -servername $URL:443 -connect $URL:443 2>/dev/null | openssl x509 -noout -dates
查看启动以来的内核日志journalctl -xeak
查看特定级别的日志journalctl -xeak -p3
# 日志级别的补充说明:
# FROM..TO. The log levels are the usual syslog log levels as documented in syslog(3), i.e. "emerg" (0), "alert" (1), "crit" (2), "err" (3), "warning" (4), "notice" (5), "info" (6), "debug" (7). If a
# single log level is specified, all messages with this log level or a lower (hence more important) log level are shown. If a range is specified, all messages within the range are shown, including both
# the start and the end value of the range. This will add "PRIORITY=" matches for the specified priorities.
journald记录的所有启动会话(boots)信息journalctl --list-boots
# 输出所有重启的历史记录
# 命令会通过序号的方式标记重启的记录, 0, -1, -2, -3 等等
查看上次系统启动的日志journalctl -xeab -1
查看所有日志不使用pager进行分页journalctl -xea --no-pager
查看特定 Service 日志journalctl -xeau SERVICE_NAME
查看并追踪特定 Service 日志journalctl -xeafu SERVICE_NAME
查看最近 100 行journalctl -n 100 --no-pager -xeau SERVICE_NAME
查看指定时间范围的日志journalctl -xea --no-pager -S "2024-01-31 18:20:00" -U "2024-01-31 18:40:00"
# 显示从什么时间开始的日志 或者 到什么时间为止的日志。
# Format: 2012-10-30 18:17:16
# 可以仅仅提供日期字段, 时间会默认全0
# -S, --since=, -U, --until=
journalctl -xea -u SERVICE_NAME --no-pager -S "2024-01-31 17:40:00" -U "2024-01-31 23:40:00"
Tshark 命令简单说明tshark -i ens5 -n -f 'tcp dst port 32123' -T fields -e frame.number -e frame.time_epoch -e frame.time_delta_displayed -e ip.src -e tcp.srcport -e ip.dst -e tcp.dstport -e tcp.time_delta -e tcp.stream -e tcp.len -e tcp.analysis.ack_rtt
# 抓取 dns 请求, 并显示 IP TTL 以及 DNS TTL, 格式化输出
> sudo tshark -i ens5 -Y 'dns' -T fields -e ip.src -e ip.dst -e dns.qry.name -e ip.ttl -E header=y -E separator='/t' -E quote='d' -E occurrence=f
Iptables 常用命令关于 iptables 的扩展内容: [[Linux/L ...
CheatSheet_Kubernetes
移除所有失败的podkubectl delete pod --field-selector="status.phase==Failed"
查看证书信息查看 AWS LoadBalancer 证书的信息,检查证书的有效期:
kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io aws-load-balancer-webhook -ojsonpath={.webhooks[0].clientConfig.caBundle} | base64 -d | openssl x509 -noout -text
使用 Debug 容器# 给特定的容器附加一个Sidecar, 并启动shell。
kubectl debug -it --image=public.ecr.aws/amazonlinux/amazonlinux:latest aws-node-cpmck
# netshoot容器, 比较方便的用来进行网络部分的调试。
# 项目仓库地址: https://github.com/nicolaka/netshoot
kubectl debug mypod -it --image=nicolaka/netshoot
查看EKS集群插件的兼容范围aws eks describe-addon-versions --kubernetes-version 1.25 --addon-name vpc-cni | grep addonVersion
列出节点上所有容器的镜像名称# ssh 到节点上面执行
nerdctl inspect $(nerdctl ps -a -q) | grep -i "image.:" | sort -f
# 清理节点上未使用的镜像,不仅仅是 dangling image
nerdctl image prune -af
# 设置 nerdctl 命令的自动补全
nerdctl completion bash > /etc/bash_completion.d/nerdctl
Nsenter 进入容器[[EKS_nsenter_into_container]]
nerdctl inspect 02182f3e9137 | grep -i pid
"Pid": 10306,
nsenter -t 10306 -n
给系统加上一点点压力nerdctl run -ti --rm reg.liarlee.site/docker.io/polinux/stress-ng:latest --help
nerdctl run -ti --rm reg.liarlee.site/docker.io/polinux/stress-ng:latest --cpu 1
强制删除 Podkubectl delete pod/applications-7c5f67d6bd-72jcf --force --grace-period=0
当然也可以删除什么其他的资源, 都可以, 实在不行就是 Finalizer 的影响, 需要删除 finalizer 了; 还有另一个问题是 pdb 可能会阻止一部分资源的删除, 也是需要查的.
CheatSheet_awscli
查看实例和对应实例的系统平台信息aws ec2 describe-instances --query "Reservations[*].Instances[*].{InstanceId:InstanceId,PlatformDetails:Platform}" --output table
查看实例和EBS的关联关系aws ec2 describe-volumes --query 'Volumes[*].[VolumeId, Attachments[0].InstanceId, Size]' --output table
CheatSheet_Databases
统计 MySQL 数据库占用的空间mysql Ver 8.4.1 for Linux on x86_64 (MySQL Community Server - GPL)
mysql>
select TABLE_SCHEMA, concat(truncate(sum(data_length)/1024/1024,2),' MB') as data_size, concat(truncate(sum(index_length)/1024/1024,2),'MB') as index_size from information_schema.tables group by TABLE_SCHEMA ORDER BY data_size desc;
+--------------------+-----------+------------+
| TABLE_SCHEMA | data_size | index_size |
+--------------------+-----------+------------+
| mysql | 7.51 MB | 0.34MB |
| test_db | 230.31 MB | 30.31MB |
| sys | 0.01 MB | 0.00MB |
| information_schema | 0.00 MB | 0.00MB |
| performance_schema | 0.00 MB | 0.00MB |
+--------------------+-----------+------------+
5 rows in set (0.17 sec)
查看 MYSQL 的版本mysql>
status;
--------------
mysql Ver 8.4.1 for Linux on x86_64 (MySQL Community Server - GPL)
Connection id: 26
Current database:
Current user: root@localhost
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 8.4.1 MySQL Community Server - GPL
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: utf8mb4
Db characterset: utf8mb4
Client characterset: latin1
Conn. characterset: latin1
UNIX socket: /var/run/mysqld/mysqld.sock
Binary data as: Hexadecimal
Uptime: 19 hours 36 min 40 sec
Threads: 3 Questions: 661 Slow queries: 4 Opens: 737 Flush tables: 3 Open tables: 150 Queries per second avg: 0.009
--------------
启用 MYSQL 慢日志官方文档: https://dev.mysql.com/doc/refman/8.4/en/slow-query-log.html
[mysqld]
slow_query_log=ON
long_query_time=10
查看当前运行中的慢日志设置
mysql>
show variables like 'slow_query%';
+---------------------+--------------------------------------+
| Variable_name | Value |
+---------------------+--------------------------------------+
| slow_query_log | OFF |
| slow_query_log_file | /var/lib/mysql/mysql-master-slow.log |
+---------------------+--------------------------------------+
2 rows in set (0.01 sec)
mysql>
show variables like 'long_query_time';
+-----------------+-----------+
| Variable_name | Value |
+-----------------+-----------+
| long_query_time | 10.000000 |
+-----------------+-----------+
1 row in set (0.01 sec)
基于Mysql的文档记录:
To include slow administrative statements in the slow query log, enable the log_slow_admin_statements system variable. Administrative statements include ALTER TABLE, ANALYZE TABLE, CHECK TABLE, CREATE INDEX, DROP INDEX, OPTIMIZE TABLE, and REPAIR TABLE.
查看 Binlog 的内容mysql>
show binlog events in 'binlog.000005';
```r
B+树笔记
Start下面的两个非常通俗的解释了 B tree 的定义, 以及btree 是如何完成数据插入以及查找的, 不同的程序会使用不同结构的 btree 变体, 是在通常的Btree 逻辑框架上面添加更多的限制条件和规定。
Understanding B-Trees: The Data Structure Behind Modern Databaseshttps://www.youtube.com/watch?v=K1a2Bk8NrYQ&t=42s
B-trees and database indexeshttps://planetscale.com/blog/btrees-and-database-indexes#how-mysql-uses-b-trees
Kubernetes NFS CSI 启动 bitnami postgres 镜像权限不足
开始准备在kubernetes 集群里面部署一个 pgsql, 存储一些自己的个人数据,postgres pod 启动不了,pod 会反复 CrashLoopback, 查看其中postgres 进程的容器, 会遇到这样的报错:
fixing permissions on existing directory /var/lib/postgresql/data … ok
creating subdirectories … ok
selecting dynamic shared memory implementation … posix
selecting default max_connections … 20
selecting default shared_buffers … 400kB
selecting default time zone … Etc/UTC
creating configuration files … ok
running bootstrap script … 2020-07-28 19:47:30.936 UTC [73] FATAL: data directory “/var/lib/postgresql/data” has wrong ownership
2020-07-28 19:47:30.936 UTC [73] HINT: The server must be started by the user that owns the data directory.
child process exited with exit code 1
initdb: removing contents of data directory “/var/lib/postgresql/data”
这看上去就是目录权限不正确 … 我看了好几遍 bitnami 的文档, 都是基础的错误排除.
两个建议:
检查权限。
使用 VolumePermission 设置 initcontainer 初始化容器权限。
使用 VolumePermission 初始化目录权限Bitnami 的这个提供 helm 参数, 可以方便的设置 initcontainer 来进行 pv 的权限设置. 于是为了偷懒, 就 set 了这个参数.尝试使用 VolumePermission 的容器来进行 Volume path 权限的初始化, 但是 initcontainer 也启动不了, 报错:
chown: changing ownership of '/bitnami/postgresql': Invalid argument
这initcontainer的报错就离谱了, 这命令怎么会有无效参数。
检查权限在另一个机器上面 pull docker 镜像, 启动容器进去看用户, 没问题 uid 1001/gid 0, 是一个非常合理的设置。然后检查目录权限, 也符合要求。docker 可以启动这个 image, 那么这个问题应该是出现在 Kubernetes 和 NFS 的配合上面。talos 创建的集群, 用了 TrueNAS 提供的 NFS 服务, 并在集群里面安装了 CSI 驱动: https://github.com/kubernetes-csi/csi-driver-nfs按照这个配置检查了一边, 没发现问题, 启动测试的 pod 也可以正常挂载和写入。 驱动本身也没有报错的日志.
算了还是在 pod 里面直接进去看看吧。尝试修改statefulset强制指定容器主进程命令: 加上 command 和 args , 使用 sleep infinity.
进入检查目录权限的时候发现这个目录的uid和gid的映射关系有问题, 长这样:
drwxr-xr-x 2 4294967294 4294967294 24 2024-09-08 11:50 data
然后继续搜索, 发现这些奇怪的资料: https://www.truenas.com/community/threads/nobody-4294967294-and-nfsv4.99352/ https://www.suse.com/support/kb/doc/?id=000017244
看完我的理解是, NFSv4 会尝试解析并在域内进行 idmapping, 如果 mapping 不到, 会将 pod 中进程使用的 uid 和 gid 转换成了 nobody, 在 TrueNAS 上面我的测试是我已经设置了正确的权限, mapalluser 给了 uid 1001. 但是这个并不好用.我不理解, 一直到我看到了: https://www.truenas.com/community/threads/issue-with-user-mapping-when-mounting-nfs-share-on-ubuntu-18-04.79451/
What you’ve called “permissions” are actually the user and group ids of any file/directory you’ve listed on your new Ubuntu server. If you’re only seeing numeric values for uid & gid in “ls” output on your Ubuntu server, it simply means there’s no user on the Ubuntu server with a uid/gid of 1001/1001. The output of getent group and getent passwd would confirm that. What non-root users have you added to your new Ubuntu server?P.S. The reason why unchecking the “NFSv3 ownership model for NFSv4” causes the permissions to read “nobody:4294967294” is explained here: https://mwl.io/archives/796So unless you plan to start using NFSv4 with kerboros, I’d stick to using NFSv4 with the NFsv3 ownership model.
将 v4 版本的权限模型, 切换回 v3 之后, 全好了。 如果有折腾这个的, 这可能是一个比较奇葩的问题, 在这几个东西互相配合的时候会出错.
TigerVNC 在 RHEL9 的安装和配置
安装 VNC 步骤记录初始化系统, 安装图形化. ( 我还安装的 虚拟化宿主机, 我需要所以就安装了.
# 更新系统软件包到最新
sudo dnf update -y
# 查看哪些可用的 Group, (方便
sudo dnf grouplist
# 安装 Server GUI
sudo dnf groupinstall -y "Server with GUI"
# 安装 KVM (按需
sudo dnf groupinstall -y "Virtualization Host"
# 安装 TigerVNC 服务
sudo dnf install -y tigervnc-server
# 关闭 SElinux
sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
# 可以重启 或者 设置为 Permissive
sudo setenforce 0
# 关闭防火墙.
sudo systemctl stop firewalld
# 关闭防火墙开机启动, 当然可以使用 firewall-cmd 管理防火墙条目, 放行端口即可.
sudo systemctl disable --now firewalld
配置 VNC config
# 复制 VNC 服务, 创建 1 号位服务.
sudo cp -prv /usr/lib/systemd/system/vncserver@.service /etc/systemd/system/vncserver\@:1.service
分配 :1 图形化接口给 ec2-user , 也可以分配给其他用户.
vim /etc/tigervnc/vncserver.users
:1=ec2-user
配置共享session, 没有安全性的设置.
vim /etc/tigervnc/vncserver-config-defaults
alwaysshared
切换到普通用户初始化su - ec2-user
# 设置密码
vncpasswd
设置分辨率
vim ~/.vnc/config
geometry=1920x1080
开机启动
sudo systemctl enable vncserver@:1.service
sudo systemctl daemon-reload && sudo systemctl restart vncserver@:1
排查方法# 查看 service 状态
sudo systemctl status vncserver@:1
# 查看日志
sudo journalctl -xefu vncserver@:1
# 查看 vncserver 日志
# 这个日志是需要到对应用户的家目录下查看的, 哪个用户创建的 vnc 会话, 就去哪个用户的家目录下看.
# 一般的日志格式是 [HOSTNAME]:1.log
cat ~/.vnc/*.log
MySQL 计算内存用量
对于 MySQL,可以按以下示例计算 RDS for MySQL 数据库实例的大致内存使用量:Maximum MySQL Memory Usage = innodb_buffer_pool_size + key_buffer_size + ((read_buffer_size + read_rnd_buffer_size + sort_buffer_size + join_buffer_size) * max_connections)
From repost: https://repost.aws/zh-Hans/knowledge-center/low-freeable-memory-rds-mysql-mariadb
From MySQL Offical:5.7 https://dev.mysql.com/doc/refman/5.7/en/memory-use.html8.0 https://dev.mysql.com/doc/refman/8.0/en/memory-use.html
存储引擎的说明:https://dev.mysql.com/doc/refman/8.0/en/innodb-in-memory-structures.html
SElinux问题排查
https://docs.redhat.com/zh_hans/documentation/red_hat_enterprise_linux/9/html/using_selinux/analyzing-an-already-found-selinux-denial_troubleshooting-problems-related-to-selinux安装相关软件.
root@ip-172-31-54-198:/var/mnt/docker_root# rpm-ostree install policycoreutils-python-utils setroubleshoot-server
Checking out tree 1873357... done
Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora updates-archive
Updating metadata for 'updates'... done
Updating metadata for 'updates-archive'... done
Importing rpm-md... done
rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2024-03-12T11:45:42Z solvables: 3
rpm-md repo 'updates'; generated: 2024-07-12T04:06:14Z solvables: 21267
rpm-md repo 'fedora' (cached); generated: 2024-04-14T18:51:11Z solvables: 74881
rpm-md repo 'updates-archive'; generated: 2024-06-30T05:31:17Z solvables: 22869
Resolving dependencies... done
Will download: 21 packages (6.0 MB)
Downloading from 'updates'... done
Downloading from 'fedora'... done
Downloading from 'updates-archive'... done
Importing packages... done
Checking out packages... done
Running pre scripts... done
Running post scripts... done
Running posttrans scripts... done
Writing rpmdb... done
Writing OSTree commit... done
Staging deployment... done
Added:
checkpolicy-3.6-3.fc40.x86_64
gobject-introspection-1.80.1-1.fc40.x86_64
ima-evm-utils-1.5-4.fc40.x86_64
initscripts-service-10.23-1.fc40.noarch
libfsverity-1.4-12.fc40.x86_64
policycoreutils-python-utils-3.6-3.fc40.noarch
python3-audit-4.0.1-1.fc40.x86_64
python3-dasbus-1.7-6.fc40.noarch
python3-dbus-1.3.2-6.fc40.x86_64
python3-gobject-base-3.48.2-1.fc40.x86_64
python3-libselinux-3.6-4.fc40.x86_64
python3-libsemanage-3.6-3.fc40.x86_64
python3-libxml2-2.12.7-1.fc40.x86_64
python3-policycoreutils-3.6-3.fc40.noarch
python3-rpm-4.19.1.1-1.fc40.x86_64
python3-setools-4.5.1-2.fc40.x86_64
python3-systemd-235-9.fc40.x86_64
rpm-build-libs-4.19.1.1-1.fc40.x86_64
rpm-sign-libs-4.19.1.1-1.fc40.x86_64
setroubleshoot-plugins-3.3.14-9.fc40.noarch
setroubleshoot-server-3.3.33-1.fc40.x86_64
Changes queued for next boot. Run "systemctl reboot" to start a reboot
root@ip-172-31-54-198:/var/mnt/docker_root# systemctl reboot
查看建议的方案这个命令会给出建议的安全上下文, 对于通常的解决方案来说应该是足够的.我这里尝试容器里面运行的mysqld进程去写用户的家目录, 遇到了拒绝, 在调整之后,还是没有办法完全匹配上下文, 导致被拒绝, 后面还是设置了 permissive.默认情况下 Fedora CoreOS SElinux 是开启的状态, 关闭的话需要同时关闭系统的selinux 以及 docker 的selinux support, 比较麻烦, 还是 permissive 吧.
root@ip-172-31-54-198:~# sealert -l "*"
SELinux is preventing mysqld from write access on the directory mysql.
***** Plugin catchall_labels (83.8 confidence) suggests *******************
If you want to allow mysqld to have write access on the mysql directory
Then you need to change the label on mysql
Do
# semanage fcontext -a -t FILE_TYPE 'mysql'
where FILE_TYPE is one of the following: bpf_t, cifs_t, container_file_t, container_var_lib_t, fusefs_t, hugetlbfs_t, nfs_t, svirt_home_t, tmpfs_t, virt_home_t.
Then execute:
restorecon -v 'mysql'
***** Plugin catchall (17.1 confidence) suggests **************************
If you believe that mysqld should be allowed write access on the mysql directory by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'mysqld' --raw | audit2allow -M my-mysqld
# semodule -X 300 -i my-mysqld.pp
Additional Information:
Source Context system_u:system_r:container_t:s0:c114,c1019
Target Context system_u:object_r:mnt_t:s0
Target Objects mysql [ dir ]
Source mysqld
Source Path mysqld
Port <Unknown>
Host ip-172-31-54-198
Source RPM Packages
Target RPM Packages
SELinux Policy RPM selinux-policy-targeted-40.22-1.fc40.noarch
Local Policy RPM selinux-policy-targeted-40.22-1.fc40.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Enforcing
Host Name ip-172-31-54-198
Platform Linux ip-172-31-54-198 6.8.11-300.fc40.x86_64 #1
SMP PREEMPT_DYNAMIC Mon May 27 14:53:33 UTC 2024
x86_64
Alert Count 1
First Seen 2024-07-13 11:16:08 CST
Last Seen 2024-07-13 11:16:08 CST
Local ID b331ca4f-5699-4d11-94d2-84638e0f0f8a
Raw Audit Messages
type=AVC msg=audit(1720840568.80:300): avc: denied { write } for pid=3596 comm="mysqld" name="mysql" dev="nvme0n1p4" ino=9437365 scontext=system_u:system_r:container_t:s0:c114,c1019 tcontext=system_u:object_r: ...
MySQL 主从复制
被问了一个数据库的问题, 主从复制需要 主库锁表 和 主库重启 吗?
Setting Up Binary Log File Position Based Replication使用docker配置一遍, 测试一下记录了步骤, 有些东西还是 MySQL 的官方文档好用.
主库需要做的事情
需要有唯一的 server id. 这个可以设置 variables , 不需要重启.
MySQL [(none)]>
SHOW variables LIKE 'server_id';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| server_id | 198 |
+---------------+-------+
1 row in set (0.001 sec)
创建一个用户用来让从库同步数据. 这个也不需要重启.
MySQL [(none)]>
CREATE USER 'replication-user'@'172.31.62.236' IDENTIFIED BY '123123';
Query OK, 0 rows affected (0.006 sec)
MySQL [(none)]>
GRANT REPLICATION SLAVE ON *.* TO 'replication-user'@'172.31.62.236';
Query OK, 0 rows affected (0.004 sec)
MySQL [(none)]>
SHOW GRANTS FOR 'replication-user'@'172.31.62.236';
+----------------------------------------------------------------------+
| Grants for replication-user@172.31.62.236 |
+----------------------------------------------------------------------+
| GRANT REPLICATION SLAVE ON *.* TO `replication-user`@`172.31.62.236` |
+----------------------------------------------------------------------+
1 row in set (0.000 sec)
MySQL [(none)]>
FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.002 sec)
MySQL [(none)]>
SELECT Host,User,authentication_string FROM mysql.user;
+---------------+------------------+------------------------------------------------------------------------+
| Host | User | authentication_string |
+---------------+------------------+------------------------------------------------------------------------+
| % | root | $A$005$CK=G-6GdO6+*f4IpBYW8v6zGb62QvkVJIztfMwOYRKYK4VHdrnOcmM/jOzB |
| 172.31.62.236 | replication-user | $A$005$t4642jI)eV[+@6NFBplxAFoO/a1p1sHIdDMiRHvtyvKD4mUFajpDiZKUPA |
| localhost | mysql.infoschema | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED |
| localhost | mysql.session | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED |
| localhost | mysql.sys | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED |
| localhost | root | $A$005$y,)(7](i&=Mm+kO9Re/i.ywV/7DTELTS9.DN9Or6yCVIPlaVm/NOkg/RA |
+---------------+------------------+------------------------------------------------------------------------+
6 rows in set (0.001 sec)
确保主库会记录binlog, 或者是特定的 database 会记录 binlog. MySQL 8.0 以后默认会启用 binlog, 所以这个实际也不会需要重启.
文档中有描述, 在这个位置应该上一个 主库 的 read lock, 是为了确保记住的 binlog 是不变的, 如果业务比较忙的情况下, 会导致 binlog position 一直动.
mysql>
FLUSH TABLES WITH READ LOCK;
在主库上面查询 binlog 的状态, 也就是之前的 show master status; 现在这个命令已经被替换成:
MySQL [test]>
SHOW BINARY LOG STATUS\G
*************************** 1. row ***************************
File: binlog.000002
Position: 1086
Binlog_Do_DB: test
Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.000 sec)
看完位置之后就可以 unlock 了.
mysql>
UNLOCK TABLES;
从库需要做的事情
确认可以连接到主库. 网络部分.
创建一个复制任务, 设置下面的信息.mysql>
CHANGE REPLICATION SOURCE TO
SOURCE_HOST='172.31.54.198',
SOURCE_USER='replication-user',
SOURCE_PASSWORD='123123',
SOURCE_LOG_FILE='binlog.000002',
SOURCE_LOG_POS=4,
GET_SOURCE_PUBLIC_KEY=1;
# 如果您使用的复制用户帐户通过 `caching_sha2_password` 插件(默认)进行身份验证,并且您没有使用安全连接,则必须指定此选项或 `SOURCE_PUBLIC_KEY_PATH` 选项向副本提供 RSA 公钥。
启动同步.mysql>
START REPLICA;
mysql>
SHOW PROCESSLIST;
+----+-----------------+---------------------+------+---------+------+----------------------------------------------------------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-----------------+---------------------+------+---------+------+----------------------------------------------------------+------------------+
| 8 | event_scheduler | localhost | NULL | Daemon | 2062 | Waiting on empty queue | NULL |
| 16 | root | 172.31.47.174:56736 | NULL | Query | 0 | init | SHOW PROCESSLIST |
| 29 | system user | connecting host | NULL | Connect | 447 | Waiting for source to send event | NULL |
| 35 | system user | | NULL | Query | 79 | Replica has read all relay log; waiting for more updates | NULL |
| 36 | system user | | NULL | Query | 2149 | Waiting for an event from Coordinator | NULL |
| 37 | system user | | NULL | Connect | 79 | Waiting for an event from Coordinator | NULL |
| 38 | system ...