iscsi多路径,nginx服务
iSCSI多路径访问
服务端准备
节点规划
iscsi-server 添加一块仅主机模式网卡,添加一块硬盘
iscsi-client 添加一块仅主机模式网卡
环境准备:
[root@iscsi-server ~ 09:37:52]# yum install -y targetd targetcli[root@iscsi-server ~ 09:40:39]# systemctl enable target --nowCreated symlinkfrom/etc/systemd/system/multi-user.target.wants/target.service to/usr/lib/systemd/system/target.service.[root@iscsi-server ~ 09:42:10]# firewall-cmd --permanent --add-service=iscsi-targetFirewallD is not running[root@iscsi-server ~ 09:42:45]# firewall-cmd --reloadFirewallD is not running[root@iscsi-server ~ 09:42:57]# targetcli /backstores/block cerate myblock1 /dev/sdbWarning: Could not load preferences file/root/.targetcli/prefs.bin.Command not found cerate[root@iscsi-server ~ 09:44:07]# targetcli /backstores/block create myblock1 /dev/sdbCreated block storage object myblock1using/dev/sdb.[root@iscsi-server ~ 09:44:41]# targetcli /iscsi create iqn.2026-04.cloud.zhu.iscsi-server:disk1Created target iqn.2026-04.cloud.zhu.iscsi-server:disk1.Created TPG 1.Global pref auto_add_default_portal=true Created default portal listening on all IPs(0.0.0.0),port 3260.[root@iscsi-server ~ 09:45:32]# targetcli /iscsi/iqn.2026-04.cloud.zhu.iscsi-server:disk1/tpg1/luns create /backstores/block/myblock1Created LUN 0.[root@iscsi-server ~ 09:47:19]# targetcli /iscsi/iqn.2026-04.cloud.zhu.iscsi-server:disk1/tpg1/acls create iqn.2026-04.cloud.zhu.iscsi-clientCreated Node ACLforiqn.2026-04.cloud.zhu.iscsi-client Created mapped LUN 0.[root@iscsi-server ~ 09:48:23]# targetcli /iscsi/iqn.2026-04.cloud.zhu.iscsi-server:disk1/tpg1/portals delete 0.0.0.0 3260Deleted network portal 0.0.0.0:3260[root@iscsi-server ~ 09:49:21]# targetcli /iscsi/iqn.2026-04.cloud.zhu.iscsi-server:disk1/tpg1/portals create 10.1.8.10 3260Usingdefault IP port 3260 Created network portal 10.1.8.10:3260.[root@iscsi-server ~ 09:49:55]# targetcli /iscsi/iqn.2026-04.cloud.zhu.iscsi-server:disk1/tpg1/portals create 10.1.1.10 3260Usingdefault IP port 3260 Created network portal 10.1.1.10:3260.[root@iscsi-server ~ 09:50:04]# targetcli saveconfigConfiguration saved to/etc/target/saveconfig.json[root@iscsi-server ~ 09:50:17]# targetcli /iscsi/iqn.2026-04.cloud.zhu.iscsi-server:disk1/tpg1/portals lso- portals..........................................................................[Portals: 2]o- 10.1.1.10:3260.........................................................................[OK]o- 10.1.8.10:3260.........................................................................[OK]配置多路径
安装软件包
[root@iscsi-client ~ 09:38:02]# yum install -y device-mapper-multipath[root@iscsi-client ~ 10:31:17]# yum install -y iscsi-initiator-utils启用多路径
[root@iscsi-client ~ 10:34:56]# mpathconf --enable[root@iscsi-client ~ 10:35:24]# systemctl enable multipathd --now发现设备
[root@iscsi-client ~ 10:35:41]# vim /etc/iscsi/initiatorname.iscsi## 插入 InitiatorName=iqn.2026-04.cloud.zhu.iscsi-client##查到有俩个硬盘[root@iscsi-client ~ 10:36:05]# iscsiadm -m discovery -t st -p 10.1.8.1010.1.8.10:3260,1 iqn.2026-04.cloud.zhu.iscsi-server:disk1 10.1.1.10:3260,1 iqn.2026-04.cloud.zhu.iscsi-server:disk1[root@iscsi-client ~ 10:36:41]# iscsiadm -m node -L allLogging in to[iface: default, target: iqn.2026-04.cloud.zhu.iscsi-server:disk1, portal: 10.1.8.10,3260](multiple)Logging in to[iface: default, target: iqn.2026-04.cloud.zhu.iscsi-server:disk1, portal: 10.1.1.10,3260](multiple)Login to[iface: default, target: iqn.2026-04.cloud.zhu.iscsi-server:disk1, portal: 10.1.8.10,3260]successful.Login to[iface: default, target: iqn.2026-04.cloud.zhu.iscsi-server:disk1, portal: 10.1.1.10,3260]successful.确定设备是同一设备
[root@iscsi-client ~ 10:37:57]# /usr/lib/udev/scsi_id -g -u /dev/sdb3600140596addb73bb8d4f379c370fd3a[root@iscsi-client ~ 10:38:23]# /usr/lib/udev/scsi_id -g -u /dev/sdc3600140596addb73bb8d4f379c370fd3a##两个的id都是同一个监控多路径
[root@iscsi-client ~ 10:38:38]# multipath -llmpatha(3600140596addb73bb8d4f379c370fd3a)dm-3 LIO-ORG,myblock1 size=20G features='0'hwhandler='0'wp=rw|-+-policy='service-time 0'prio=1 status=active##主节|`-3:0:0:0 sdb 8:16 active ready running `-+-policy='service-time 0'prio=1 status=enabled## 从节`-4:0:0:0 sdc 8:32 active ready running使用设备
##格式化[root@iscsi-client ~ 10:39:01]# mkfs.xfs /dev/mapper/mpathameta-data=/dev/mapper/mpatha isize=512 agcount=4,agsize=1310720 blks = sectsz=512 attr=2,projid32bit=1 = crc=1 finobt=0,sparse=0data= bsize=4096 blocks=5242880,imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=2560,version=2 = sectsz=512 sunit=0 blks,lazy-count=1 realtime =none extsz=4096 blocks=0,rtextents=0##创建挂载点[root@iscsi-client ~ 10:42:00]# mkdir /mpatha##挂载[root@iscsi-client ~ 10:42:13]# mount /dev/mapper/mpatha /mpatha[root@iscsi-client ~ 10:42:38]# df -hFilesystem Size Used Avail Use% Mounted on devtmpfs 2.0G 0 2.0G 0%/dev tmpfs 2.0G 0 2.0G 0%/dev/shm tmpfs 2.0G 12M 2.0G 1%/run tmpfs 2.0G 0 2.0G 0%/sys/fs/cgroup/dev/mapper/centos_contenos7-root 50G 1.9G 49G 4%//dev/sda1 1014M 170M 845M 17%/boot/dev/mapper/centos_contenos7-home 146G 33M 146G 1%/home tmpfs 394M 0 394M 0%/run/user/0/dev/mapper/mpatha 20G 33M 20G 1%/mpatha##持久化挂载[root@iscsi-client ~ 10:42:41]# vim /etc/fstab##插入 /dev/mapper/mpatha/ /mpatha xfs _netdev 0 0##重启配置文件[root@iscsi-client ~ 10:45:23]# systemctl daemon-reload##测试挂载[root@iscsi-client ~ 10:45:40]# umount /mpatha[root@iscsi-client ~ 10:46:19]# mount /dev/mapper/mpatha /mpathamount:/dev/mapper/mpatha is already mounted or/mpatha busy/dev/mapper/mpatha is already mounted on/mpatha[root@iscsi-client ~ 10:47:00]# df -h /mpatha/Filesystem Size Used Avail Use% Mounted on/dev/mapper/mpatha 20G 33M 20G 1%/mpatha高可用性测试
server机器断开其中一个网卡ens36
[root@iscsi-server ~ 10:52:38]# nmcli device disconnect ens36Device'ens36'successfully disconnected.断开网卡之后,查看是否能继续使用硬盘
断开网卡之后,主从会切换
[root@iscsi-client ~ 10:53:47]# touch abc[root@iscsi-client ~ 10:54:29]# vim abc[root@iscsi-client ~ 10:55:31]# multipath -llmpatha(3600140596addb73bb8d4f379c370fd3a)dm-3 LIO-ORG,myblock1 size=20G features='0'hwhandler='0'wp=rw|-+-policy='service-time 0'prio=1 status=active|`-3:0:0:0 sdb 8:16 active ready running `-+-policy='service-time 0'prio=0 status=enabled `-4:0:0:0 sdc 8:32 failed faulty running[root@iscsi-client ~ 10:55:51]# cat abc123##断开其中一条网卡路径,另外一条会替补上,文件能够继续使用nginx服务器
节点规划
nginx部署
##下载服务[root@nginx-server ~ 11:45:23]# yum install -y wget[root@nginx-server ~ 11:31:52]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo[root@nginx-server ~ 11:45:49]# yum -y install nginx##启动服务[root@nginx-server ~ 11:46:02]# systemctl enable nginx --nowCreated symlinkfrom/etc/systemd/system/multi-user.target.wants/nginx.service to/usr/lib/systemd/system/nginx.service.##备份[root@nginx-server ~ 11:46:14]# mv /usr/share/nginx/html/index.html{,.ori}[root@nginx-server ~ 11:46:46]# vim /usr/share/nginx/html/index.html##里面追加内容##关闭防火墙[root@nginx-server ~ 11:47:37]# systemctl stop firewalld测试结果
虚拟主机
同一个 web 服务器提供多个站点。
虚拟主机支持多种方式:
- 主机名
- 端口号
- IP地址(基本不用)
[root@nginx-server ~ 14:08:50]# cp /etc/nginx/nginx.conf /etc/nginx/conf.d/vhost-name.conf[root@nginx-server ~ 14:10:10]# vim /etc/nginx/conf.d/vhost-name.conf[root@nginx-server ~ 14:14:51]# mkdir /usr/share/nginx/web{1,2}[root@nginx-server ~ 14:15:10]# echo web1.zhu.cloud > /usr/share/nginx/web1/index.html[root@nginx-server ~ 14:15:43]# echo web2.zhu.cloud > /usr/share/nginx/web2/index.html[root@nginx-server ~ 14:15:51]# systemctl restart nginx[root@nginx-server ~ 14:16:10]# vim /etc/hosts##添加,可以通过域名访问10.1.8.10 web1.zhu.cloud web2.zhu.cloud测试
[root@nginx-client ~ 14:19:45]# curl http://web1.zhu.cloud/web1.zhu.cloud[root@nginx-client ~ 14:19:48]# curl http://web2.zhu.cloud/web2.zhu.cloudwindows里面需要修改配置
配置目录:C盘/windows/system32/dirvers/etc/hosts
根据 port(端口)
[root@nginx-server ~ 14:33:13]# vim /etc/nginx/conf.d/vhost-port.confserver{listen 8081;server_name www.zhu.cloud;root/usr/share/nginx/8081;}server{listen 8082;server_name www.zhu.cloud;root/usr/share/nginx/8082;}[root@nginx-server ~ 14:50:08]# mkdir /usr/share/nginx/808{1,2}[root@nginx-server ~ 14:50:41]# echo hello8081 > /usr/share/nginx/8081/index.html[root@nginx-server ~ 14:51:20]# echo hello8082 > /usr/share/nginx/8082/index.html[root@nginx-server ~ 14:51:26]# systemctl restart nginx客户端测试
[root@nginx-client ~ 14:57:23]# vim /etc/hosts[root@nginx-client ~ 14:58:24]# curl http://www.zhu.cloud:8081hello8081[root@nginx-client ~ 14:58:41]# curl http://www.zhu.cloud:8082hello8082配置 SSL/TLS
[root@nginx-server certs 15:24:43]# openssl genrsa -out www.key 2048Generating RSA private key,2048 bit long modulus.....................+++.............................................+++e is 65537(0x10001)[root@nginx-server certs 15:25:18]# openssl req -new -key www.key -out www.csr -subj "/C=CN/ST=JS/L=NJ/O=LG/OU=DEVOPS/CN=www.zhu.cloud/emailAddress=webadmin@zhu.cloud"[root@nginx-server certs 15:27:42]# openssl x509 -req -days 3650 -in www.csr -signkey www.key -out www.crtSignature ok subject=/C=CN/ST=JS/L=NJ/O=LG/OU=DEVOPS/CN=www.zhu.cloud/emailAddress=webadmin@zhu.cloud Getting Private key[root@nginx-server certs 15:28:39]# mkdir /etc/ssl/certs/www.zhu.cloud[root@nginx-server certs 15:29:16]# mv www* /etc/ssl/certs/www.zhu.cloud[root@nginx-server certs 15:29:35]# cp /etc/nginx/nginx.conf /etc/nginx/conf.d/vhost-www.zhu.cloud-ssl.conf[root@nginx-server certs 15:30:12]# cd[root@nginx-server ~ 15:34:50]# vim /etc/nginx/conf.d/vhost-www.zhu.cloud-ssl.confserver{listen 443 ssl http2;listen[::]:443 ssl http2;server_name www.zhu.cloud;root/usr/share/nginx/html;#证书ssl_certificate"/etc/ssl/certs/www.zhu.cloud/www.crt";#密钥ssl_certificate_key"/etc/ssl/certs/www.zhu.cloud/www.key";}server{listen 80;listen[::]:80;server_name www.zhu.cloud;root/usr/share/nginx/html;# 添加重定向return301 https://$host$request_uri;}[root@nginx-server ~ 15:41:17]# systemctl restart nginx测试
[root@nginx-client ~ 15:39:30]# curl -k https://www.zhu.cloudhello 123123[root@nginx-client ~ 15:41:39]# curl -Lk https://www.zhu.cloud/hello 123123[root@nginx-client ~ 15:41:53]# curl -Lk http://www.zhu.cloud/hello 123123[root@nginx-client ~ 15:42:03]# curl http://www.zhu.cloud/<html> <head><title>301 Moved Permanently</title></head> <body> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx/1.20.1</center> </body> </html>配置基本认证
[root@nginx-server ~ 15:47:11]# yum install -y httpd-tools[root@nginx-server ~ 16:23:59]# vim /etc/nginx/conf.d/vhost-www.zhu.cloud-ssl.confserver{listen 443 ssl http2;listen[::]:443 ssl http2;server_name www.zhu.cloud;root/usr/share/nginx/html;ssl_certificate"/etc/ssl/certs/www.zhu.cloud/www.crt";ssl_certificate_key"/etc/ssl/certs/www.zhu.cloud/www.key";location/auth-basic/{auth_basic"Basic Auth";auth_basic_user_file"/etc/nginx/.htpasswd";}}server{listen 80;listen[::]:80;server_name www.zhu.cloud;root/usr/share/nginx/html;# 添加重定向return301 https://$host$request_uri;}[root@nginx-server ~ 16:24:34]# systemctl restart nginx[root@nginx-server ~ 16:24:36]# htpasswd -b -c /etc/nginx/.htpasswd zhu 123Adding passwordforuser zhu[root@nginx-server ~ 16:25:15]# mkdir /usr/share/nginx/html/auth-basic[root@nginx-server ~ 16:25:37]# vim /usr/share/nginx/html/auth-basic/index.html##输入一些内容测试
[root@nginx-client ~ 16:30:56]# curl -ku zhu:123 https://10.1.8.10/auth-basic/qwertgyhgfdserfg[root@nginx-client ~ 16:35:25]# curl -ku zhu:12 https://10.1.8.10/auth-basic/<html> <head><title>401 Authorization Required</title></head> <body> <center><h1>401 Authorization Required</h1></center> <hr><center>nginx/1.20.1</center> </body> </html>