lvs筆記之nat&dr模型簡(jiǎn)單實(shí)現(xiàn)
lvs筆記之nat&dr模型簡(jiǎn)單實(shí)現(xiàn)
lvs
集群
實(shí)現(xiàn)
負(fù)載均衡
nat
ipvsadm使用說(shuō)明
-A: 添加一個(gè)集群服務(wù) -t: tcp -u: udp -f: firewall mark, 通常用于將兩個(gè)或以上的服務(wù)綁定為一個(gè)服務(wù)進(jìn)行處理時(shí)使用 例如httpd和https iptables mongo表一起使用 service-address: -t IP:port -u ip:port -f firewall_mark -s 調(diào)度方法,默認(rèn)為wlc -p timeout: persistent connection, 持久連接 -E 修改定義過(guò)的集群服務(wù) -D -t|u|f service-address:刪除指定的集群服務(wù) RS相關(guān): -a:向指定的Cluster services中添加RS -t|-u|-f service-address:指明將RS添加至哪個(gè)Cluster Service中 -r: 指定RS,可包含{IP[:port]},只有支持端口映射的LVS類(lèi)型才允許此處使用跟集群服務(wù)中不同的端口 LVS類(lèi)型: -g: Gateway模式,就是DR模型(默認(rèn)) -i: ipip模式,TUN模型 -m: masquerade地址偽裝自動(dòng)完成后半段的原地址轉(zhuǎn)換,NAT 指定RS權(quán)重: -w # 省略權(quán)重為1 -e: 修改指定的RS屬性 -d -t|u|f service-address -r server-address:從指定的集群服務(wù)中刪除某RS -C :清空集群服務(wù)配置 保存規(guī)則: ipvsadm-save ipvsadm -S 載入指定的規(guī)則: ipvsadm-restore ipvsadm -R 查看ipvs規(guī)則: -L [options] -n: 數(shù)字格式顯示IP地址 -c: 顯示連接數(shù)相關(guān)信息 --stats: 顯示統(tǒng)計(jì)數(shù)據(jù) --rate: 速率 --exact:顯示統(tǒng)計(jì)數(shù)據(jù)的精確值 --timeout: 超時(shí)時(shí)間 -Z: 計(jì)數(shù)器清零;
lvs-nat的簡(jiǎn)單實(shí)現(xiàn)
實(shí)驗(yàn)拓?fù)淙缦聢D:
如圖所示lvs-nat實(shí)現(xiàn)對(duì)后端2臺(tái)real server(搭建apache httpd服務(wù))進(jìn)行負(fù)載均衡。
step1:
1、按照拓?fù)渌九渲煤肐P地址 2、director需要準(zhǔn)備兩塊網(wǎng)卡(一塊公網(wǎng)網(wǎng)卡配置vip,一塊內(nèi)網(wǎng)網(wǎng)卡配置dip) 3、后端兩臺(tái)real server準(zhǔn)備一塊網(wǎng)卡(內(nèi)網(wǎng)網(wǎng)卡)配置rip,注意網(wǎng)關(guān)需要指向dip
step2各節(jié)點(diǎn)網(wǎng)絡(luò)設(shè)置如下:
以下是director上的配置: [root@localhost ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:37:63:02 brd ff:ff:ff:ff:ff:ff inet 192.168.2.146/24 brd 192.168.2.255 scope global dynamic eno16777736 valid_lft 4229sec preferred_lft 4229sec inet6 fe80::20c:29ff:fe37:6302/64 scope link valid_lft forever preferred_lft forever 3: eno33554984: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:37:63:0c brd ff:ff:ff:ff:ff:ff inet 192.168.253.153/24 brd 192.168.253.255 scope global eno33554984 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe37:630c/64 scope link valid_lft forever preferred_lft forever [root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno16777736 TYPE="Ethernet" BOOTPROTO="static" IPADDR=192.168.2.146 NETMASK=255.255.255.0 DEFROUTE="yes" PEERDNS="yes" PEERROUTES="yes" IPV4_FAILURE_FATAL="no" IPV6INIT="yes" IPV6_AUTOCONF="yes" IPV6_DEFROUTE="yes" IPV6_PEERDNS="yes" IPV6_PEERROUTES="yes" IPV6_FAILURE_FATAL="no" NAME="eno16777736" UUID="ee2e45b1-4b9c-41cb-8507-de93e3827896" DEVICE="eno16777736" ONBOOT="yes" [root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno33554984 TYPE="Ethernet" BOOTPROTO="static" IPADDR=192.168.253.153 NETMASK=255.255.255.0 DEFROUTE="yes" PEERDNS="yes" PEERROUTES="yes" IPV4_FAILURE_FATAL="no" IPV6INIT="yes" IPV6_AUTOCONF="yes" IPV6_DEFROUTE="yes" IPV6_PEERDNS="yes" IPV6_PEERROUTES="yes" IPV6_FAILURE_FATAL="no" NAME="eno33554984" DEVICE="eno33554984" ONBOOT="yes"
以下是rs1、和rs2的配置:
注意real server的網(wǎng)關(guān)要指向dip
step3在real server上進(jìn)行如下操作:
以下是real server1上進(jìn)行的操作: # yum install httpd -y &> /dev/null && echo success || echo failure #RS1安裝httpd success #安裝成功 #route add default gw 192.168.253.153 #設(shè)置默認(rèn)網(wǎng)關(guān)為Director的DIP # echo "<h1>This is Real Server 1 </h1>" > /var/www/html/index.html #添加測(cè)試網(wǎng)頁(yè) #systemctl start httpd #啟動(dòng)httpd服務(wù) 以下是real server2上進(jìn)行的操作: # yum install httpd -y &> /dev/null && echo success || echo failure #RS1安裝httpd success #安裝成功 #route add default gw 192.168.253.153 #設(shè)置默認(rèn)網(wǎng)關(guān)為Director的DIP # echo "<h1>This is Real Server 2 </h1>" > /var/www/html/index.html #添加測(cè)試網(wǎng)頁(yè) #systemctl start httpd #啟動(dòng)httpd服務(wù)
step4在director上進(jìn)行操作:
# yum install ipvsadm -y #安裝ipvsadm [root@localhost ~]# curl 192.168.253.169 #測(cè)試rs是否可以服務(wù) <h1>This is Real Server 1 </h1> [root@localhost ~]# curl 192.168.253.168 #測(cè)試rs是否可以服務(wù) <h1>This is Real Server 2 </h1> [root@localhost ~]# vim /etc/sysctl.conf #編輯內(nèi)核文件開(kāi)啟內(nèi)核路由轉(zhuǎn)發(fā) # System default settings live in /usr/lib/sysctl.d/00-system.conf. # To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file # # For more information, see sysctl.conf(5) and sysctl.d(5). net.ipv4.ip_forward = 1 #添加該項(xiàng)設(shè)置 [root@localhost ~]# sysctl -p #對(duì)剛才的修改進(jìn)行立即生效 net.ipv4.ip_forward = 1 #可見(jiàn)已經(jīng)生效 [root@localhost ~]# cat /proc/sys/net/ipv4/ip_forward #查看是否開(kāi)啟路由轉(zhuǎn)發(fā)功能 1 #開(kāi)啟 [root@localhost ~]# ipvsadm -A -t 192.168.2.146:80 -s rr [root@localhost ~]# ipvsadm -a -t 192.168.2.146:80 -r 192.168.253.169 -m [root@localhost ~]# ipvsadm -a -t 192.168.2.146:80 -r 192.168.253.168 -m [root@localhost ~]# ipvsadm -Ln #查看規(guī)則 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.2.146:80 rr -> 192.168.253.168:80 Masq 1 0 0 -> 192.168.253.169:80 Masq 1 0 0 [root@localhost ~]# curl http://192.168.2.146 <h1>This is Real Server 2 </h1> [root@localhost ~]# curl http://192.168.2.146 <h1>This is Real Server 1 </h1>
測(cè)試:
踩過(guò)的坑1
在實(shí)驗(yàn)過(guò)程中遇到了瀏覽器訪問(wèn)vip失敗,但是director上curlvip卻可以的情況后來(lái)排查發(fā)現(xiàn)real server開(kāi)啟了兩塊網(wǎng)卡(一塊外網(wǎng),一塊內(nèi)網(wǎng),瀏覽器可以通過(guò)外網(wǎng)ip訪問(wèn)real server上的web服務(wù)),當(dāng)我只啟用了內(nèi)網(wǎng)網(wǎng)卡時(shí),瀏覽器成功訪問(wèn)vip。
因此當(dāng)你實(shí)驗(yàn)失敗時(shí)請(qǐng)從以下兩點(diǎn)排查:
1、real server是否啟用了多塊網(wǎng)卡,如果是請(qǐng)禁用多余的 只啟用內(nèi)網(wǎng)網(wǎng)卡。并且把網(wǎng)關(guān)指向dip
2 關(guān)閉iptables和seLinux(或者設(shè)置合適的規(guī)則)
lvs-dr實(shí)現(xiàn)
實(shí)驗(yàn)拓?fù)淙缦拢?
后端兩臺(tái)real server搭建httpd服務(wù)(默認(rèn)已搭建完成并啟動(dòng)),各節(jié)點(diǎn)iptables和seLinux均已關(guān)閉
step1在director上執(zhí)行如下操作:
#ifconfig eno16777736:0 192.168.2.11/32 broadcast 192.168.2.11 up #route add -host 192.168.2.11 dev eno16777736:0
step2在real server1上執(zhí)行如下操作:
#echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore #echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce #echo 1 > /proc/sys/net/ipv4/conf/eno16777736/arp_ignore # echo 2 > /proc/sys/net/ipv4/conf/eno16777736/arp_announce #添加如上內(nèi)核參數(shù),注意要在director節(jié)點(diǎn)添加ipvs規(guī)則前做此步操作 #ifconfig lo:0 192.168.2.11/32 broadcast 192.168.2.11 up #route add -host 192.168.2.11 dev lo:0 #echo "<h1>This is Real Server 1 </h1>" > /var/www/html/index.html
step3在real server2上執(zhí)行如下操作:
#echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore #echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce #echo 1 > /proc/sys/net/ipv4/conf/eno16777736/arp_ignore # echo 2 > /proc/sys/net/ipv4/conf/eno16777736/arp_announce #添加如上內(nèi)核參數(shù),注意要在director節(jié)點(diǎn)添加ipvs規(guī)則前做此步操作 #ifconfig lo:0 192.168.2.11/32 broadcast 192.168.2.11 up #route add -host 192.168.2.11 dev lo:0 #echo "<h1>This is Real Server 2 </h1>" > /var/www/html/index.html
step4在director節(jié)點(diǎn)執(zhí)行如下操作:
#ipvsadm -A -t 192.168.2.11:80 -s rr #ipvsadm -a -t 192.168.2.11:80 -r 192.168.2.117 -g #ipvsadm -a -t 192.168.2.11:80 -r 192.168.2.135 -g
測(cè)試:
以上lvs-dr模型簡(jiǎn)單實(shí)現(xiàn)到此為止
總結(jié)
lvs特點(diǎn)是模型工作原理比較復(fù)雜,但是其服務(wù)配置卻非常簡(jiǎn)單,lvs集群的配置難點(diǎn)主要集中在對(duì)模型的理解上,對(duì)網(wǎng)絡(luò)的配置及排錯(cuò)上。
這僅僅只是lvs的簡(jiǎn)單實(shí)現(xiàn)后期會(huì)與其他集群服務(wù)配合進(jìn)行更復(fù)雜的拓?fù)洹?/p>