如何建立Varnish CDN集群 - AFF大佬 - 提供主机优惠信息深度测评和服务器运维编程技术
当前位置: 主页 » 技术干货 » 如何建立Varnish CDN集群

如何建立Varnish CDN集群

      2019年05月05日   阅读 492 次     2 评论   Tags: · ·

在构建CDN的架构上,Varnish和squid用的最多,各有优劣点,本文以Varnish为基础创建Varnish的CDN,加速网站,可以用DNS解析的方式,把吃灰小鸡利用起来,比如某小鸡对移动线路优秀,那么把来自移动的流量导向此小鸡,联通电信同理,在结合Varnish集群,大大减轻源站的负载压力并合理利用吃灰小鸡。

参考:

1.使用Varnish从内存加速你的WordPress网站
2.基于运营商的分布式DNS搭建
3.自建DNS权威服务器全过程(多域名解析)
4.为博客在HK自建了一个CDN节点

Varnish架构图

集群实验环境:Centos-7.6  Varnish-6.2  Nginx-1.16.0

一般情况下,会在Nginx或者Haproxy负载均衡层,进行心跳检测和浮动IP,而且会加上健康检测,来避免宕机的危险,正常流程是外网映射内网,内网机器之间实现LVS的VIP功能,由于本站小鸡全部是外网,没有内网所以只能实现在外网之间的通信。

流程:
http->https->varnish集群->源站
https->varnish集群->源站

由于Varnish 暂不支持 https协议,收费版支持https协议,所以我们用Nginx来反代实现HTTPS协议,Nginx负载均衡鸡配置如下:



cat /etc/nginx/conf.d/affdalao.conf 

server {
    listen 80;
    server_name  www.affdalao.com affdalao.com;
 
    rewrite ^/(.*) https://www.affdalao.com/$1 permanent;
}


upstream affdalao_com{
#    ip_hash;
    server 127.0.0.1:6081;
    server Varnish节点外网IP:6081;
}

server {
        listen 443 ssl http2 fastopen=3 reuseport;

	server_name  www.affdalao.com affdalao.com;

	error_log  /var/log/nginx/error.log  crit;
        ssl_certificate      /etc/letsencrypt/live/affdalao.com/fullchain.pem;
        ssl_certificate_key  /etc/letsencrypt/live/affdalao.com/privkey.pem;
        ssl_trusted_certificate /etc/letsencrypt/live/affdalao.com/chain.pem;
	ssl_dhparam /etc/nginx/ssl/dhparam.pem;

	ssl_protocols TLSv1.2;
	ssl_prefer_server_ciphers on;

	ssl_ciphers "ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS";


	
	ssl_session_cache shared:SSL:10m;
	ssl_session_timeout 10m;

	
	# OCSP Stapling ---
	# fetch OCSP records from URL in ssl_certificate and cache them
	ssl_stapling on;
	ssl_stapling_verify on;

	location /status {
            stub_status on;
            access_log off;
	}



        location / {
            proxy_pass http://affdalao_com;
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Forwarded-Port 443;
            proxy_set_header Host $host;
 
 

        }

}


Varnish节点,只需要安装Varnish即可,并按以下进行Varnish节点配置,Varnish(1-N)配置文件一致,唯一不一致的是运行配置文件,运行配置文件需要指定监听Varnish节点外网IP,配置如下:



###Varnish配置文件:

cat /etc/varnish/default.vcl 
vcl 4.0;


backend default {
  .host = "源站外网IP";
  .port = "8081";           
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
  .max_connections = 128;
}

acl purger {
   "localhost";
   "127.0.0.1";
}

sub vcl_recv {

	if (req.method == "PURGE") {
		if (!client.ip ~ purger) {
			return(synth(405, "This IP is not allowed to send PURGE requests."));
  		}
		return (purge);
	}


	if (req.restarts == 0) {
		if (req.http.X-Forwarded-For) {
		set req.http.X-Forwarded-For = client.ip;
  		}
	}


	if (req.http.Authorization || req.http.Cookie ~ "wordpress_logged" || req.http.Cookie ~ "comment_") {
         /* Not cacheable by default */
         	return (pass);
	}

	if (req.url ~ "/feed") {
		return (pass);
	}
	if (req.url ~ "wp-admin|wp-login") {
		return (pass);
	}

	if (req.url ~ "^(.*)\.(jpg|png|gif|jpeg|flv|bmp|gz|tgz|bz2|tbz|js|css|html|htm)($|\?)" ) { 
         #移除cookie,以便能缓存到varnish 
        unset req.http.cookie; 
	} 



	set req.http.cookie = regsuball(req.http.cookie, "wp-settings-\d+=[^;]+(; )?", "");
	set req.http.cookie = regsuball(req.http.cookie, "wp-settings-time-\d+=[^;]+(; )?", "");

	if (req.http.cookie == "") {
		unset req.http.cookie;
  	}


}


sub vcl_purge {
 	set req.method = "GET";
 	set req.http.X-Purger = "Purged";
	return (restart);
}


sub vcl_backend_response {


	set beresp.ttl = 24h;
 	set beresp.grace = 1h;
	if (bereq.url !~ "wp-admin|wp-login|product|cart|checkout|my-account|/?remove_item=") {
		unset beresp.http.set-cookie;
	}
}

sub vcl_deliver {

	unset resp.http.Via;
	unset resp.http.X-Varnish;


	if (obj.hits > 0) {
	set resp.http.X-Cache =  "HIT from " + server.hostname;
	} else {
	set resp.http.X-Cache =  "MISS from " + server.hostname;
	}

}


sub vcl_synth {


if (resp.status == 404) {

    set resp.http.Content-Type = "text/html; charset=utf-8";
    set resp.http.Retry-After = "5";
    synthetic( {"<!DOCTYPE html>
<html>
  <head>
    <title>"} + resp.status + " " + resp.reason + {"</title>
  </head>
  <body>
    <h1>Error "} + resp.status + " " + resp.reason + {"</h1>
    <p>"} + resp.reason + {"</p>
    <h3>Guru Meditation:</h3>
    <p>XID: "} + req.xid + {"</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>
"} );
}
    return (deliver);
}


Varnish运行配置文件:


### Varnish运行配置文件,如果Nginx负载均衡鸡和Varnish在同台机器,那么Varnish鸡运行配置文件不需要指定IP,默认是127.0.0.1:
ExecStart=/usr/sbin/varnishd -a :6081 -f /etc/varnish/default.vcl -s file,/tmp/varnish/cache.bin,5G


### Varnish运行配置文件,如果Nginx负载均衡鸡和Varnish不在同台机器,那么Varnish鸡需要指定运行配置文件的外网IP:
cat /usr/lib/systemd/system/varnish.service
ExecStart=/usr/sbin/varnishd -a 外网IP:6081 -f /etc/varnish/default.vcl -s file,/tmp/varnish/cache.bin,5G


源站Nginx配置文件:




cat /etc/nginx/conf.d/affdalao.conf
    server {
        listen 8081;
        server_name  affdalao.com;
	port_in_redirect off;



    location / {
        root   /home/affdalao;
        index   index.php index.html index.htm;
      try_files $uri $uri/ /index.php?$args;
   
    }


    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
#    location ~ \.php$ {
    location ~ .*\.php(\/.*)*$ {
        root           /home/affdalao;

#       try_files $uri =404;
#       fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        fastcgi_param HTTPS on;

	fastcgi_pass   unix:/dev/shm/php7-fpm.sock;
        fastcgi_param  SCRIPT_FILENAME  /home/affdalao$fastcgi_script_name;
        include        fastcgi_params;
    }

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}



检验Varnish集群是否生效,关闭任意一台Varnish机器,网站正常运行:
Varnish关闭命令:systemctl stop varnish

至此Varnish集群搭建完毕。

  • 版权声明:本文版权归AFF大佬和原作者所有,未经许可不得转载。文章部分来源于网络仅代表作者看法,如有不同观点,欢迎进行交流。除非注明,文章均由 AFF大佬 整理发布,欢迎转载,转载请带版权。

  • 来源:AFF大佬 ( https://www.affdalao.com/ ),提供主机优惠信息深度测评和服务器运维编程技术。
  • 链接:https://www.affdalao.com/635.html
  • 评论(2

    1. cad说道:

      缓存如何更新呢?

      • affdalao说道:

        三种解决方案:
        1、插件:Proxy Cache Purge
        2、手动重启varnish程序 (在用)
        3、等2H小时缓存过期

    cad进行回复 取消回复

    电子邮件地址不会被公开。 必填项已用*标注