By

Contents
  1. 1. FastDFS 简介
    1. 1.1. github软件仓库
  2. 2. 安装
    1. 2.1. 下载安装文件
    2. 2.2. 安装FastDFS
  3. 3. FastDFS配置
    1. 3.1. 配置Tracker跟踪器
      1. 3.1.1. 开放tracker监听端口访问
      2. 3.1.2. 修改配置文件
      3. 3.1.3. 设置开机自启动
      4. 3.1.4. 确认tracker是否启动成功
      5. 3.1.5. 安装并配置 nginx 反向代理
    2. 3.2. 配置Storage存储节点
      1. 3.2.1. 开放tracker监听端口访问
      2. 3.2.2. 修改配置文件
      3. 3.2.3. 设置开机自启动
      4. 3.2.4. 确认storage是否启动成功
    3. 3.3. client客户端配置
    4. 3.4. FastDFS测试
  4. 4. FastDFS客户端
    1. 4.1. PHP扩展安装
    2. 4.2. PHP扩展使用
    3. 4.3. PHP客户端上传图片测试
  5. 5. fastdfs-nginx-module 安装配置
    1. 5.1. storage server 安装 nginx 服务器
    2. 5.2. 防火墙开放http服务
    3. 5.3. 安装并启用nginx
    4. 5.4. 查看前面安装的nginx编译参数
    5. 5.5. 下载nginx源码
    6. 5.6. 安装nginx依赖的软件包
    7. 5.7. 重新编译nginx
    8. 5.8. fastdfs模块配置
    9. 5.9. Storage服务器nginx配置
    10. 5.10. 重启Nginx

FastDFS 简介

FastDFS是一个国产开源的轻量级分布式文件系统，它对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务，如相册网站、视频网站等等。FastDFS服务端有两个角色：跟踪器（tracker）和存储节点（storage）。跟踪器主要做调度工作，在访问上起负载均衡的作用。

官方论坛： http://www.csource.org
下载地址: http://sourceforge.net/projects/fastdfs/files/

github软件仓库

安装

本安装使用 CentOS 7 x86_64版操作系统，按照以下网络结构进行部署：
FastDFS网络结构

下载安装文件

FastDFS 5.x 取消了对 libevent 的依赖，添加了对 libfastcommon 的依赖。
本部署说明用到的软件版本：

  • libfastcommon v1.13
  • FastDFS v5.06
  • fastdfs-nginx-module v1.17
    可从上面的sourceforge或github中下载，或者直接下载本文附带的压缩包。详细的安装说明可参照代码中的INSTALL。

安装FastDFS

在每一台tracker和storage服务器上执行

1
2
3
4
[root@localhost ~]# tar xjvf fdfs-5.06.tar.bz2
[root@localhost ~]# yum install -y gcc perl   # 安装依赖的软件包
[root@localhost ~]# cd ~/fdfs/libfastcommon && ./make.sh && ./make.sh install
[root@localhost ~]# cd ~/fdfs/fastdfs && ./make.sh && ./make.sh install

安装完成后，所有可执行文件在目录/usr/bin下，以fdfs_开头：

1
2
3
4
5
6
7
8
[root@localhost ~]# ls /usr/bin/fdfs_*
/usr/bin/fdfs_appender_test   /usr/bin/fdfs_monitor
/usr/bin/fdfs_appender_test1  /usr/bin/fdfs_storaged
/usr/bin/fdfs_append_file     /usr/bin/fdfs_test
/usr/bin/fdfs_crc32           /usr/bin/fdfs_test1
/usr/bin/fdfs_delete_file     /usr/bin/fdfs_trackerd
/usr/bin/fdfs_download_file   /usr/bin/fdfs_upload_appender
/usr/bin/fdfs_file_info       /usr/bin/fdfs_upload_file

配置文件在目录/etc/fdfs下：

1
2
[root@localhost ~]# ls /etc/fdfs
client.conf.sample  storage.conf.sample  tracker.conf.sample

FastDFS配置

配置Tracker跟踪器

开放tracker监听端口访问

1
2
firewall-cmd --zone=public --add-port=22122/tcp --permanent
firewall-cmd --reload

修改配置文件

1
2
3
4
5
6
7
mkdir -p /data/fastdfs
cd /etc/fdfs
cp tracker.conf.sample tracker.conf
cp /root/fdfs/fastdfs/conf/http.conf .
cp /root/fdfs/fastdfs/conf/mime.types .
sed -i 's:base_path=.*:base_path=/data/fastdfs:g' tracker.conf
sed -i 's:http.server_port=.*:http.server_port=80:g' tracker.conf

设置开机自启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
bash -c 'cat > /usr/lib/systemd/system/fdfs_trackerd.service << EOF
[Unit]
Description=fastdfs tracker server
After=network.target
[Service]
Type=forking
PIDFile=/data/fastdfs/data/fdfs_trackerd.pid
ExecStart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf
ExecReload=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
ExecStop=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop
[Install]
WantedBy=multi-user.target
EOF'
systemctl enable fdfs_trackerd.service
systemctl start fdfs_trackerd.service

确认tracker是否启动成功

1
cat /data/fastdfs/logs/trackerd.log

安装并配置 nginx 反向代理

  • 防火墙开放http服务

    1
    2
    firewall-cmd --permanent --zone=public --add-service=http
    firewall-cmd --reload

  • 安装并启用nginx

    1
    2
    3
    4
    yum install -y epel-release    # 安装 EPEL 软件仓库
    yum install -y nginx
    systemctl enable nginx
    systemctl start nginx

  • 配置反向代理
    打开 /etc/nginx/nginx.conf，在 http {} 中添加：

    1
    2
    3
    4
    upstream fdfs {
        server   192.168.71.127:80;
        server   192.168.71.128:80;
    }

在 server{} 中添加：

1
2
3
location /M00 {
    proxy_pass http://fdfs;
}

配置Storage存储节点

开放tracker监听端口访问

1
2
firewall-cmd --zone=public --add-port=23000/tcp --permanent
firewall-cmd --reload

修改配置文件

1
2
3
4
5
6
7
8
9
mkdir -p /data/fastdfs
cd /etc/fdfs
cp storage.conf.sample storage.conf
cp /root/fastdfs/fastdfs/conf/http.conf .
cp /root/fastdfs/fastdfs/conf/mime.types .
sed -i 's:base_path=.*:base_path=/data/fastdfs:g' storage.conf
sed -i 's:store_path0=.*:store_path0=/data/fastdfs:g' storage.conf
sed -i 's/tracker_server=.*/tracker_server=192.168.71.126:22122/g' storage.conf
sed -i 's:http.server_port=.*:http.server_port=80:g' storage.conf

设置开机自启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
bash -c 'cat > /usr/lib/systemd/system/fdfs_storaged.service << EOF
[Unit]
Description=fastdfs storage server
After=network.target
[Service]
Type=forking
PIDFile=/data/fastdfs/data/fdfs_storaged.pid
ExecStart=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf
ExecReload=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
ExecStop=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf stop
[Install]
WantedBy=multi-user.target
EOF'
systemctl enable fdfs_storaged.service
systemctl start fdfs_storaged.service

确认storage是否启动成功

1
cat /data/fastdfs/logs/storaged.log

client客户端配置

在tracker, storage之外的一台主机上安装FastDFS，然后执行：

1
2
3
4
5
mkdir -p /data/fastdfs
cd /etc/fdfs
cp client.conf.sample client.conf
sed -i 's:base_path=.*:base_path=/data/fastdfs:g' client.conf
sed -i 's/tracker_server=.*/tracker_server=192.168.71.126:22122/g' client.conf

FastDFS测试

  • 上传测试:

    1
    2
    joelhy@arminix: ~ $ fdfs_upload_file /etc/fdfs/client.conf pom.xml
    group1/M00/00/00/wKhHf1S-oryAZCpgAAAE2uRlJkA126.xml

  • 查看文件信息:

    1
    2
    3
    4
    5
    6
    7
    joelhy@arminix: ~ $ fdfs_file_info /etc/fdfs/client.conf
    group1/M00/00/00/wKhHf1S-oryAZCpgAAAE2uRlJkA126.xml
    source storage id: 0
    source ip address: 192.168.71.127
    file create timestamp: 2015-01-26 02:47:24
    file size: 1242
    file crc32: 3831834176 (0xE4652640)

  • 下载测试:

    1
    2
    3
    4
    joelhy@arminix: ~ $ fdfs_download_file /etc/fdfs/client.conf \
        group1/M00/00/00/wKhHf1S-oryAZCpgAAAE2uRlJkA126.xml downtest.xml
    joelhy@arminix: ~ $ ls
    downtest.xml

FastDFS客户端

PHP扩展安装

client主机上安装的是php v5.4.33

1
2
3
4
5
6
cd ~/fdfs/FastDFS/php_client
/usr/local/php-5.4.33/bin/phpize
./configure  --with-php-config=/usr/local/php-5.4.33/bin/php-config
make && make install
mkdir /usr/local/php-5.4.33/etc/conf.d
cp fastdfs_client.ini /usr/local/php-5.4.33/etc/conf.d

PHP扩展使用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
<?php
// fastdfs.php
class Fdfs {
    private $fdfs, $tracker, $storage;
    public function __construct() {
        $this->fdfs = new FastDFS();
        // get a connected tracker server
        $this->tracker = $this->fdfs->tracker_get_connection();
        if (!$this->tracker) {
            throw new Exception('cannot connect to tracker server:[' .  
                $this->fdfs->get_last_error_no() . '] ' . 
                $this->fdfs->get_last_error_info());
        }
        // get the storage server info and connect to it
        $this->storage = $this->fdfs->tracker_query_storage_store();
        $this->server = $this->fdfs->connect_server(
            $this->storage['ip_addr'], $this->storage['port']);
        if ($this->server === false) {
            throw new Exception('cannot connect to storage server' . 
                $this->storage['ip_addr'] . ':' . 
                $this->storage['port'] .  ' :[' .  
                $this->fdfs->get_last);
        }
        $this->storage['sock'] = $this->server['sock'];
    }
   
    public function upload($localfile, $ext_name) {
        //$info = $this->fdfs->storage_upload_by_filename($localfile);
        $info = $this->fdfs->storage_upload_by_filename($localfile, $ext_name, 
            array(), null, $this->tracker, $this->storage);
        if (is_array($info)) {
            $group_name = $info['group_name'];
            $remote_filename = $info['filename'];
            $source_info = $this->fdfs->get_file_info($group_name, 
                $remote_filename);
            $source_ip = $source_info['source_ip_addr'];
            $file_size = $source_info['file_size'];
            return compact('group_name', 'remote_filename', 
                'source_ip', 'file_size');
        }
        return false;
    }
    public function download_to_buff($group_name, $remote_filename) {
        $content = $this->fdfs->storage_download_file_to_buff(
            $group_name, $remote_filename);
        return $content;
    }
    public function download_to_file($group_name, 
            $remote_filename, $dst_localfile) {
        return $this->fdfs->storage_download_file_to_file($group_name, 
            $remote_filename, $dst_localfile);
    }
    public function delete($group_name, $remote_filename) {
        return $this->fdfs->storage_delete_file($group_name, $remote_filename);
    }
    public function exists($group_name, $remote_filename) {
        return $this->fdfs->storage_file_exist($group_name, $remote_filename);
    }
    public function get_file_info($group_name, $remote_filename) {
        return $this->fdfs->get_file_info($group_name, $remote_filename);
    }
}

PHP客户端上传图片测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# cat test.php
<?php
require_once('fastdfs.php');
$fdfs = new FDFS();
$localfile = './test.xml';
$fileinfo = $fdfs->upload($localfile);
if ($fileinfo) {
  // update file info in the database etc
}
var_dump($fileinfo);
1
2
3
4
5
6
7
8
9
10
11
# php test.php 
array(4) {
  ["group_name"]=>
  string(6) "group1"
  ["remote_filename"]=>
  string(44) "M00/00/00/wKhHf1S-qbuAISbeAAAE2uRlJkA789.xml"
  ["source_ip"]=>
  string(14) "192.168.71.127"
  ["file_size"]=>
  int(1242)
}

fastdfs-nginx-module 安装配置

fastdfs-nginx-module用于解决同步延迟问题：

同组之间的服务器需要复制文件,有延迟的问题.
假设Tracker服务器将文件上传到了192.168.1.80,文件ID已经返回客户端,
这时,后台会将这个文件复制到192.168.1.30,如果复制没有完成,客户端就用这个ID在192.168.1.30取文件,肯定会出现错误
这个fastdfs-nginx-module可以重定向连接到源服务器取文件,避免客户端由于复制延迟的问题,出现错误。

fastdfs-nginx-module模块只需要安装到storage上。

storage server 安装 nginx 服务器

防火墙开放http服务

1
2
firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --reload

安装并启用nginx

1
2
3
4
yum install -y epel-release    # 安装 EPEL 软件仓库
yum install -y nginx
systemctl enable nginx
systemctl start nginx

查看前面安装的nginx编译参数

1
2
3
4
5
[root@localhost ~]# nginx -V
nginx version: nginx/1.6.2
built by gcc 4.8.2 20140120 (Red Hat 4.8.2-16) (GCC) 
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_spdy_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-mail_ssl_module --with-pcre --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

下载nginx源码

1
curl -O http://nginx.org/download/nginx-1.6.2.tar.gz

安装nginx依赖的软件包

1
2
3
yum install -y redhat-rpm-config pcre-devel openssl-devel libxml2-devel \
    libxslt-devel gd-devel perl-ExtUtils-Embed GeoIP-devel gperftools \
    gperftools-devel

重新编译nginx

1
2
3
4
5
6
tar xvf nginx-1.6.2.tar.gz
cd nginx-1.6.2
./configure --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_spdy_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-mail_ssl_module --with-pcre --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' --add-module=/root/fastdfs/fastdfs-nginx-module/src
mv /usr/sbin/nginx /usr/sbin/nginx.`date +'%Y%m%d'`
cp objs/nginx /usr/sbin/nginx

fastdfs模块配置

1
2
3
4
cp /root/fastdfs/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/
touch /data/fastdfs/logs/mod_fastdfs.log
chown nginx:nginx /data/fastdfs/logs/mod_fastdfs.log
vi /etc/fdfs/mod_fastdfs.conf

修改:

1
2
3
4
base_path=/data/fastdfs
tracker_server=192.168.71.126:22122
store_path0=/data/fastdfs
log_filename=/data/fastdfs/logs/mod_fastdfs.log

Storage服务器nginx配置

server{} 里添加

1
2
3
4
location /M00 {
   alias /data/fastdfs/data;
   ngx_fastdfs_module;
}

重启Nginx

1
2
systemctl stop nginx
systemctl start nginx
FastDFS
Contents
  1. 1. FastDFS 简介
    1. 1.1. github软件仓库
  2. 2. 安装
    1. 2.1. 下载安装文件
    2. 2.2. 安装FastDFS
  3. 3. FastDFS配置
    1. 3.1. 配置Tracker跟踪器
      1. 3.1.1. 开放tracker监听端口访问
      2. 3.1.2. 修改配置文件
      3. 3.1.3. 设置开机自启动
      4. 3.1.4. 确认tracker是否启动成功
      5. 3.1.5. 安装并配置 nginx 反向代理
    2. 3.2. 配置Storage存储节点
      1. 3.2.1. 开放tracker监听端口访问
      2. 3.2.2. 修改配置文件
      3. 3.2.3. 设置开机自启动
      4. 3.2.4. 确认storage是否启动成功
    3. 3.3. client客户端配置
    4. 3.4. FastDFS测试
  4. 4. FastDFS客户端
    1. 4.1. PHP扩展安装
    2. 4.2. PHP扩展使用
    3. 4.3. PHP客户端上传图片测试
  5. 5. fastdfs-nginx-module 安装配置
    1. 5.1. storage server 安装 nginx 服务器
    2. 5.2. 防火墙开放http服务
    3. 5.3. 安装并启用nginx
    4. 5.4. 查看前面安装的nginx编译参数
    5. 5.5. 下载nginx源码
    6. 5.6. 安装nginx依赖的软件包
    7. 5.7. 重新编译nginx
    8. 5.8. fastdfs模块配置
    9. 5.9. Storage服务器nginx配置
    10. 5.10. 重启Nginx