{"id":15778983,"url":"https://github.com/v5tech/taobao-tfs-example","last_synced_at":"2025-10-11T11:46:35.259Z","repository":{"id":149461850,"uuid":"53304846","full_name":"v5tech/taobao-tfs-example","owner":"v5tech","description":"taobao tfs install and config","archived":false,"fork":false,"pushed_at":"2016-03-07T10:14:46.000Z","size":319,"stargazers_count":2,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-11T18:13:45.308Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/v5tech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2016-03-07T07:29:56.000Z","updated_at":"2016-12-02T06:18:14.000Z","dependencies_parsed_at":"2023-04-19T03:16:50.918Z","dependency_job_id":null,"html_url":"https://github.com/v5tech/taobao-tfs-example","commit_stats":null,"previous_names":["v5tech/taobao-tfs-example"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v5tech%2Ftaobao-tfs-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v5tech%2Ftaobao-tfs-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v5tech%2Ftaobao-tfs-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v5tech%2Ftaobao-tfs-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/v5tech","download_url":"https://codeload.github.com/v5tech/taobao-tfs-example/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246490921,"owners_count":20786080,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-04T18:01:52.750Z","updated_at":"2025-10-11T11:46:30.208Z","avatar_url":"https://github.com/v5tech.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# taobao-tfs-example\n\ntaobao tfs安装及配置\n\n\u003e 环境\nubuntu 12.04.5\ngcc 4.6.3\n\n### 1. 安装相关依赖\n\n```bash\nsudo apt-get install automake autoconf build-essential subversion uuid-dev libncurses5-dev libtool zlib1g-dev libreadline-dev\n```\n\n### 2. 设置TBLIB_ROOT环境变量\n\n```bash\nvim ~/.bash_profile\nexport TBLIB_ROOT=/home/ubuntu/lib\nsource ~/.bash_profile \n```\n\n### 3. 安装tb-common-utils\n\n```bash\nsvn checkout -r 18 http://code.taobao.org/svn/tb-common-utils/trunk/ tb-common-utils\ncd tb-common-utils\nchmod +x build.sh\n./build.sh\n```\n\n### 4. 安装tfs\n\n```bash\nsvn co http://code.taobao.org/svn/tfs/branches/dev_for_outer_users tfs\ncd tfs\n./build.sh init\n./configure --prefix=/home/ubuntu/apps/tfs --with-release --without-tcmalloc\nmake\nmake install \n\n```\n*注意:*\n\n–prefix 指定tfs安装路径，默认会被安装到~/tfs_bin目录。\n\n–with-release 指定按release版本的参数进行编译，如果不指定这个参数，则会按开发版本比较严格的参数编译，包含-Werror参数，所有的警告都会被当错误，在高版本gcc下会导致项目编译不过，很多开源用户反馈的编译问题都跟这个有关，因为gcc高版本对代码的检查越来越严格，淘宝内部使用的gcc版本是gcc-4.1.2。\n\n* make时uuid相关错误\n\n```\nsession_util.cpp:(.text+0x1e): undefined reference to `uuid_generate'\nsession_util.cpp:(.text+0x2d): undefined reference to `uuid_unparse'\n```\n\n这两个方法一个是zlib的，一个是uuid的。\n\n使用ldconfig检查链接库\n\n```bash\nldconfig -p|grep libz.so  \nldconfig -p|grep liuuid.so \n```\n\n经过测试，发现是gcc链接的时候，-lz -luuid动态链接选项需要放到gcc选项的最后面\n\n修改src/tools/nameserver/Makefile文件，在-lc末尾添加-lz -luuid\n\n修改src/tools/transfer/Makefile文件，在-lc末尾添加-lz -luuid\n\n```bash\nvim src/tools/nameserver/Makefile\nLIBS = -lrt -lpthread -lm -ldl -lc -lz -luuid\n```\n```bash\nvim src/tools/transfer/Makefile\nLIBS = -lrt -lpthread -lm -ldl -lc -lz -luuid\n```\n\n最终编译好的目录为\n\n```\ndrwxrwxr-x 2 ubuntu ubuntu 4096  3月  3 17:29 bin/\ndrwxrwxr-x 2 ubuntu ubuntu 4096  3月  3 17:29 conf/\ndrwxrwxr-x 2 ubuntu ubuntu 4096  3月  3 17:29 include/\ndrwxrwxr-x 2 ubuntu ubuntu 4096  3月  3 17:29 lib/\ndrwxrwxr-x 2 ubuntu ubuntu 4096  3月  3 17:29 logs/\ndrwxrwxr-x 3 ubuntu ubuntu 4096  3月  3 17:29 scripts/\ndrwxrwxr-x 4 ubuntu ubuntu 4096  3月  3 17:29 sql/\n```\n\nbin：包含tfs所有的可执行程序文件，如nameserver(NS)、dataserver(DS)、tfstool。\n\nconf：包含tfs的配置文件，如NS的配置文件ns.conf，DS的配置文件ds.conf。\n\ninclude：包含TFS客户端库相关的头文件，应用程序使用TFS需要包含这些头文件。\n\nlib： 包含TFS客户端的静/动态库，应用程序使用TFS需要连接libtfsclient。\n\nlogs：用于存储TFS运行过程中的日志。\n\nscript：包含tfs常用的一些运维脚本，如stfs用于格式化DS， tfs启动/停止NS、DS。\n\n### 5. 创建磁盘分区\n\n在虚拟机中新添加三个磁盘,分别为/dev/sdb,/dev/sdc,/dev/sdd\n\n```bash\nsudo fdisk /dev/sdb\n命令(输入 m 获取帮助)： o\n命令(输入 m 获取帮助)： n\nPartition type:\n   p   primary (0 primary, 0 extended, 4 free)\n   e   extended\nSelect (default p): p\n分区号 (1-4，默认为 1)： 1\n起始 sector (2048-20971519，默认为 2048)： \n将使用默认值 2048\nLast sector, +扇区 or +size{K,M,G} (2048-20971519，默认为 20971519)： \n将使用默认值 20971519\n命令(输入 m 获取帮助)： w\nThe partition table has been altered!\nsudo mkfs.ext4 /dev/sdb1\n```\n\n```bash\nsudo fdisk /dev/sdc\n命令(输入 m 获取帮助)： o\n命令(输入 m 获取帮助)： n\nPartition type:\n   p   primary (0 primary, 0 extended, 4 free)\n   e   extended\nSelect (default p): p\n分区号 (1-4，默认为 1)： 1\n起始 sector (2048-20971519，默认为 2048)： \n将使用默认值 2048\nLast sector, +扇区 or +size{K,M,G} (2048-20971519，默认为 20971519)： \n将使用默认值 20971519\n命令(输入 m 获取帮助)： w\nThe partition table has been altered!\nsudo mkfs.ext4 /dev/sdc1\n```\n\n```bash\nsudo fdisk /dev/sdd\n命令(输入 m 获取帮助)： o\n命令(输入 m 获取帮助)： n\nPartition type:\n   p   primary (0 primary, 0 extended, 4 free)\n   e   extended\nSelect (default p): p\n分区号 (1-4，默认为 1)： 1\n起始 sector (2048-20971519，默认为 2048)： \n将使用默认值 2048\nLast sector, +扇区 or +size{K,M,G} (2048-20971519，默认为 20971519)： \n将使用默认值 20971519\n命令(输入 m 获取帮助)： w\nThe partition table has been altered!\nsudo mkfs.ext4 /dev/sdd1\n```\n\n具体操作参阅http://askubuntu.com/questions/154180/how-to-mount-a-new-drive-on-startup\n\n修改/etc/fstab文件实现开机自动挂载\n\n```bash\nsudo vim /etc/fstab\n/dev/sdb1       /home/ubuntu/data/disk1       ext4    defaults    0    1\n/dev/sdc1       /home/ubuntu/data/disk2       ext4    defaults    0    1\n/dev/sdd1       /home/ubuntu/data/disk3       ext4    defaults    0    1\n```\n\n### 6. 挂载硬盘\n\n创建挂载点\n\n```bash\nubuntu@s1:~/data$ mkdir disk1 disk2 disk3\nubuntu@s1:~/data$ ls\ndisk1  disk2  disk3\n```\n\n挂载磁盘\n\n```bash\nubuntu@s1:~/data$ sudo mount /dev/sdb1 /home/ubuntu/data/disk1\nubuntu@s1:~/data$ sudo mount /dev/sdc1 /home/ubuntu/data/disk2\nubuntu@s1:~/data$ sudo mount /dev/sdd1 /home/ubuntu/data/disk3\n```\n\n查看磁盘\n\n```bash\nubuntu@s1:~/data$ sudo df -h\n文件系统        容量  已用  可用 已用% 挂载点\n/dev/sda1        18G  3.4G   14G   21% /\nudev            2.0G  4.0K  2.0G    1% /dev\ntmpfs           394M  788K  394M    1% /run\nnone            5.0M     0  5.0M    0% /run/lock\nnone            2.0G     0  2.0G    0% /run/shm\noverflow        1.0M  4.0K 1020K    1% /tmp\n/dev/sdb1       9.8G   23M  9.2G    1% /home/ubuntu/data/disk1\n/dev/sdc1       9.8G   23M  9.2G    1% /home/ubuntu/data/disk2\n/dev/sdd1       9.8G   23M  9.2G    1% /home/ubuntu/data/disk3\n```\n\n取消挂载\n\n```bash\nubuntu@s1:~/data/disk$ sudo umount /home/ubuntu/data/disk1\nubuntu@s1:~/data/disk$ sudo umount /home/ubuntu/data/disk2\nubuntu@s1:~/data/disk$ sudo umount /home/ubuntu/data/disk3\n```\n\n### 7. 修改tfs/conf/ns.conf和tfs/conf/ds.conf\n\nns.conf\n\n```\n[public]\n\n#日志文件的级别, default info，上线使用建议设置为INFO，调试设为DEBUG\nlog_level=info\n\n#监听端口\nport = 8100\n\n#工作目录\nwork_dir= /home/ubuntu/apps/tfs\n\n#网络设备\ndev_name= eth0\n\n#本机IP地址(vip)，配置ha时为vip，没配置可以为主ns的ip\nip_addr = 192.168.64.136\n\n[nameserver]\n\n#nameserver IP地址列表(master, salve的ip地址，只能以'|'分隔)\n#单台nameserver时，另一个ip配置为无效ip即可\nip_addr_list = 192.168.64.136|192.168.0.2\n\n#用于区分dataserver所在的子网，选择不同子网的dataserver备份数据\ngroup_mask = 255.255.255.255\n\n#Block 最大备份数, default: 2\n#单台dataserver时，需要配置为1\nmax_replication = 1\n\n#Block 最小备份数, default: 2\n#单台dataserver时，需要配置为1\nmin_replication = 1\n```\nds.conf\n\n```\n[public]\n\n#日志文件的级别, default info，线上使用建议设为info，调试设为debug \nlog_level=info\n\n#监听端口\nport = 8200 \n\n#工作目录\nwork_dir= /home/ubuntu/apps/tfs\n\n#网络设备\ndev_name= eth0\n\n#本机IP地址\nip_addr = 192.168.64.136\n\n[dataserver]\n\n#nameserver ip地址\nip_addr = 192.168.64.136\n\n#nameserver ip地址列表(master, salve的ip地址，只能以'|'分隔)\nip_addr_list = 192.168.64.136|192.168.0.2 \n\n#nameserver 监听端口\nport = 8100\n\nblock_max_size = 8625464 \n\n#备件类型, 1: tfs, 2: nfs\nbackup_type = 1\n\n#备件路径\n#backup_path = /home/admin/tfs\n\n#mount路径\nmount_name = /home/ubuntu/data/disk\n\n#mount 时磁盘的大小, 单位(KB)，不要要过文件系统实际剩余空间\nmount_maxsize = 8625464\n\n#主块的大小, 单位(字节)\nmainblock_size = 8625464 \n\n#扩展块的大小, 单位(字节) \nextblock_size = 4194304\n```\n\n具体配置参考\n\nhttp://code.taobao.org/p/tfs/wiki/deploy/\nhttp://code.taobao.org/p/tfs/wiki/deploy/ns.conf/\nhttp://code.taobao.org/p/tfs/wiki/deploy/ds.conf/\n\n### 8. 启动nameserver\n\n```bash\nubuntu@s1:~/apps/tfs/scripts$ ./tfs  start_ns\n nameserver is up SUCCESSFULLY pid: 4249 \n```\n\n### 9. 查看nameserver\n\n```bash\nubuntu@s1:~/apps/tfs/scripts$ netstat -ntpl |grep name\n（并非所有进程都能被检测到，所有非本用户的进程信息将不会显示，如果想看到所有信息，则必须切换到 root 用户）\nProto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name\ntcp        0      0 0.0.0.0:8100            0.0.0.0:*               LISTEN      4249/nameserver\n```\n\n### 10. 格式化dataserver\n\n```bash\nubuntu@s1:~/apps/tfs/scripts$ ./stfs format 1-3\n```\n\n### 11. 启动dataserver\n\n```bash\nubuntu@s1:~/apps/tfs/scripts$ ./tfs start_ds 1-3\n dataserver 1 is up SUCCESSFULLY pid: 5852 \n dataserver 2 is up SUCCESSFULLY pid: 5904 \n dataserver 3 is up SUCCESSFULLY pid: 5956\n```\n\n### 12. 查看dataserver\n\n```bash\nubuntu@s1:~/apps/tfs/scripts$ netstat -ntpl |grep data\n（并非所有进程都能被检测到，所有非本用户的进程信息将不会显示，如果想看到所有信息，则必须切换到 root 用户）\ntcp        0      0 0.0.0.0:8205            0.0.0.0:*               LISTEN      5956/dataserver \ntcp        0      0 0.0.0.0:8200            0.0.0.0:*               LISTEN      5852/dataserver \ntcp        0      0 0.0.0.0:8201            0.0.0.0:*               LISTEN      5852/dataserver \ntcp        0      0 0.0.0.0:8202            0.0.0.0:*               LISTEN      5904/dataserver \ntcp        0      0 0.0.0.0:8203            0.0.0.0:*               LISTEN      5904/dataserver \ntcp        0      0 0.0.0.0:8204            0.0.0.0:*               LISTEN      5956/dataserver \n```\n\n### 13. tfs client java编译\n\n```bash\nsvn co http://code.taobao.org/svn/tfs-client-java/tags/tfs-client-java-2.1.3/  tfs-client-java\n```\n下载http://code.taobao.org/p/tair-client-java/file/40/tair-client-2.3.1.jar\n\n修改pom.xml文件\n\n删除`\u003cparent\u003e\u003c/parent\u003e`块\n\n修改pom.xml中的依赖为\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.taobao.common.tair\u003c/groupId\u003e\n  \u003cartifactId\u003ecommon-tair\u003c/artifactId\u003e\n  \u003cversion\u003e2.3.1\u003c/version\u003e\n  \u003cscope\u003esystem\u003c/scope\u003e\n  \u003csystemPath\u003e${basedir}/tair-client-2.3.1.jar\u003c/systemPath\u003e \n\u003c/dependency\u003e\n```\nhttp://code.taobao.org/p/tair-client-java/wiki/index/\n\n```bash\ncd tfs-client-java\nmvn package\n```\n或者直接下载http://code.taobao.org/p/tfs-client-java/file/404/tfs-javaclient-2.1.1.jar\n\ntfs-client-java使用具体见https://github.com/sxyx2008/taobao-tfs-example\n\n### 13. 参考文档\n\nhttp://code.taobao.org/p/tfs/wiki/index\n\nhttp://blog.yunnotes.net/index.php/install_document_for_tfs\n\nhttp://www.qinglin.net/25.html\n\nhttp://ylw6006.blog.51cto.com/470441/d-18\n\nhttp://www.cnblogs.com/starlitnext/p/4132069.html\n\nhttp://code.taobao.org/p/tfs/issue/1835\n\nhttp://www.cnphp6.com/archives/20772\n\nhttp://askubuntu.com/questions/154180/how-to-mount-a-new-drive-on-startup\n\nhttp://code.taobao.org/p/tfs/issue/250/\n\nhttp://xwsoul.com/posts/548 UBUNTU 12.04下编译安装 GCC4.1.2\n\nhttp://code.taobao.org/p/tfs/wiki/jclient/ tfs-client-java wiki","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fv5tech%2Ftaobao-tfs-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fv5tech%2Ftaobao-tfs-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fv5tech%2Ftaobao-tfs-example/lists"}