WOSGateway

WOSGateway (Storage Gateway) is a high-performance gateway tool developed by CDNetworks, that offers support for caching functionality, NFS sharing, and SMB sharing. It enables you to mount Object Storage Service (OSS) buckets as if they were local file systems on Linux. This allows you to handle OSS objects using the same approach you use for managing local files.

Please be aware that using WOSGateway necessitates having over 100GB of storage space available on your server, which will be utilized as a cache disk.

Installation (CentOS Environment)

1. Begin by downloading the installation package and extracting its contents to a designated local directory. Find the Download link here.
2. Unpack the wosgw-*.zip package.
3. Navigate into the wosgw-* directory using the command: cd wosgw-, and proceed with the installation using: rpm -ivh *.rpm.
4. If you encounter any dependency errors, address them by executing: yum -y install fuse fuse-libs libibverbs librdmacm gperftools-libs.
5. In case you receive an error related to missing leveldb dependencies, you can install it using the following steps:

wget http://download-ib01.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-13.noarch.rpm
rpm -Uvh epel-release-7-13.noarch.rpm
yum -y install leveldb

6. Once the installation is successful, you’ll find the following files and directories created:

• Binary path: /usr/bin/wosgw
• Configuration file: /etc/wosgw/wosgw.conf
• Log path: /var/log/wosgw
• Runtime directory: /var/run/wosgw
• Database path: /var/run/wosgw/db

Configurations

1. Revise configuration file: /etc/wosgw/wosgw_ctl.toml

[wosgw_ctl]

# Access key credentials
AK = "your ak"
SK = "your sk"

# Endpoint adress
server = "http://endpoint"

# Bucket name
bucket = "regionName"

# Local mount directory
mount_dir = "/mnt/xxx"

# Local cache directory
use_cache = "/cache5/s3fs_cache"

conf_path = "/etc/wosgw/wosgw.conf"
client = "client.wosgw"

[params]
args = [
	"-o enable_noobj_cache",
	"-o ensure_diskfree=300",
	"-o use_xattr",
	"-o allow_other",
	"-o umask=000,noatime",
	"-o force_purge_rocksdb_stat",
	"-o use_path_request_style",
	"-o curldbg",
	"-o dbglevel=dbg",
]

Instructions

2. Include the configuration file /etc/wosgw/wosgw.conf and adjust the following details as required. (Note: If none of the settings require customization, you can choose not to add this configuration file.)

wosgw_run_mode=async
debug_wosgw=60
wosgw_rgw_host_regex = ".*"

wosgw_cachefile_life_time=1
wosgw_cachefile_remove_size=200
wosgw_cachefile_remove_interval=2
wosgw_cachefile_remove_thread_num=2
wosgw_cachefile_quantity_watermark_high=0
wosgw_cachefile_quantity_watermark_low=0
wosgw_cachefile_gc_thread_num=3

wosgw_db_path=/cache2/wosgw-db

wosgw_stat_mode=1

wosgw_sync_log_interval=10
wosgw_num_shards=5
wosgw_sync_sleep_time=0

wosgw_resume_upload=True

wosgw_vid_prealloc=100

[client.wosgw.default]
log file=/cache2/wosgw-log/$name.log
admin socket = /usr/local/etc/wosgw/$name.asok

Wosgw Configuration Instructions

How to use

1. Start

wosgw_ctl -action start

2. Stop

wosgw_ctl -action stop

Goofys

Goofys is an open-source tool that enables the mounting of S3 storage buckets onto a local file system.

Installation Steps (using CentOS as an example)

1. Download goofys using: wget https://github.com/kahing/goofys/releases/latest/download/goofys.
2. Provide execution permission to the goofys file: chmod 755 goofys.
3. Add your credentials to a certificate file:

vim ~/.aws/credentials
[default]
aws_access_key_id = AKID1234567890
aws_secret_access_key = MY-SECRET-KEY

4. Start the mounting command:

./goofys --endpoint http://s3-cn-south-6.wcsapi.com --region cn-south-6 yourBucketName /mnt/mount -o allow_other

5. Verify the successful mount:

mount
> bucket on /mnt/mount type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions)

6. The logs are by default printed to syslog. You can refer to the configuration of syslog for different systems. For example, the default syslog for CentOS is located at: /var/log/message; Ubuntu systems can execute the dmesg command to output system logs.