Monthly Archives: June 2012

mounting s3 bucket as Linux file system partition

You can mount amazon s3 bucket as linux partition for example mounting the bucket (my-s3-bucket) as /mnt/my-s3-bucket in your linux server (ec2 or even any other linux OS) by using s3fs and fuse.

and today I will let you know how to do that:

first of all you will need to make sure from the requirements software are installed by run the following commands:


# yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap

now we must install fuse and make sure it installed correctly by doing the following commands: (run the command as root)

before installing fuse you can check if the required fuse version are installed or not (s3fs require fuse version  2.8.4 and above) by running the comand


# rpm -qa | grep fuse
# pkg-config --modversion fuse

if the version of fuze is less than 2.8.4 you must remove fuse and then install it , you can remove fuse by


# yum remove fuse fuse* fuse-devel

then install fuse by running the commands:


# cd /usr/local/src
# wget http://sourceforge.net/projects/fuse/files/fuse-2.X/2.9.0/fuse-2.9.0.tar.gz/download
# tar zxvf fuse-2.9.0.tar.gz
# mv fuse-2.9.0 fuse
# cd fuse
# ./configure --prefix=/usr
# make
# make install
# export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
# ldconfig
# modprobe fuse
# pkg-config --modversion fuse

when running the command pkg-config –modversion fuse confirm that 2.9.0 is the version displayed or it will display a version above 2.8.4.

Then we must continute installing the software by running the following commands (as root)

# cd /usr/local/src/
# wget http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz
# tar zxvf s3fs-1.61.tar.gz
# mv s3fs-1.61 s3fs
# ./configure --prefix=/usr
# make
# make install

After that s3fs will be installed on your linux and then we will try to use it by mounting s3 backut into this linux server

Now we must create a new s3 bucket or use any exists bucket (let says that its name is my-s3-bucket)

I will mount the s3 bucket (my-s3-bucket) into (/mnt/my-s3-bucket)

now i will configure the s3 user in the file ~/.passwd-s3fs (if you are root the password file will be /root/.passwd-s3fs)

so I will create the file .passwd-s3fs by do the following command


touch /root/.passwd-s3fs

and then chmod the .passwd-s3fs to 600 by


chmod 600 /root/.passwd-s3fs

now we will edit the file .passwd-s3fs and add the user info which we can have from amazon portal as (Access Key Id,Secret Access Key) as the follwing


edit /root/.passwd-s3fs

and add the user and the password as


my-s3-bucket:accessKeyId:secretAccessKey

where my-s3-bucket is the s3 bucket name and the accessKeyId is the user access id and secretAccessKey is the user secret key. and save the file

now i will make the mount point folder by running the command


mkdir /mnt/my-s3-bucket

now i will mount the s3 bucket by running the following command:


/usr/bin/s3fs my-s3-bucket /mnt/my-s3-bucket

now you can use the folder /mnt/my-s3-bucket as a linux partition while any files in this partition will directly saved to s3

you can refer for s3fs manual page to see more options.

references:

Advertisements