[CentOS] Amazon S3FS Automounter of sorts

Tue Apr 13 13:29:11 UTC 2010
Matt Keating <Matt_Keating at dennis.co.uk>

> From: Geerd-Dietger Hoffmann <didi at ribalba.de>
> Reply-To: CentOS mailing list <centos at centos.org>
> Date: Tue, 13 Apr 2010 13:11:59 +0100
> To: CentOS mailing list <centos at centos.org>
> Subject: Re: [CentOS] Amazon S3FS Automounter of sorts
> 
> On 12/04/2010 17:44, Matt Keating wrote:
>> I found it works well with FTPing into the server and uploading to the
>> mounted bucket.
>> Reason its like this is that there are lots of different people who upload
>> throughout our company. It was much easier giving out the FTP details, which
>> were totally under our control (Username/Pass,Firewall,etc), rather than
>> giving out the S3 logins.
> 
> Sounds like a good reason :)
> 
> 
>>> I hope you are aware that everything you put on your s3 is publicly
>>> available if someone knows your bucket name.
>>> 
>> Yes, I am aware of that - its all being served on the net anyway.
>> If I remove the Pubic read only, will the files still be accessible via
>> cloudfront?
> 
> I hope not. And a little test confirms this. If you are serving it out
> anyway that is fine. I just had a client that had all his backup files
> publicly readable, because of this type of configuration error.
> 
>>> I tried mounting it like you, but just ran into too many problems,
>>> especially if you access files from many machines.
>>> 
>> What issues did you run into? As I haven't had any problems as of yet.
> 
> If it is a one way transfer it is fine. But if you modify files etc
> caching issues where horrible. Files overwritten etc ... But if you are
> just pushing stuff onto a server it should work.
>Thanks for reply, 
>
We never need to modify things, so once they are live - they stay that way.
So I guess my situation is ok then.

> For the backup I have used s3tools too. I have a little script that
> looks at what is in the bucket and what is in the local folder and then
> syncs them up. But I suppose that is what the fuse file system does :)
>
For keeping things in sync, I have a cron job that downloads the list of
files plus hashes from amazon and stores them in a db. I compare that list
to the latest list downloaded and put the new files into a new table. I have
another script running that checks the 'download' table and downoads the
files in there, once completed puts that file into the main list table.
It works fine - no intention on changing that.

> For your auto-mount script. Can't you mount it when someone logs on over
> ftp. And then if no one is logged on any more unmount it.
> 
> Cheers Didi
Never thought about writing my own automounter, will give it a go.

Thanks for the input.
MattNOTE: The information in this email is confidential and may be legally
privileged, unless stated to the contrary. If you are not the intended recipient, you must not read, use or disseminate that information.
Any opinions or comments are personal to the writer and do not represent the
official view of Dennis Publishing Ltd. If you have received this email and are not a named addressee, please contact itdirector at dennis.co.uk immediately by reply email and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person.
Although this email and any attachments are believed to be free of any virus, or other defects, it is the responsibility of the recipient to ensure that they are virus free and no responsibility is accepted by Dennis Publishing Ltd for any loss or damage arising from the receipt or use thereof.
Company registered in England No. 1138891 
Registered office: 30, Cleveland Street, London, W1T 4JD