The Big Fat Open Directory in the Sky

My boyfriend, Robert, is very smart and very creative (and, you know, sexy) and makes very beautiful stuff out of pixels and sound. He uses Processing to program up these amazing interactive screen toys that respond to anything they hear, and you can use your keyboard to change the way they interact with those sounds, too.
Lately, he’s been building very complicated and extremely processor-intensive media toys that the average — or even the above-average — computer has a hard time dealing with in real time, so he sets them up to render overnight and then he creates a Quicktime movie out of the results and has been posting these rather large and rather beautiful creations to his blog, and everything was going along fine and dandy.
Then, suddenly, everyone started to discover what he was doing and wanted to see his art first-hand, so one of his fat bandwidth creations got blogged and delicious’d and linked to from all sorts of places, and all sorts of people were downloading the movies and he was very, very happy.
Until he received his bandwidth bill from his not-so-understanding ISP and discovered how much popularity costs in this new video-centric Web world in which we live.
Luckily for him, there’s an answer that all of us can use right now, and it’s not a Flash-based video site that compresses your beautiful movies to the point that you can’t tell your daughter from your dog.


The answer is really, really simple. In fact, that’s what it’s called, and it’s provided for you by your favorite supserstore, Amazon. It’s called Amazon S3 for Simple Storage Solution and if you have an Amazon account, you can have an Amazon S3 account in a matter of minutes, and start to upload your multi-megabyte videos and MP3 podcasts and family portraiture immediately.
The advantages of using Amazon S3 are numerous.

  1. You get unlimited storage space. Unlimited.
  2. The size limit of any single uploaded file is 5Gb. That’s Gigs.
  3. You only pay for what you use, and the rates are very affordable. (15¢ per Gig of storage per month, and 20¢ per Gig of data transferred per month, all charged to the same credit card you already use at Amazon)
  4. You can specify which of the files you upload are public and which ones are private.
  5. You can access your stuff from anywhere you can access the web.

Here’s what you’ll need to make the whole thing simple and easy:

And does it really work? Of course it does!

The Firefox browser and plug-in are very simple to get and install. The AWS account is easy to get, but getting the keys you need to access your S3 account are a bit confusing. The service is intended for developers and offers an application programming interface (API) to tie in S3 to web applications and several services are popping up that use S3 to do things like back-up your files for you, or store your iTunes library and so on. Of course, you can do it all yourself, too, with a little effort.
You sign in to your Amazon account using the same login and password you use to buy things, and when you sign up you’re signing up for the entire package of services, so if you’re so inclined you can use Amazon E-Commerce Services (like setting up your own mini Amazon, assuming you’re a programmer) or Alexa Web Services and a couple of others that sound really cool, but which I personally don’t understand at all.
You’ll need to wait for Amazon to let you know that your account is ready before you can use it. They’ll send an email message to your default Amazon email address advising you that your account has been set up, and then you need to go back and log in to gain access to your S3 space.
So anyway, once you’ve signed up, you need to get the two keys Amazon provides that identifies you and your files within the service. One is public, and one is private. You’ll need them both to use anything to access your S3 buckets (directories) and objects (files). Your keys are accessed online after logging in to your AWS account by clicking on the badge that says “Your Web Services Account” which will open a pop-up with some menu choices, and the one you want is “View Access Key Identifiers.” Click on that and there they are!
Copy them to something like a text file on your computer so you can cut and paste them into the S3Fox Organizer, which acts like Windows Explorer and allows you to transfer files of any sort to your S3 space.
S3 uses buckets to hold your objects, and you can have up to 120 100 buckets and unlimited objects. Bucket names are global, so no one else has the bucket name you’re adding to S3, that’s why you only get 100. If you have a bucket called “bucket,” I couldn’t have a bucket called “bucket” because you now ‘own’ “bucket.” S3 is just a great big disk array with a bunch of buckets on the same space. Bucket names are created and ‘owned’ by s3 users. No 2 bucket names can be the same.
And since you can put everything in a single bucket, there’s no need to have a lot of buckets. Unless you want or need 100 buckets. Locally, you can set up folders on your laptop using Jungle Disk (see below) or another S3 utility, but those are virtual folders on your own computer. You’re specifying in Jungle Disk which bucket to use for storage — S3 is just for storage. It doesn’t care about organization as long as all buckets are named something different.
If you’re setting up one bucket as read-everyone and one as read-you and one as read-user, you’ll need 3 buckets. But, again, you also set “read” on files, so the bucket ACL doesn’t matter except that you need to make it read-everyone in order for any objects inside it to be read-everyone.
Name your bucket(s) carefully, though. Once you set up a bucket, you can’t change it. You can’t move objects from one bucket to another (reassign files to another folder) and you can’t delete a bucket unless there’s nothing in it.
First thing you want to do is add a bucket for your objects and define the access control list (ACL) for the bucket(s) and the object(s). In S3Fox, click on Create Directory (the blue folder icon) and name your bucket something. I called mine “glassdog” because, like, duh. After creating it, click on its name and either right-click it and select “Edit ACL” or click on the Edit ACL icon, which is a pencil on a piece of paper.
You’ll see some green checkmarks (meaning yes) and red x’s (meaning no) and there’ll be 3 green checks next to your name, and the rest (for Everyone and Authenticated Users) will have no access. Simply click on the red X under “Read” for Everyone and the objects you subsequently upload to that bucket will be publicly accessible to anyone.
Next, upload a file or two and repeat the same procedures, changing the ACL for the files within your publicly accessible bucket to be Read by Everyone.
Guess what? You’re done! To access the file you uploaded via a web browser (or to create a link on a web page) just use http://s3.amazonaws.com/bucket_name_here/ substituting your own bucket name, obviously, and follow that up with the file name of the object you uploaded and, whoop, there it is!
It’s important to note that you cannot delete a bucket if there’s anything in it. You have delete all the objects contained in a bucket, and then you can delete empty buckets.
You can now use S3 for all your big file needs and pay for what you use for a lot cheaper than almost any other method — except for free services like YouTube, that is, but you get what you pay for, right?
Once you have your S3 account, I’d also suggest downloading and installing Jungle Disk, a free utility for Windows or Mac that will create a virtual networked drive on your computer that looks like it’s residing on your local computer but is really out on your S3 space. Jungle Disk allows you to treat S3 like another drive so you can drag and drop anything into the buckets you’ve created, giving you two methods of getting stuff online.
Backup your entire iTunes library or set up a weekly (nightly?) backup on your computer to your Jungle Disk. At 15¢ per Gig, my 60+Gb library of music would cost me $9.00 a month. That’s like 3 coffees for state of mind — plus I can get to it from anywhere.
And that’s it! Have fun with your new virtually unlimited storage space and start sharing your high-quality audio and video with your friends and the world.

This entry was posted in The Wonderful WWW. Bookmark the permalink.

2 Responses to "The Big Fat Open Directory in the Sky"