This is yet another set of PHP scripts for S3. It's primary difference from other packages is the us of the CURL functions to reduce memory usage when PUTing or GETing large files. Mime type detection is from the /usr/bin/file command which is only opened once, there were issues when exec()ing more than 1000 times. Some other packages also used a bucket:object_key convention we do not. Choice of delimiter is '/', there is not option to change it.
The primary command is s3.php, make a symlink as /usr/bin/s3 to make life simple. Currently there is also a command titled s3sync too. This script will shortly be merged with the s3.php script.
Creates a "REST Query String Authentication" url to allow access to objects
s3 auth /bucket/prefix-00/object.001 http://s3.amazonaws.com/bucket/prefix-00/object.001?AWSAccessKeyId=123456789&Expires=141889120&Signature=edoceoybdZaNmGa%2ByT272YEAiv4%3D
s3 ls ( or s3 ls /) b-- 07/13/06 03:53 0b bucket b-- 07/13/06 03:53 0b bucket.test b-- 07/13/06 03:53 0b bucket.real
List bucket contents
s3 ls /bucket/ -p- 12/31/69 04:00 0b prefix-00/ -p- 12/31/69 04:00 0b prefix-01/ -p- 12/31/69 04:00 0b prefix-02/
List buckets recursively
s3 ls -r some.bucket: --o 04/16/07 02:01 3799b prefix-00/object.one --o 09/15/06 04:00 48923197b prefix-01/object.two --o 04/20/07 16:20 2457600b prefix-02/object.nnn
s3 get /bucket/prefix-00/prefix-01/object.key /path/to/local
s3 put /path/to/local /bucket/prefix-00/prefix-01/object.key
There are three objects provided by this library S3Cache, S3Connection and S3Response. The S3Connection object provides some ftp like methods to operate, get, ls, put, rm, stat. These methods return S3Response objects which provide a bunch of information about the request. The S3Cache is used by the sync feature but could be used by any consumer.
$s3c = new S3Connection($key,$secret,$config); $s3r = $s3c->ls('/bucket/object/key',null,'/'); $s3r = $s3c->get('/bucket/object/key','/path/to/local/file'); $s3r = $s3c->put('/path/to/local/file','/some/object/key'); $s3r = $s3c->stat('/bucket/object/key');
When uploading large files a display can be forced by settings the curl_progress parameter. The value set here will trigger the CURL progress meter when files are larger than this value. A null or zero value will disable progress meters.
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 85 109M 0 0 85 93.2M 0 67540 0:28:21 0:24:07 0:04:14 70389
This was when putting a file and processing a cache of about 8700 objects. Cache file size was 1M MB and the Cache is fully memory resident in this software.
UID C SZ RSS PSR STIME TTY TIME CMD you 6 14486 17216 0 22:43 pts/0 00:00:04 /usr/bin/php s3sync bucket /full/local/path /prefix/in/bucket/
This is from a 35424 object chache, the cache file was 2.7 MB
UID C SZ RSS PSR STIME TTY TIME CMD you 48 12770 44608 0 00:31 pts/0 00:00:45 /usr/bin/php s3sync bucket /full/local/path/ /prefix/in/bucket/
This package requires the PEAR package Crypt_HMAC and the SimpleXMLElement extensions for PHP. Install Crypt_HMAC as indicated below.
pear install --onlyreqdeps Crypt_HMAC
SimpleXMLElement should be included when building PHP. From source:
# ./configure [whatever] --with-simplexml [whatever]
Gentoo USE flags:
Available from Edoceo or your local PHP developer, please start with our contact page. If the volume of requests increases we'll add direct email request or forums/email-list or ???.
- 28 Apr 2007 - Add auth() routines /djb
- 27 Apr 2007 - Cleaned S3 toolkit, now using single instance of file command /djb
- 17 Apr 2007 - Released /djb