How to upload to Amazon S3 using REST

mulatinho - - 2 mins read

Hello friends!

Look on the internet a alternative way to upload files on Amazon S3 without AWS-SDK and then you will know that is a dificult thing for many; Wait a second, why someone will need to use S3 without the Amazon SDK? That’s the problem! We all know that sometimes you can’t do simple a import/require in some lib written by another guy/girl to turn things to into a easy way. That’s the beauty of an challenge.

So to win that, all you need to do is read two docs of Amazon:

The first documentation describes how you handle the HTTP PUT using simple REST and the second describes how to implements the Amazon signature based on AWS_ACCESS_KEY and AWS_SECRET_KEY;

Backing to the problem, days ago in my work we need to face this problem: the development version of a framework that we use can’t import AWS-SDK into project and after the mess that this problem caused, it make me think in do this post on blog;

I was already do it a year ago using shell-script when I need to automatize a build process of MacOS to send the .IPA into S3, so I start to do some changes in code and finally written two new version: one in shell script and other in node.js; The principal problem found, the root of many problems to some guys is the signature process, as described on Amazon docs all you need to sign is:

HTTP-RESOURCE
MD5_BASE64_HASH
mimetype
DateInIsoUTCTime
someHeadersOfAmazon
fullPathOfObjectFile
  • HTTP-RESOURCE is just the HTTP REQUEST TYPE: PUT/GET..
  • MD5_BASE64_HASH is one of the tricks of Amazon Doc (bad trick)
  • mimetype i just use application/octet-stream works for all types of file
  • DateInIsoUTCTime like -> Wed, 26 Oct 2016 12:09:45 +0000
  • someHeadersOfAmazon extra header of Amazon that you need to pass
  • fullPathOfObjectFile /yourBucketName/dir1/subdir2/yourFile.jpg

Or in resume:

PUT
YUORQaKyyJWlorHlFqBtCQ==
application/octet-stream
Fri, 25 Nov 2016 14:27:42 +0000
x-amz-acl:public-read
/2e8dae9c-aad2-4d10-8abc-a7e24d80b4af/darwinorigins.jpg

If you type one character byte wrong then your signature will receive the error below when you try to send your file:

SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method.AKIAIJ5QILNVUULONEGQ

So, later I will post more details here if someone needs and also the node.js code using just the builtin libs: crypto, fs, https and buffer; until then that’s the cURL version

https://gist.github.com/mulatinho/cf148b96248ad5d1d49e8fd84c982454

And the node.JS version, using just native modules;

https://gist.github.com/mulatinho/28e01801d92d3d0000e7935d854b9fd8

Hope it helps someone! :)