Skip to main content

S3 Encrypt NodeJS upload/server files

https://cloud.google.com/solutions/mysql-remote-access
https://stackoverflow.com/questions/19665863/how-do-i-use-a-self-signed-certificate-for-a-https-node-js-server
https://github.com/gilt/node-s3-encryption-client
https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html
https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingEncryption.html

S3 encryption with customer provided key SSE-C required SSL. What does it mean ?
Protect data at rest (?), at transfer.
https://cloud.google.com/compute/docs/load-balancing/tcp-ssl/
https://www.aptible.com/documentation/enclave/tutorials/faq/file-uploads.html

SSL sample
https://github.com/coolaj86/nodejs-ssl-example
But only SSE-C require SSL ?

Wall of text, API reference class, property ...
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#upload-property

http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property


https://github.com/badunk/multer-s3
https://medium.com/@FalabellaDaniel/amazon-s3-file-upload-873b0b345089

https://github.com/Automattic/knox
http://frontendgods.com/using-q-promise-and-async-waterfall/

https://aws.amazon.com/blogs/developer/generating-amazon-s3-pre-signed-urls-with-sse-c-part-4/

Request download generated by S3 web console:
https://s3.us-east-2.amazonaws.com/vn-test/uploads/chat/image70_1508924754278.png?response-content-disposition=attachment
&X-Amz-Security-Token=FQoDYXdzEDwaDFtNHrqje2si2S4DOiL6AVMIRe4p2tWUXS0ktfr5hoT4TNbP1LQYZH4BYCWxxwMUm68l1GWzoN1x3MrV5TqDwvmebg67W3NJyiYhBIg5gso3%2BtDScMLKP%2FPLTVnhm9de%2Bb8LwmDkCnLaWEygQBrHjnCyBMITARSEv1VgGEUEUUEEgHEebZxCWytlh9l2YrqNQ2BNRW6f9ZDKvLE1OFLki0h5qujyQcTYbiTSHB0amlxB6dlLonwH1PNYdNnEkpt9u6DpNB%2FHTv9pb9bwHHlYTIo77ODiylfVzx183B9tfTx6FbgfROdcR4zDK8hVKHncrHlxnGtyZJaDjv%2FugN5pvvlpTGzNQol%2FC%2FzwU%3D
&X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Date=20171025T095258Z
&X-Amz-SignedHeaders=host
&X-Amz-Expires=300
&X-Amz-Credential=ASIAJEZ23IISCAEUEUEU%2F20171025%2Fus-east-2%2Fs3%2Faws4_request
&X-Amz-Signature=981075c5882doioeueue36d1776432b139dbf436be497dc157caca513ca80c263

KMSKeyID arn:aws:kms:us-east-2:462604284947:key/f8f234.p.p3-cfbb-4475-88d5-ce210d9c10c5

Signed by AWS CLI:
aws s3 presign s3://vn-test/uploads/chat/7.jpg

https://s3.us-east-2.amazonaws.com/vn-test/uploads/chat/7.jpg?
X-Amz-Expires=3600
&X-Amz-SignedHeaders=host
&X-Amz-Date=20171025T100012Z
&X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=AKIAIL4RWJUE>>BFJVQ%2F20171025%2Fus-east-2%2Fs3%2Faws4_request
&X-Amz-Signature=b9dd511ad497iyy4y441e18a87064071964845b6848eb5e8e0b81c09c0f9dd8f3b



S3 v4 has some brilliant implements. This post also discuss about performance when sign signed-url, if it not expire then serve already etc.

Full disclosure I am an aggressive rebaser.

aws v4 max expires in one week ~ 604800 seconds

Serve s3 url over express Request (module). This module seem not use since SSO not use Web yet. But with this API endpoint we need it.
https://medium.com/@stockholmux/node-js-streams-proxies-and-amazon-s3-50b4fabdedbd

Proxy s3 url over express:
1. Use knox s3 client lib:
var aws = require('knox').createClient({
      key: '',
      secret: '',
      bucket: ''
    })

    app.get('/image/:id', function (req, res, next) {
      if (!req.user.is.authenticated) {
        var err = new Error()
        err.status = 403
        next(err)
        return
      }

      aws.get('/image/' + req.params.id)
      .on('error', next)
      .on('response', function (resp) {
        if (resp.statusCode !== 200) {
          var err = new Error()
          err.status = 404
          next(err)
          return
        }

        res.setHeader('Content-Length', resp.headers['content-length'])
        res.setHeader('Content-Type', resp.headers['content-type'])

        // cache-control?
        // etag?
        // last-modified?
        // expires?

        if (req.fresh) {
          res.statusCode = 304
          res.end()
          return
        }

        if (req.method === 'HEAD') {
          res.statusCode = 200
          res.end()
          return
        }

        resp.pipe(res)
      })
    })

OR use request:
    // S3 presigned-URL endpoint
    // @param: im_id: instant_message.id
    router.get('/s3-file/:im_id', requireLogin, function(req, res) {
        // Verify request, if valid then serve URL (return )
        // If presign expired then sign new URL, save then return
        // Update s3 expire time
        // if expire is NULL && || s3_url null => serve direct s3 (old data)



    });

https://developers.google.com/drive/android/auth

Comments

Post a Comment

Popular posts from this blog

Rand mm 10

https://stackoverflow.com/questions/2447791/define-vs-const Oh const vs define, many time I got unexpected interview question. As this one, I do not know much or try to study this. My work flow, and I believe of many programmer is that search topic only when we have task or job to tackle. We ignore many 'basic', 'fundamental' documents, RTFM is boring. So I think it is a trade off between the two way of study language. And I think there are a bridge or balanced way to extract both advantage of two method. There are some huge issue with programmer like me that prevent we master some technique that take only little time if doing properly. For example, some Red Hat certificate program, lesson, course that I have learned during Collage gave our exceptional useful when it cover almost all topic while working with Linux. I remember it called something like RHEL (RedHat Enterprise Linux) Certificate... I think there are many tons of documents, guide n books about Linux bu

Martin Fowler - Software Architecture - Making Architecture matter

  https://martinfowler.com/architecture/ One can appreciate the point of this presentation when one's sense of code smell is trained, functional and utilized. Those controlling the budget as well as developer leads should understand the design stamina hypothesis, so that the appropriate focus and priority is given to internal quality - otherwise pay a high price soon. Andrew Farrell 8 months ago I love that he was able to give an important lesson on the “How?” of software architecture at the very end: delegate decisions to those with the time to focus on them. Very nice and straight-forward talk about the value of software architecture For me, architecture is the distribution of complexity in a system. And also, how subsystems communicate with each other. A battle between craftmanship and the economics and economics always win... https://hackernoon.com/applying-clean-architecture-on-web-application-with-modular-pattern-7b11f1b89011 1. Independent of Frameworks 2. Testable 3. Indepe