Skip to main content

S3 Encrypt NodeJS upload/server files

https://cloud.google.com/solutions/mysql-remote-access
https://stackoverflow.com/questions/19665863/how-do-i-use-a-self-signed-certificate-for-a-https-node-js-server
https://github.com/gilt/node-s3-encryption-client
https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html
https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingEncryption.html

S3 encryption with customer provided key SSE-C required SSL. What does it mean ?
Protect data at rest (?), at transfer.
https://cloud.google.com/compute/docs/load-balancing/tcp-ssl/
https://www.aptible.com/documentation/enclave/tutorials/faq/file-uploads.html

SSL sample
https://github.com/coolaj86/nodejs-ssl-example
But only SSE-C require SSL ?

Wall of text, API reference class, property ...
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#upload-property

http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property


https://github.com/badunk/multer-s3
https://medium.com/@FalabellaDaniel/amazon-s3-file-upload-873b0b345089

https://github.com/Automattic/knox
http://frontendgods.com/using-q-promise-and-async-waterfall/

https://aws.amazon.com/blogs/developer/generating-amazon-s3-pre-signed-urls-with-sse-c-part-4/

Request download generated by S3 web console:
https://s3.us-east-2.amazonaws.com/vn-test/uploads/chat/image70_1508924754278.png?response-content-disposition=attachment
&X-Amz-Security-Token=FQoDYXdzEDwaDFtNHrqje2si2S4DOiL6AVMIRe4p2tWUXS0ktfr5hoT4TNbP1LQYZH4BYCWxxwMUm68l1GWzoN1x3MrV5TqDwvmebg67W3NJyiYhBIg5gso3%2BtDScMLKP%2FPLTVnhm9de%2Bb8LwmDkCnLaWEygQBrHjnCyBMITARSEv1VgGEUEUUEEgHEebZxCWytlh9l2YrqNQ2BNRW6f9ZDKvLE1OFLki0h5qujyQcTYbiTSHB0amlxB6dlLonwH1PNYdNnEkpt9u6DpNB%2FHTv9pb9bwHHlYTIo77ODiylfVzx183B9tfTx6FbgfROdcR4zDK8hVKHncrHlxnGtyZJaDjv%2FugN5pvvlpTGzNQol%2FC%2FzwU%3D
&X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Date=20171025T095258Z
&X-Amz-SignedHeaders=host
&X-Amz-Expires=300
&X-Amz-Credential=ASIAJEZ23IISCAEUEUEU%2F20171025%2Fus-east-2%2Fs3%2Faws4_request
&X-Amz-Signature=981075c5882doioeueue36d1776432b139dbf436be497dc157caca513ca80c263

KMSKeyID arn:aws:kms:us-east-2:462604284947:key/f8f234.p.p3-cfbb-4475-88d5-ce210d9c10c5

Signed by AWS CLI:
aws s3 presign s3://vn-test/uploads/chat/7.jpg

https://s3.us-east-2.amazonaws.com/vn-test/uploads/chat/7.jpg?
X-Amz-Expires=3600
&X-Amz-SignedHeaders=host
&X-Amz-Date=20171025T100012Z
&X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=AKIAIL4RWJUE>>BFJVQ%2F20171025%2Fus-east-2%2Fs3%2Faws4_request
&X-Amz-Signature=b9dd511ad497iyy4y441e18a87064071964845b6848eb5e8e0b81c09c0f9dd8f3b



S3 v4 has some brilliant implements. This post also discuss about performance when sign signed-url, if it not expire then serve already etc.

Full disclosure I am an aggressive rebaser.

aws v4 max expires in one week ~ 604800 seconds

Serve s3 url over express Request (module). This module seem not use since SSO not use Web yet. But with this API endpoint we need it.
https://medium.com/@stockholmux/node-js-streams-proxies-and-amazon-s3-50b4fabdedbd

Proxy s3 url over express:
1. Use knox s3 client lib:
var aws = require('knox').createClient({
      key: '',
      secret: '',
      bucket: ''
    })

    app.get('/image/:id', function (req, res, next) {
      if (!req.user.is.authenticated) {
        var err = new Error()
        err.status = 403
        next(err)
        return
      }

      aws.get('/image/' + req.params.id)
      .on('error', next)
      .on('response', function (resp) {
        if (resp.statusCode !== 200) {
          var err = new Error()
          err.status = 404
          next(err)
          return
        }

        res.setHeader('Content-Length', resp.headers['content-length'])
        res.setHeader('Content-Type', resp.headers['content-type'])

        // cache-control?
        // etag?
        // last-modified?
        // expires?

        if (req.fresh) {
          res.statusCode = 304
          res.end()
          return
        }

        if (req.method === 'HEAD') {
          res.statusCode = 200
          res.end()
          return
        }

        resp.pipe(res)
      })
    })

OR use request:
    // S3 presigned-URL endpoint
    // @param: im_id: instant_message.id
    router.get('/s3-file/:im_id', requireLogin, function(req, res) {
        // Verify request, if valid then serve URL (return )
        // If presign expired then sign new URL, save then return
        // Update s3 expire time
        // if expire is NULL && || s3_url null => serve direct s3 (old data)



    });

https://developers.google.com/drive/android/auth

Comments

Post a Comment

Popular posts from this blog

AWS Elasticache Memcached connection

https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/accessing-elasticache.html#access-from-outside-aws http://hourlyapps.blogspot.com/2010/06/examples-of-memcached-commands.html Access memcached https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/GettingStarted.AuthorizeAccess.html Zip include hidden file https://stackoverflow.com/questions/12493206/zip-including-hidden-files phpmemcachedadmin ~ phpMyAdmin or phpPgAdmin ... telnet mycachecluster.eaogs8.0001.usw2.cache.amazonaws.com 11211 stats items stats cachedump 27 100 https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/VPCs.EC.html https://lzone.de/cheat-sheet/memcached VPC ID Security Group ID (sg-...) Cluster: The identifier for the cluster memcached1 Creation Time: The time (UTC) when the cluster was created January 9, 2019 at 11:47:16 AM UTC+7 Configuration Endpoint: The configuration endpoint of the cluster memcached1.ahgofe.cfg.usw1.cache.amazonaws.com:11211 St...

Notes Windows 10 Virtualbox config, PHP Storm Japanese, custom PHP, Apache build, Postgresql

 cmd => Ctrl + Shift + Enter mklink "C:\Users\HauNT\Videos\host3" "C:\Windows\System32\drivers\etc\hosts" https://www.quora.com/How-to-create-a-router-in-php https://serverfault.com/questions/225155/virtualbox-how-to-set-up-networking-so-both-host-and-guest-can-access-internet 1 NAT + 1 host only config https://unix.stackexchange.com/questions/115464/how-to-properly-set-up-2-network-interfaces-in-centos-running-in-virtualbox DEVICE=eth0 TYPE=Ethernet #BOOTPROTO=dhcp BOOTPROTO=none #IPADDR=10.9.11.246 #PREFIX=24 #GATEWAY=10.9.11.1 #IPV4_FAILURE_FATAL=yes #HWADDR=08:00:27:CC:AC:AC ONBOOT=yes NAME="System eth0" [root@localhost www]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 # Advanced Micro Devices, Inc. [AMD] 79c970 [PCnet32 LANCE] DEVICE=eth1 IPADDR=192.168.56.28 <= no eff => auto like DHCP #GATEWAY=192.168.56.1 #BOOTPROTO=dhcp BOOTPROTO=static <= no eff ONBOOT=yes HWADDR=08:00:27:b4:20:10 [root@localhost www]# ...

Rocket.Chat DB schema

_raix_push_notifications avatars.chunks avatars.files instances meteor_accounts_loginServiceConfiguration meteor_oauth_pendingCredentials meteor_oauth_pendingRequestTokens migrations rocketchat__trash rocketchat_cron_history rocketchat_custom_emoji rocketchat_custom_sounds rocketchat_import rocketchat_integration_history rocketchat_integrations rocketchat_livechat_custom_field rocketchat_livechat_department rocketchat_livechat_department_agents rocketchat_livechat_external_message rocketchat_livechat_inquiry rocketchat_livechat_office_hour rocketchat_livechat_page_visited rocketchat_livechat_trigger rocketchat_message rocketchat_oauth_apps rocketchat_oembed_cache rocketchat_permissions rocketchat_raw_imports rocketchat_reports rocketchat_roles rocketchat_room rocketchat_settings rocketchat_smarsh_history rocketchat_statistics rocketchat_subscription rocketchat_uploads system.indexes users usersSessions https://rocket.chat/docs/developer-guides/sc...