Skip to main content

PHP CSV pagination (chunk), Postgresql

https://github.com/symfony/http-foundation/blob/master/File/File.php
https://stackoverflow.com/questions/39972653/laravel-chunk-method-using-eloquent
./src/Illuminate/Database/Concerns/BuildsQueries.php@chunk()
https://medium.com/@barryvdh/streaming-large-csv-files-with-laravel-chunked-queries-4158e484a5a2
https://stackoverflow.com/questions/3705318/simple-php-pagination-script
Use virtual mc as old version system config to figure out what feature worth, case study: multi threaded in psql 9.6+

Kl idea on build some computing powerhouse lab. For example running multiple server with software both old and new to experiment and performance compare...
Scale up to a lab for education & training is cool.

https://en.wikipedia.org/wiki/Kunming%E2%80%93Singapore_railway

https://stackoverflow.com/jobs/293378/devops-engineer-automate-and-scale-securityscorecard

Job description
The Opportunity

SecurityScorecard is hiring an Ops Engineer to bridge the gap between our global development and operational teams who is motivated to help continue automating and scaling our infrastructure. The Ops Engineer will be responsible for setting up and managing the operation of project development and test environments as well as the software configuration management processes for the entire application development lifecycle. Your role would be to ensure the optimal availability, latency, scalability, and performance of our product platforms. You would also be responsible for automating production operations, promptly notifying backend engineers of platform issues, and checking long term quality metrics.

Our infrastructure is based on AWS with a mix of managed services like RDS, ElastiCache, and SQS, as well as hundreds of EC2 instances managed with Ansible and Terraform. We are actively using three AWS regions, and have equipment in several data centers across the world.

This role will be remote in North America, or in our HQ in NY, NY.

Responsibilities

Training, mentoring, and lending expertise to coworkers with regards to operational and security best practises.
Reviewing and providing feedback on GitHub Pull Requests to team members AND development teams- a significant percentage of our Software Engineers have written Terraform.
Identifying opportunities for technical and process improvement and owning the implementation.
Championing the concepts of immutable containers, Infrastructure as Code, stateless applications, and software observability throughout the organization.
Systems performance tuning with a focus on high availability and scalability.
Building tools to ease the usability and automation of processes
Keeping products up and operating at full capacity
Assisting with migration processes as well as backup and replication mechanisms
Working on a large-scale distributed environment where you were focused on scalability/reliability/performance
Ensuring proper monitoring / alerting are configured
Investigating incidents and performance lapses
Come help us with projects such as…

Extending our compute clusters to support low latency, on-demand job execution
Turning pets into cattle
Cross region replication of systems and corresponding data to support low latency access
Rolling out application performance monitoring to existing services, extending integrations where required
Migration from self hosted ELK to a SaaS stack
Continuous improvement of CI/CD processes making builds & deployments faster, safer, and more consistent
Extending a Global VPN WAN to a datacenter with IPSec+BGP
Requirements

3+ years of DevOps and/or Operations experience
1+ years of production environment experience with Amazon Web Services (AWS)
1+ years using SQL databases (MySQL, Oracle, Postgres)
Scripting ability (Bash, Python, C++ a plus)
Strong Experience with CI/CD processes (Jenkins, Ansible) and automated configuration tools (Puppet/Chef/Ansible)
Experience with container orchestration (AWS ECS, Kubernetes, Marathon/Mesos)
Ability to work as part of a highly collaborative team
Understanding of monitoring tools like DataDog
Strong written and verbal communication skills
Nice to Have

You knew exactly what is meant by "Turning pets into cattle"
Experience working with Kubernetes on bare-metal and/or the AWS Elastic Kubernetes Service.
Experience with RabbitMQ, MongoDB, or Apache Kafka.
Experience with Presto or Apache Spark.
Familiarity with computation orchestration tools such as HTCondor, Apache Airflow, or Argo.
Understanding of network concepts- OSI layers, firewalls, DNS, split horizon DNS, VPN, routing, BGP, etc.
A deep understanding of AWS IAM, and how it interacts with S3 buckets.
Experience with SAFe.
Strong programming skills in 2+ languages.

https://www.google.com/maps/d/u/0/viewer?mid=1w_34rP76i_ZoQE173vJpAMRxOJU&ll=20.89964452183693%2C102.26290617959216&z=10

http://treasuresoflaos.blogspot.com/2014/03/chinas-highspeed-train-for-laos-waiting.html https://www.seat61.com/Laos.htm#Bangkok

https://www.gnu.org/ghm/2011/paris/slides/jim-meyering-goodbye-world.pdf
https://explainextended.com/2009/09/16/not-in-vs-not-exists-vs-left-join-is-null-postgresql/
https://stackoverflow.com/questions/16996002/postgresql-query-is-slow-when-using-not-in

https://www.sqlskills.com/blogs/conor/countstar-vs-countcolumn-vs-countdistinct-vs-countcol-over/

https://www.php.net/manual/en/language.generators.overview.php
https://stackoverflow.com/questions/36786690/php-getting-out-of-memory

Comments

Popular posts from this blog

AWS Elasticache Memcached connection

https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/accessing-elasticache.html#access-from-outside-aws http://hourlyapps.blogspot.com/2010/06/examples-of-memcached-commands.html Access memcached https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/GettingStarted.AuthorizeAccess.html Zip include hidden file https://stackoverflow.com/questions/12493206/zip-including-hidden-files phpmemcachedadmin ~ phpMyAdmin or phpPgAdmin ... telnet mycachecluster.eaogs8.0001.usw2.cache.amazonaws.com 11211 stats items stats cachedump 27 100 https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/VPCs.EC.html https://lzone.de/cheat-sheet/memcached VPC ID Security Group ID (sg-...) Cluster: The identifier for the cluster memcached1 Creation Time: The time (UTC) when the cluster was created January 9, 2019 at 11:47:16 AM UTC+7 Configuration Endpoint: The configuration endpoint of the cluster memcached1.ahgofe.cfg.usw1.cache.amazonaws.com:11211 St...

Notes Windows 10 Virtualbox config, PHP Storm Japanese, custom PHP, Apache build, Postgresql

 cmd => Ctrl + Shift + Enter mklink "C:\Users\HauNT\Videos\host3" "C:\Windows\System32\drivers\etc\hosts" https://www.quora.com/How-to-create-a-router-in-php https://serverfault.com/questions/225155/virtualbox-how-to-set-up-networking-so-both-host-and-guest-can-access-internet 1 NAT + 1 host only config https://unix.stackexchange.com/questions/115464/how-to-properly-set-up-2-network-interfaces-in-centos-running-in-virtualbox DEVICE=eth0 TYPE=Ethernet #BOOTPROTO=dhcp BOOTPROTO=none #IPADDR=10.9.11.246 #PREFIX=24 #GATEWAY=10.9.11.1 #IPV4_FAILURE_FATAL=yes #HWADDR=08:00:27:CC:AC:AC ONBOOT=yes NAME="System eth0" [root@localhost www]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 # Advanced Micro Devices, Inc. [AMD] 79c970 [PCnet32 LANCE] DEVICE=eth1 IPADDR=192.168.56.28 <= no eff => auto like DHCP #GATEWAY=192.168.56.1 #BOOTPROTO=dhcp BOOTPROTO=static <= no eff ONBOOT=yes HWADDR=08:00:27:b4:20:10 [root@localhost www]# ...

Rocket.Chat DB schema

_raix_push_notifications avatars.chunks avatars.files instances meteor_accounts_loginServiceConfiguration meteor_oauth_pendingCredentials meteor_oauth_pendingRequestTokens migrations rocketchat__trash rocketchat_cron_history rocketchat_custom_emoji rocketchat_custom_sounds rocketchat_import rocketchat_integration_history rocketchat_integrations rocketchat_livechat_custom_field rocketchat_livechat_department rocketchat_livechat_department_agents rocketchat_livechat_external_message rocketchat_livechat_inquiry rocketchat_livechat_office_hour rocketchat_livechat_page_visited rocketchat_livechat_trigger rocketchat_message rocketchat_oauth_apps rocketchat_oembed_cache rocketchat_permissions rocketchat_raw_imports rocketchat_reports rocketchat_roles rocketchat_room rocketchat_settings rocketchat_smarsh_history rocketchat_statistics rocketchat_subscription rocketchat_uploads system.indexes users usersSessions https://rocket.chat/docs/developer-guides/sc...