Skip to main content

PHP CSV pagination (chunk), Postgresql

https://github.com/symfony/http-foundation/blob/master/File/File.php
https://stackoverflow.com/questions/39972653/laravel-chunk-method-using-eloquent
./src/Illuminate/Database/Concerns/BuildsQueries.php@chunk()
https://medium.com/@barryvdh/streaming-large-csv-files-with-laravel-chunked-queries-4158e484a5a2
https://stackoverflow.com/questions/3705318/simple-php-pagination-script
Use virtual mc as old version system config to figure out what feature worth, case study: multi threaded in psql 9.6+

Kl idea on build some computing powerhouse lab. For example running multiple server with software both old and new to experiment and performance compare...
Scale up to a lab for education & training is cool.

https://en.wikipedia.org/wiki/Kunming%E2%80%93Singapore_railway

https://stackoverflow.com/jobs/293378/devops-engineer-automate-and-scale-securityscorecard

Job description
The Opportunity

SecurityScorecard is hiring an Ops Engineer to bridge the gap between our global development and operational teams who is motivated to help continue automating and scaling our infrastructure. The Ops Engineer will be responsible for setting up and managing the operation of project development and test environments as well as the software configuration management processes for the entire application development lifecycle. Your role would be to ensure the optimal availability, latency, scalability, and performance of our product platforms. You would also be responsible for automating production operations, promptly notifying backend engineers of platform issues, and checking long term quality metrics.

Our infrastructure is based on AWS with a mix of managed services like RDS, ElastiCache, and SQS, as well as hundreds of EC2 instances managed with Ansible and Terraform. We are actively using three AWS regions, and have equipment in several data centers across the world.

This role will be remote in North America, or in our HQ in NY, NY.

Responsibilities

Training, mentoring, and lending expertise to coworkers with regards to operational and security best practises.
Reviewing and providing feedback on GitHub Pull Requests to team members AND development teams- a significant percentage of our Software Engineers have written Terraform.
Identifying opportunities for technical and process improvement and owning the implementation.
Championing the concepts of immutable containers, Infrastructure as Code, stateless applications, and software observability throughout the organization.
Systems performance tuning with a focus on high availability and scalability.
Building tools to ease the usability and automation of processes
Keeping products up and operating at full capacity
Assisting with migration processes as well as backup and replication mechanisms
Working on a large-scale distributed environment where you were focused on scalability/reliability/performance
Ensuring proper monitoring / alerting are configured
Investigating incidents and performance lapses
Come help us with projects such as…

Extending our compute clusters to support low latency, on-demand job execution
Turning pets into cattle
Cross region replication of systems and corresponding data to support low latency access
Rolling out application performance monitoring to existing services, extending integrations where required
Migration from self hosted ELK to a SaaS stack
Continuous improvement of CI/CD processes making builds & deployments faster, safer, and more consistent
Extending a Global VPN WAN to a datacenter with IPSec+BGP
Requirements

3+ years of DevOps and/or Operations experience
1+ years of production environment experience with Amazon Web Services (AWS)
1+ years using SQL databases (MySQL, Oracle, Postgres)
Scripting ability (Bash, Python, C++ a plus)
Strong Experience with CI/CD processes (Jenkins, Ansible) and automated configuration tools (Puppet/Chef/Ansible)
Experience with container orchestration (AWS ECS, Kubernetes, Marathon/Mesos)
Ability to work as part of a highly collaborative team
Understanding of monitoring tools like DataDog
Strong written and verbal communication skills
Nice to Have

You knew exactly what is meant by "Turning pets into cattle"
Experience working with Kubernetes on bare-metal and/or the AWS Elastic Kubernetes Service.
Experience with RabbitMQ, MongoDB, or Apache Kafka.
Experience with Presto or Apache Spark.
Familiarity with computation orchestration tools such as HTCondor, Apache Airflow, or Argo.
Understanding of network concepts- OSI layers, firewalls, DNS, split horizon DNS, VPN, routing, BGP, etc.
A deep understanding of AWS IAM, and how it interacts with S3 buckets.
Experience with SAFe.
Strong programming skills in 2+ languages.

https://www.google.com/maps/d/u/0/viewer?mid=1w_34rP76i_ZoQE173vJpAMRxOJU&ll=20.89964452183693%2C102.26290617959216&z=10

http://treasuresoflaos.blogspot.com/2014/03/chinas-highspeed-train-for-laos-waiting.html https://www.seat61.com/Laos.htm#Bangkok

https://www.gnu.org/ghm/2011/paris/slides/jim-meyering-goodbye-world.pdf
https://explainextended.com/2009/09/16/not-in-vs-not-exists-vs-left-join-is-null-postgresql/
https://stackoverflow.com/questions/16996002/postgresql-query-is-slow-when-using-not-in

https://www.sqlskills.com/blogs/conor/countstar-vs-countcolumn-vs-countdistinct-vs-countcol-over/

https://www.php.net/manual/en/language.generators.overview.php
https://stackoverflow.com/questions/36786690/php-getting-out-of-memory

Comments

Popular posts from this blog

Rand mm 10

https://stackoverflow.com/questions/2447791/define-vs-const Oh const vs define, many time I got unexpected interview question. As this one, I do not know much or try to study this. My work flow, and I believe of many programmer is that search topic only when we have task or job to tackle. We ignore many 'basic', 'fundamental' documents, RTFM is boring. So I think it is a trade off between the two way of study language. And I think there are a bridge or balanced way to extract both advantage of two method. There are some huge issue with programmer like me that prevent we master some technique that take only little time if doing properly. For example, some Red Hat certificate program, lesson, course that I have learned during Collage gave our exceptional useful when it cover almost all topic while working with Linux. I remember it called something like RHEL (RedHat Enterprise Linux) Certificate... I think there are many tons of documents, guide n books about Linux bu

Martin Fowler - Software Architecture - Making Architecture matter

  https://martinfowler.com/architecture/ One can appreciate the point of this presentation when one's sense of code smell is trained, functional and utilized. Those controlling the budget as well as developer leads should understand the design stamina hypothesis, so that the appropriate focus and priority is given to internal quality - otherwise pay a high price soon. Andrew Farrell 8 months ago I love that he was able to give an important lesson on the “How?” of software architecture at the very end: delegate decisions to those with the time to focus on them. Very nice and straight-forward talk about the value of software architecture For me, architecture is the distribution of complexity in a system. And also, how subsystems communicate with each other. A battle between craftmanship and the economics and economics always win... https://hackernoon.com/applying-clean-architecture-on-web-application-with-modular-pattern-7b11f1b89011 1. Independent of Frameworks 2. Testable 3. Indepe