Skip to main content

Note on Jenkins and AWS CI/CD

Image
By

https://unix.stackexchange.com/questions/401068/web-browser-from-console-centos

https://devops.stackexchange.com/questions/6863/what-is-better-between-jenkins-aws-codedeploy-for-ci-cd

https://linuxize.com/post/how-to-install-jenkins-on-centos-7/

https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-using-firewalld-on-centos-8

https://forums.aws.amazon.com/thread.jspa?threadID=83249

cat /etc/issue

cat /etc/system-release


https://github.com/vicenteguerra/git-deploy
http://lessgeneric.com/server/deploy-bitbucket-webhook-phpbash/

https://support.atlassian.com/bitbucket-cloud/docs/create-and-trigger-a-webhook-tutorial/

~/Downloads/ngrok http -host-header=rewrite concepts.local:80


https://searchaws.techtarget.com/tip/Compare-AWS-CodePipeline-vs-Jenkins-for-CI-CD

sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
https://stackoverflow.com/questions/61344317/im-getting-error-public-key-for-jenkins-2-232-1-1-noarch-rpm-is-not-installed


Add apache to Git pull

https://gist.github.com/cyakimov/6456518

Not easy

https://jondavidjohn.com/git-pull-from-a-php-script-not-so-simple/


https://unix.stackexchange.com/questions/55106/disable-user-shell-for-security-reasons

usermod -s /sbin/nologin apache

Add user to bash
sudo chsh -s /bin/bash webapp

Add sudoers
sudo visudo
webapp ALL = NOPASSWD: /usr/bin/git

Old version ngrok Archives
https://dl.equinox.io/ngrok/ngrok/stable/archive

Init Jenkins password
cat /var/lib/jenkins/secrets/initialAdminPassword
https://stackoverflow.com/questions/8072700/how-to-restart-jenkins-manually
https://stackoverflow.com/questions/32108380/jenkins-how-to-build-a-specific-branch
Jenkins Log
./log/jenkins/jenkins.log
I often using find command to figure out where logging going to.
First see where jenkins files:
where jenkins
After know that jenkins most likely stored in /var, I go to /var/
find . -mmin -10
This mean I find all files that changed within 10 minutes. If there are any logs, 
this show be shown in the result.

./log/secure
./log/jenkins/jenkins.log

Bingo => I found this log. There are many other log but you can guest which one is. Or use
wc -l log/jenkins/jenkins.log 
first to see any line in this.
Log is a precious view to see what going on. GUI Jenkins do not show many useful logs.
2021-05-07 03:19:39.128+0000 [id=414]	INFO	i.j.p.b.f.r.BitBucketPPRRepositoryPushActionFilter#shouldTriggerBuild: Should trigger build for the bitbucket action: BitBucketPPRRepositoryAction?
2021-05-07 03:19:39.128+0000 [id=414]	INFO	i.j.p.b.util.BitBucketPPRUtils#matches: no matches for allowed branches pattern: refs/heads/staging-dev
2021-05-07 03:19:39.128+0000 [id=414]	INFO	i.j.p.b.f.r.BitBucketPPRRepositoryPushActionFilter#shouldTriggerBuild: Should trigger build for the bitbucket action: BitBucketPPRRepositoryAction?
2021-05-07 03:19:39.128+0000 [id=414]	INFO	i.j.p.b.util.BitBucketPPRUtils#matches: no matches for allowed branches pattern: refs/heads/staging-dev

Bitbucket webhook (need admin permission), ie.
http://greenmatter.com:8080/bitbucket-hook/
Noted that on my demo no https, port 8080. Double check (Edit button) event on Push/change 
more specifictly.

Post build HTTP call.
https://stackoverflow.com/questions/22571868/jenkins-invoking-rest-api-or-wget-at-end-of-jenkins-job-without-writing-any-s

https://stackoverflow.com/questions/25522360/ngrok-configure-multiple-port-in-same-domain

Console Output
Started by Bitbucket PPR: repository event by Dzung Ng
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/brakes-dev
The recommended git tool is: NONE
using credential d3f6bb7a-5053-47ac-91bd-5a980222866b
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/brakes-dev/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://bitbucket.org/se/brakes-website.git # timeout=10
Fetching upstream changes from https://bitbucket.org/se/brakes-website.git
 > git --version # timeout=10
 > git --version # 'git version 2.27.0'
using GIT_ASKPASS to set credentials 
 > git fetch --tags --force --progress -- https://bitbucket.org/se/brakes-website.git +refs/heads/*:refs/remotes/origin/* # timeout=10
skipping resolution of commit 6a27331bb48c9cba70c28fb69cf8a719e3eea2c3, since it originates from another repository
Seen branch in repository origin/PBRBRAKESUSA-SUDIP-03-APR-2019
Seen branch in repository origin/master
Seen branch in repository origin/staging-dev
Seen branch in repository origin/staging-dev-jenkin
Seen branch in repository origin/task/refactor-config
Seen 19 remote branches
 > git show-ref --tags -d # timeout=10
Checking out Revision 6a27331bb48c9cba70c28fb69cf8a719e3eea2c3 (origin/staging-dev-jenkin)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6a27331bb48c9cba70c28fb69cf8a719e3eea2c3 # timeout=10
Commit message: "jk 12"
 > git rev-list --no-walk 50191a3bc28e45ebc6c36b0e81f310b3a0217b58 # timeout=10
Run condition [Always] enabling prebuild for step [[HTTP Request, HTTP Request]]
Run condition [Always] enabling perform for step [[HTTP Request, HTTP Request]]
HttpMethod: GET
URL: https://dev.brakes.com/script/web-hooks.php
Sending request to url: https://dev.brakes.com/script/web-hooks.php
ERROR: [flexible-publish] HTTP Request (jenkins.plugins.http_request.HttpRequest@5e699e42) aborted due to exception
java.lang.IllegalStateException: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
	at jenkins.plugins.http_request.HttpRequestExecution.call(HttpRequestExecution.java:263)
	at jenkins.plugins.http_request.HttpRequestExecution.call(HttpRequestExecution.java:80)
	at hudson.remoting.LocalChannel.call(LocalChannel.java:47)
	at jenkins.plugins.http_request.HttpRequest.perform(HttpRequest.java:414)
	at org.jenkins_ci.plugins.flexible_publish.builder.FailAtEndBuilder.perform(FailAtEndBuilder.java:106)
	at org.jenkins_ci.plugins.run_condition.BuildStepRunner$2.run(BuildStepRunner.java:112)
	at org.jenkins_ci.plugins.run_condition.BuildStepRunner$Fail.conditionalRun(BuildStepRunner.java:156)
	at org.jenkins_ci.plugins.run_condition.BuildStepRunner.perform(BuildStepRunner.java:107)
	at org.jenkins_ci.plugins.flexible_publish.strategy.FailAtEndExecutionStrategy.perform(FailAtEndExecutionStrategy.java:86)
	at org.jenkins_ci.plugins.flexible_publish.ConditionalPublisher.perform(ConditionalPublisher.java:206)
	at org.jenkins_ci.plugins.flexible_publish.FlexiblePublisher.perform(FlexiblePublisher.java:124)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:752)
	at hudson.model.Build$BuildExecution.post2(Build.java:177)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:697)
	at hudson.model.Run.execute(Run.java:1932)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
	at sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
	at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
	at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
	at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
	at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152)
	at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1383)
	at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1291)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:435)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at jenkins.plugins.http_request.util.HttpClientUtil.execute(HttpClientUtil.java:121)
	at jenkins.plugins.http_request.HttpRequestExecution.executeRequest(HttpRequestExecution.java:392)
	at jenkins.plugins.http_request.HttpRequestExecution.authAndRequest(HttpRequestExecution.java:328)
	at jenkins.plugins.http_request.HttpRequestExecution.call(HttpRequestExecution.java:260)
	... 19 more
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
	at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:386)
	at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:291)
	at sun.security.validator.Validator.validate(Validator.java:271)
	at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:223)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
	at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
	... 45 more
Caused by: java.security.cert.CertPathValidatorException: validity check failed
	at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
	at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:220)
	at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
	at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
	at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
	at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:381)
	... 51 more
Caused by: java.security.cert.CertificateExpiredException: NotAfter: Fri Aug 14 12:00:00 UTC 2020
	at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:277)
	at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:677)
	at sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190)
	at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144)
	at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
	... 56 more
HttpMethod: GET
URL: http://188.217.197.37/web-hooks.php
Sending request to url: http://188.217.197.37/web-hooks.php
Response Code: HTTP/1.1 200 OK
Success code from [100‥399]
Build step 'Flexible publish' changed build result to FAILURE
Build step 'Flexible publish' marked build as failure
Finished: FAILURE
Wget -q = quiet (no output).

wget -q  --no-check-certificate  http://dev.brakes.com/script/web-hooks.php

https://www.cyberciti.biz/faq/how-to-curl-ignore-ssl-certificate-warnings-command-option/


https://stackoverflow.com/questions/27198326/execute-shell-script-from-workspace-on-remote-machine-after-build-successful-je
https://stackoverflow.com/questions/26797219/echo-off-in-jenkins-console-output
https://askubuntu.com/questions/147241/execute-sudo-without-password
Bash shell look like:
#!/bin/bash
cd /var/www/testjk.com/
#echo "#&$#(PASS" | sudo -S su && su user1 && git pull origin dev-jenkin
PASSWD='#&$#(PASS'
echo $PASSWD  | sudo -S su user1-c 'git pull origin dev-jenkin'

The reason why we do not run git pull on root because of many things. One of these is we set .git
folder permission to user1 (event root can not run git pull). 
With custom build branch:

https://stackoverflow.com/questions/32108380/jenkins-how-to-build-a-specific-branch
https://stackoverflow.com/questions/10433105/dynamically-fill-jenkins-choice-parameter-with-git-branches-in-a-specified-repo

GIT branch inside bash:
#!/bin/bash
PASSWD='abc****###'

cd /var/www/devusa.com/

BUILD_BRANCH_NAME=$1 # This param passed from Jenkins Shell; export not work.
echo "${BUILD_BRANCH_NAME}"

if [ -z "${BUILD_BRANCH_NAME}" ]; then
    echo "Default" 
    BUILD_BRANCH_NAME='dev'
fi

echo $PASSWD  | sudo -S su user1 -c "git fetch && git checkout ${BUILD_BRANCH_NAME}" 
echo $PASSWD  | sudo -S su user1 -c "git pull origin ${BUILD_BRANCH_NAME}"

# Because ssh user do not have permission to run git pull (due to our business / rules).
https://stackoverflow.com/questions/5750450/how-can-i-print-each-command-before-executing
Change user that run Jenkins, used in for example when your Web server same with Jenkins server.
So you do not have problems (or have to work around) with jenkins user permission (ie. git pull).
http://blog.manula.org/2013/03/running-jenkins-under-different-user-in.html


Comments

Post a Comment

Popular posts from this blog

Rand mm 10

By
https://stackoverflow.com/questions/2447791/define-vs-const Oh const vs define, many time I got unexpected interview question. As this one, I do not know much or try to study this. My work flow, and I believe of many programmer is that search topic only when we have task or job to tackle. We ignore many 'basic', 'fundamental' documents, RTFM is boring. So I think it is a trade off between the two way of study language. And I think there are a bridge or balanced way to extract both advantage of two method. There are some huge issue with programmer like me that prevent we master some technique that take only little time if doing properly. For example, some Red Hat certificate program, lesson, course that I have learned during Collage gave our exceptional useful when it cover almost all topic while working with Linux. I remember it called something like RHEL (RedHat Enterprise Linux) Certificate... I think there are many tons of documents, guide n books about Linux bu
Post a Comment
Read more

Martin Fowler - Software Architecture - Making Architecture matter

By
  https://martinfowler.com/architecture/ One can appreciate the point of this presentation when one's sense of code smell is trained, functional and utilized. Those controlling the budget as well as developer leads should understand the design stamina hypothesis, so that the appropriate focus and priority is given to internal quality - otherwise pay a high price soon. Andrew Farrell 8 months ago I love that he was able to give an important lesson on the “How?” of software architecture at the very end: delegate decisions to those with the time to focus on them. Very nice and straight-forward talk about the value of software architecture For me, architecture is the distribution of complexity in a system. And also, how subsystems communicate with each other. A battle between craftmanship and the economics and economics always win... https://hackernoon.com/applying-clean-architecture-on-web-application-with-modular-pattern-7b11f1b89011 1. Independent of Frameworks 2. Testable 3. Indepe
Post a Comment
Read more