Skip to main content

Posts

MySQL table_exists and rename table execution time

https://stackoverflow.com/questions/9479056/table-exists-method-might-not-be-working-properly https://dba.stackexchange.com/questions/53836/how-long-will-it-take-to-rename-table INSERT with check exist (normally we often using UNIQUE key then INSERT IGNORE). https://stackoverflow.com/questions/3164505/mysql-insert-record-if-not-exists-in-table https://stackoverflow.com/questions/38601880/unable-to-rename-table-due-to-metadata-lock RENAME TABLE  will terminate the transaction. You can do both renames in a single, atomic, statement (no transaction): RENAME TABLE `main_territorypricing` TO `main_territorypricing2`, `main_territorypricing1` TO `main_territorypricing`; It will have to wait for any other connections that are busy with any of the tables; make sure you don't have some connection not letting go. https://tech.smartling.com/wrangling-gigantic-tables-in-mysql-universal-language-40274d89151b Copy data between databases. https://stackoverflow.com/questions/12242...

Optical Character Recognition OCR for Phone number and email address images to text

Sometime we want to crawling data from a website. First, crawling data is illegal (in most cases?). But I tried anyway, for technical study, apply first. It turned out that there are many good tools for this. So I start to use these tools for current project. It mostly consist of NodeJS Cheerio for read HTML (like jQuery with browser). OCR Tesseract for read some text from images. Install locally or use node package. And obviously Database, I am using MySQL. http://www.leptonica.org/ https://realpython.com/setting-up-a-simple-ocr-server/ https://nanonets.com/blog/ocr-with-tesseract/ Test (training ?) data. https://osdn.net/projects/sfnet_tesseract-ocr-alt/downloads/tesseract-ocr-3.02.eng.tar.gz/ OCR + OpenCV (a lot of example to use ) (English only ?) https://nanonets.com/blog/ocr-with-tesseract/ https://stackoverflow.com/questions/14800730/tesseract-running-error Some Android app I have tried work really well with Vietnamese (tried with printed Shopee invoice). Will try another app an...

Grepping huge file with fgrep

  https://stackoverflow.com/questions/13913014/grepping-a-huge-file-80gb-any-way-to-speed-it-up LC_ALL=C fgrep -A 5 -B 5 'db_pd.Clients' eightygigsfile.sql -A 5 -B 5 mean get 5 lines in for example CREATE TABLE `r1_category` ( `id` int(11) NOT NULL AUTO_INCREMENT, `cat_name` varchar(100) NOT NULL, `caption` varchar(100) NOT NULL, `cat_caption` varchar(100) NOT NULL, `coupon_catid` int(11) DEFAULT NULL, // Trimmed line KEY `cat_name` (`cat_name`,`discount_status`) ) ENGINE=InnoDB AUTO_INCREMENT=19 DEFAULT CHARSET=latin1; /*!40101 SET character_set_client = @saved_cs_client */; => If you want full columns in CREATE query, increase A B parameter: LC_ALL=C fgrep -A 15 -B 15 'db_pd.Clients' eightygigsfile.sql > Client.sql // grep -Fn --text "1-Piston Calipers; Oversized" product_NOV_22.csv // -F for treat input pattern search as fixed string

Notes on product variants database design and some about ngrok

https://magento.stackexchange.com/questions/10338/stock-control-race-condition-between-checkout-and-payment-completion https://essay.utwente.nl/78020/1/Emous_van_MA_EEMCS.pdf https://github.com/inconshreveable/ngrok/issues/579 https://stackoverflow.com/questions/31632534/targeting-dev-sites-with-vhost-setup-using-ngrok https://stackoverflow.blog/2019/07/18/building-community-inclusivity-stack-overflow/ ngrok Open http://localhost:4040 https://stackoverflow.com/questions/24923469/modeling-product-variants You could have a design like: More detail... https://stackoverflow.com/questions/8077978/mysql-queries-extremely-slow-in-fresh-xampp-installation Get background ngrok URL  https://stackoverflow.com/questions/34322988/view-random-ngrok-url-when-run-in-background I know that localhost:4040 work but forgot how to get URL on bash since on Server no GUI browser. W3m or lynx do not work with JS. https://martinfowler.com/books/refactoringDatabases.html https://www.red-gate.com/simple-talk...

Note on Jenkins and AWS CI/CD

https://unix.stackexchange.com/questions/401068/web-browser-from-console-centos https://devops.stackexchange.com/questions/6863/what-is-better-between-jenkins-aws-codedeploy-for-ci-cd https://linuxize.com/post/how-to-install-jenkins-on-centos-7/ https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-using-firewalld-on-centos-8 https://forums.aws.amazon.com/thread.jspa?threadID=83249 cat /etc/issue cat /etc/system-release https://github.com/vicenteguerra/git-deploy http://lessgeneric.com/server/deploy-bitbucket-webhook-phpbash/ https://support.atlassian.com/bitbucket-cloud/docs/create-and-trigger-a-webhook-tutorial/ ~/Downloads/ngrok http -host-header=rewrite concepts.local:80 https://searchaws.techtarget.com/tip/Compare-AWS-CodePipeline-vs-Jenkins-for-CI-CD sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key https://stackoverflow.com/questions/61344317/im-getting-error-public-key-for-jenkins-2-232-1-1-noarch-rpm-is-not-installed Add apache to Git pu...

Notes on complex business “illogic” that makes business software so difficult

Even if a company unifies the technology for integration, they run into problems with differences in business process and conceptual dissonance with the data. One division of the company may think a customer is someone with whom it has a current agreement; another division also counts those that had a contract but don’t any longer; another counts product sales but not service sales. That may sound easy to sort out, but when you have hundreds of records in which every field can have a subtly different meaning, the sheer size of the problem becomes a challenge—even if the only person who knows what the field really means is still with the company. (And, of course, all of this changes without warning.) As a result, data has to be constantly read, munged, and written in all sorts of different syntactic and semantic formats. Then there’s the matter of what comes under the term “business logic.” I find this a curious term because there are few things that are less logical than business logic...

Shell Bash get datetime and format

https://stackoverflow.com/questions/1401482/yyyy-mm-dd-format-date-in-shell-script In bash (>=4.2) it is preferable to use printf's built-in date formatter (part of bash) rather than the external date (usually GNU date). As such: # put current date as yyyy-mm-dd in $date # -1 -> explicit current date, bash >=4.3 defaults to current time if not provided # -2 -> start time for shell printf -v date '%(%Y-%m-%d)T\n' -1  # put current date as yyyy-mm-dd HH:MM:SS in $date printf -v date '%(%Y-%m-%d %H:%M:%S)T\n' -1  # to print directly remove -v flag, as such: printf '%(%Y-%m-%d)T\n' -1 # -> current date printed to terminal In bash (<4.2): # put current date as yyyy-mm-dd in $date date=$(date '+%Y-%m-%d') # put current date as yyyy-mm-dd HH:MM:SS in $date date=$(date '+%Y-%m-%d %H:%M:%S') # print current date directly echo $(date '+%Y-%m-%d') Other available date formats can be viewed from the date man pages (for external ...