Disable Homebrew Auto Update

Homebrew will do self update when we install a package. Sometimes this is not what we want. We are okay using an old version of Homebrew and some old packages.

To prevent Homebrew doing automatic update just set an environment variable named `HOMEBREW_NO_AUTO_UPDATE`.

$ HOMEBREW_NO_AUTO_UPDATE=1 brew install [PACKAGE]
<link crossorigin='anonymous' href='https://cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shCoreDefault.min.css' integrity='sha256-+6BtzNuRjIOAeRxs56BdQS0n2/5w5HLMEoJsXdMP5TI=' rel='stylesheet'/>

References

• https://github.com/Homebrew/brew/issues/1670

Share on Facebook Twitter

Terraform: Define AWS Security Group for All ICMP Traffic

Define AWS Security Group for All ICMP Traffic in Terraform

It is not quite well documented in Terraform what "from" and "to" port number that need to define to allow All ICMP traffic. If you're having hard time trying to figure out here is the solution to All ICMP traffic in Security Group.

...
ingress {
  ...
  from_port   = -1
  to_port     = -1
  protocol    = "icmp"
  ...
}
...

That's it. The special number "-1" did the trick.

Share on Facebook Twitter

How to Kill Background Child Process in Bash

Terminate Background Child Process in Bash

In Bash by child process that sent into background is still alive when main program is terminated. Take a look an example below.

#!/bin/bash

# Run a Python web server
python -m http.server src/ &

# Run SASS watcher and compiler...
sass --watch src/scss:src/css &

wait

When we run script above and terminate using CTRL+C Python and SASS are still running.

Solution to terminate Background Child Process in Bash

The solution to above problem is using shell built-in trap command.

#!/bin/bash

# Kill all child process (Python and SASS) when exit
trap "kill 0" EXIT

# Run a Python web server
python -m http.server src/ &

# Run SASS watcher and compiler...
sass --watch src/scss:src/css &

wait

Now when the script is exited all the child process even which has been sent as background process will also terminated.

References

• https://spin.atomicobject.com/2017/08/24/start-stop-bash-background-process/

Share on Facebook Twitter

Terraform S3 Having Problem with Leading Slash

Problem with S3 Leading Slash in Terraform

S3 accept leading slash "/" and automatically strip them off. When we use it in Terraform as a S3 key it may looks fine until we use it in another object. See example below.

# This bucket is used to store Lambda function and layer
resource "aws_s3_bucket" "deno" {
  bucket = var.default_bucket
  acl = "private"
  tags = var.default_tags
}

# Upload the layer to S3
resource "aws_s3_bucket_object" "deno_func" {
  bucket = aws_s3_bucket.deno.id
  tags = var.default_tags
  # / in front "deno-custom-runtime/function.zip" below creating problem
  key = "/deno-custom-runtime/function.zip"
  source = "${path.module}/../build/function.zip"
  etag = filemd5("${path.module}/../build/function.zip")
}

# Deno Layer
resource "aws_lambda_layer_version" "deno" {
  layer_name = "TeknocerdasDenoRuntime"
  s3_bucket = aws_s3_bucket.deno.id
  s3_key = aws_s3_bucket_object.deno_layer.key
  s3_object_version = aws_s3_bucket_object.deno_layer.version_id
  compatible_runtimes = ["provided"]
  description = "Custom Deno runtime by TeknoCerdas.com"
  source_code_hash = filebase64sha256("${path.module}/../build/layer.zip")
}

When applying the resources we should get error below.

Error: Error creating lambda layer: InvalidParameterValueException: Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist.
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "888bed7e-5345-4d5e-ab0e-0d8c683f49b2"
  },
  Message_: "Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist.",
  Type: "User"
}

Solution to S3 Leading Slash in Terraform

The solution is simply remove the leading slash from the key or filename. So instead of writing /deno-custom-runtime/function.zip use deno-custom-runtime/function.zip.

Problem solved. Simple and stupid.

References

• https://github.com/hashicorp/terraform/pull/15738

Share on Facebook Twitter

How to Escape Character when Running SSH Remote Command

Solution of Escaping Character on SSH

The solution is DO NOT escape it. Use cat HEREDOC to build the string of commands and then pass it to SSH.

$ export VARIABLE="FOO BAR"
$ cat <<EOF | ssh user@myserver bash
echo "This is complex command"
echo "Another command that take a $VARIABLE."
sudo mkdir /tmp/foo && echo "SUCCESS"
EOF

If you have a lot of $ dollar signs in your script and do not want local shell to interpret it then use single quote HEREDOC.

$ export VARIABLE_NAME="FOO BAR"
$ cat <<'EOF' | ssh user@myserver bash
export VARIABLE="SSH VAR"
echo "This is complex command"
echo "Another command that take a $VARIABLE name."
sudo mkdir /tmp/foo && echo "SUCCESS"
EOF

On command above the variable $VARIABLE value is "SSH VAR" since it is took the value from remote server not from local machine.

Share on Facebook Twitter

Variable Variables in Shell

Subtitute Variable inside Variable in Bash

Example below is using Bash for variable variables subtitution.

$ hello="Hello World"
$ foobar="hello"
$ echo "${!foobar}"
Hello World

Subtitute Variable inside Variable using eval

This is for other shell which do not recognize "${!}" syntax. It utilise eval so use it with caution.

$ hello="Hello World"
$ foobar="hello"
$ eval echo "\$${foobar}"
Hello World

References for Variable Variables in Shell

• https://stackoverflow.com/a/2634767/2566677
• http://mywiki.wooledge.org/BashFAQ/006

Share on Facebook Twitter

Terraform: Force Destroy Resource when prevent_destroy is true

How to Force Destroy Resource in Terraform

Terraform resource that having lifecycle prevent_destroy = true can not be destroyed. You need to manually edit the file inplace and change the value prevent_destroy to false manually each time you want to destroy the resource. Instead of having to edit manually and make git status dirty we can automate this using simple shell script.

Automate Force Destroy Resource in Terraform

The idea is simple.

1. Search all *.tf files and look the value of prevent_default = true to prevent_default = false
2. Run terraform destroy command
3. Revert the changes back to prevent_default = true

Here is the implementation in Bash.

#!/bin/bash

$ find . -name '*.tf' -type f \
-exec perl -i -pe 's@prevent_destroy = true@prevent_destroy = false@g' {} \;

# Run terraform destroy
[ "$IS_PLAN" = "yes" ] && terraform plan -destroy || terraform destroy $@

# Revert the changes
$ find . -name '*.tf' -type f \
 -exec perl -i -pe 's@prevent_destroy = false@prevent_destroy = true@g' {} \;

Save the file with name e.g: terraform-force-destroy.sh. To issue terraform plan -destroy command use the following.

$ IS_PLAN=yes bash terraform-force-destroy.sh

To force destroy the resource use the following command.

$ bash terraform-force-destroy.sh -auto-approve

You can give normal terraform's arguments just like the original terraform destroy.

References for Terraform Force Destroy

• https://github.com/hashicorp/terraform/issues/3640
• https://stackoverflow.com/questions/38354725/prevent-sed-from-adding-newlines-at-the-end-of-files
• https://spacelift.io/blog/how-to-destroy-terraform-resources

Share on Facebook Twitter

Expose module in global scope using Browserify or Webpack

Goal

You want to expose a module as in global scope so it can be called in HTML file. For example we will create a small function for reversing a string. We will expose it as StrReverse.

// File main.js
module.exports = function(str) {
  return str.split('').reverse().join('');
}

Browserify

$ browserify --standalone StrReverse main.js --outfile bundle.js

The key is --standalone parameter.

Webpack

$ webpack-cli --mode=none --output-library StrReverse main.js --output bundle.js

The key is --output-library parameter.

Test in HTML

Create a HTML file and include bundle.js via <script> tag.

<!DOCTYPE html>
<html>
<body>
<script src="bundle.js"></script>
var reversed = StrReverse("Hello World");
document.write(reversed);
</body>
</html>

Share on Facebook Twitter

Compile Swoole Extension on MacOS using Homebrew

What is Swoole

Swoole is Production-Grade Async programming Framework for PHP. It helps you write high-performance asynchronous non-blocking I/O. Similar with Go or NodeJS.

How to Compile using Homebrew

I am using MacOS High Sierra and PHP 7.2.20.

$ export LD_LIBRARY_PATH=$( brew --prefix openssl )/lib
$ export CPATH=$( brew --prefix openssl)/include
$ export PKG_CONFIG_PATH=$( brew --prefix openssl )/lib/pkgconfig
$ pecl install swoole
...
Configuring for:
PHP Api Version:         20170718
Zend Module Api No:      20170718
Zend Extension Api No:   320170718
enable sockets supports? [no] : yes
enable openssl support? [no] : yes
enable http2 support? [no] : no
enable mysqlnd support? [no] : no
...
... some long message about compiling
...
Build process completed successfully
Installing '/usr/local/Cellar/php@7.2/7.2.20/include/php/ext/swoole/config.h'
Installing '/usr/local/Cellar/php@7.2/7.2.20/pecl/20170718/swoole.so'
install ok: channel://pecl.php.net/swoole-4.4.2
Extension swoole enabled in php.ini

Read more »

Share on Facebook Twitter

MySQL DISTINCT with Case Sensitive

Goals

We want MySQL distinct to use case sensitive grouping because by default MySQL use case insensitive.

Solution of MySQL DISTINCT case sensitive

We can use binary operator to convert the character set.

SELECT DISTINCT CAST(expr as BINARY)

As an alternative we can just use BINARY.

SELECT BINARY expr

References

• https://stackoverflow.com/questions/19462919/mysql-select-distinct-should-be-case-sensitive
• https://dev.mysql.com/doc/refman/5.6/en/charset-binary-set.html

Share on Facebook Twitter

Disable Word Wrap on MySQL Shell

Goals

Turn off or disable word wrap on MySQL shell

Solution of Disable Word Wrap on MySQL Shell

We can use external pager such as less to do the job. Pager in MySQL shell actually is a pipe to another program.

mysql> pager less -SFX
PAGER set to 'less -SFX'

That's it. Simple and easy. Now when you have very long output horizontally it will not wrap.

Reference

• http://blog.clearandfizzy.com/post/145657760526/mysql-turn-off-word-wrap

Share on Facebook Twitter

Curl Dump Response Headers to STDOUT and Ignore Response Body

Goals

Return only HTTP response header when opening a web page using curl. This is useful when we are interested in processing response headers only.

Command

We will utilize /dev/stdout and /dev/null to achieve what we want.

$ curl https://notes.rioastamal.net -D /dev/stdout -o /dev/null --silent
HTTP/2 200
date: Sun, 06 May 2018 23:42:37 GMT
content-type: text/html; charset=UTF-8
set-cookie: __cfduid=d6347d57f364b276150034b241b19cdb01525650157; expires=Mon, 06-May-19 23:42:37 GMT; path=/; domain=.rioastamal.net; HttpOnly
expires: Sun, 06 May 2018 23:42:37 GMT
cache-control: private, max-age=0
last-modified: Sun, 06 May 2018 23:42:05 GMT
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 416f4deb6f51a320-HKG

Reference

• https://unix.stackexchange.com/questions/16357/usage-of-dash-in-place-of-a-filename

Share on Facebook Twitter

Generate Random String in Shell Using /dev/urandom

Goals

Generate random string in Shell and using /dev/urandom as the source. This random string typically useful to be used as encryption key.

Implementation

We will use combination of tr and head to generate 32 random characters. Command below will only output alphanumeric and some characters symbol only.

$ </dev/urandom tr -dc 'A-Za-z0-9!"#$%&()*+,-./:;<=>?@[\]^_`{|}~' | head -c 32 && echo
}s9s2c8W7aZlI:yg<{bg&-<7YnyJEk.u

On Mac OS X system you may need to define LC_ALL=C environment variable as shown below.

$ LC_ALL=C </dev/urandom tr -dc 'A-Za-z0-9!"#$%&()*+,-./:;<=>?@[\]^_`{|}~' | head -c 32 && echo
f(s_TPj*.H3Z/s[*:zLe[=9&0$FF"*8[

References

• https://www.howtogeek.com/howto/30184/10-ways-to-generate-a-random-password-from-the-command-line/
• https://unix.stackexchange.com/questions/230673/how-to-generate-a-random-string

Share on Facebook Twitter

How to Flatten Multidimensional Array in PHP

Goals

Turn PHP multi-dimensional array into one dimensional array (Flatten).

Solution

We will use Standard PHP Library (SPL) to tackle the problem. Assume we have an array like below.

$origin = [
    'Level 1',
    '_2_' => [
        'Level 2',
        '_3_' => [
            'Level 3',
            '_4_' => [
                'Level 4',
                '_5_' => [
                    'Level 5'
                 ]
            ]
        ]
   ],
   'Another Level 1',
   '_2_1' => [
       'Another Level 2'
   ]
];

Turn it into one dimensional array by using SPL RecursiveIteratorIterator and RecursiveArrayIterator class.

Read more »

Share on Facebook Twitter

How to Remove Multiple Redis Cache

Goals

Remove multiple Redis cache key using one liner command.

Steps

Assume keys that we want to delete are keys beginning with 'laravel:' prefix.

$ redis-cli KEYS 'laravel:*'
1) "laravel:featured:2bf46b40ed15"
2) "laravel:list:5509f57fdaef"
3) "laravel:list:2bf46b40ed15"
4) "laravel:list:e1bf5bfc2ab8"
5) "laravel:list-total-rec:adc81409d693"
6) "laravel:list-total-rec:e1bf5bfc2ab8"
7) "laravel:promotion-list:adc81409d693"
8) "laravel:list:d1f68333e3bb"
9) "laravel:list:c6a45c5cf5c5"

Just pipe the output above to xargs command. By default redis-cli will use raw format when STDOUT is not tty, which in case is true for xargs.

$ redis-cli KEYS 'laravel:*' | xargs redis-cli DEL
(integer) 9

Reference

• https://redis.io/commands/

Share on Facebook Twitter

Redis: How to Increase File Descriptor Limits

Problem

When you run redis server it complains can not set maximum open files because it has reached the OS max file descriptor limits. Here is the sample output.

$ ./bin/redis-server
28436:C 25 Nov 20:10:03.978 # Warning: no config file specified, using the default config. In order to specify a config file use ./bin/redis-server /path/to/redis.conf
28436:M 25 Nov 20:10:03.979 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
28436:M 25 Nov 20:10:03.979 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
28436:M 25 Nov 20:10:03.979 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
[...CUT...]

When you try to increase the maximum file descriptor using ulimit as root by issuing sudo it returns an error.

$ sudo ulimit -n 65000
sudo: ulimit: command not found

Wow, WTF is that? ulimit is a shell built so giving sudo an instruction to run a command called ulimit will not work. It will the same as statement below.

Read more »

Share on Facebook Twitter

How to Fix No Sound After Mute and Unmute on XFCE

Problem

There is no sound after doing mute then unmute on XFCE 4 Ubuntu 14.04.

Solution

Try running amixer command to see the status of the Master sound.

$ amixer get Master
Simple mixer control 'Master',0
  Capabilities: pvolume pswitch pswitch-joined
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 65536
  Mono:
  Front Left: Playback 65536 [100%] [off]
  Front Right: Playback 65536 [100%] [off]

In above result the status of Front Left and Front Right is [off]. Meaning it is still muted even has been unmuted from the XFCE panel. Try to toggle the switch to make it [on].

Read more »

Share on Facebook Twitter

How to Extract Specific Directory from Tarball

Problem

We have huge file of gzipped tarball and we want to extract only specific directory from the tarball.

Solution

Make sure the pattern we want to extract by searching it first. As an example we want to extract directory named johndoe-website, but we did not know the full pattern of the directory.

$ tar tvf the-archive.tar.gz | grep johndoe-website
home/sites/clients/johndoe-website/javascripts/main.js
home/sites/clients/johndoe-website/styles/main.css
home/sites/clients/johndoe-website/index.html

From the output above we knew that the pattern of the directory is home/sites/clients/johndoe-website. Command below will extract johndoe-website from the archive and strip the 3 leading directories.

$ tar xvf the-archive.tar.gz --strip-components=3 -C /destination/path home/sites/clients/johndoe-website

Command above works in GNU Tar and BSD Tar (Mac OS X).

Read more »

Share on Facebook Twitter

Quickest Way: Using STDIN and Pipe to Copy SSH Public Key to Server

Goal

Copy SSH public key to another machine without using external tools such as ssh-copy-id - Only pure shell built-in or at least standard commands.

Solution

The solution is using shell STDIN and PIPE it to ssh.

$ cat ~/.ssh/id_rsa.pub | ssh user@hostname 'cat >> .ssh/authorized_keys -'

The quote for the ssh arguments is important because without it the redirection will goes to your local machine instead of remote machine. The "-" at the last of cat command on the remote indicate it reads the input from STDIN.

Reference

• http://askubuntu.com/questions/4830/easiest-way-to-copy-ssh-keys-to-another-machine

Share on Facebook Twitter

Expose Port Inside Running Container on Docker Toolbox for Mac

Problem

Docker only allows to define port that need to be exposed when doing container creation. When the container already running and new port need to be exposed, you're out of luck.

Goal

You want to expose new port which run by application inside a running container, so you can hit the docker-vm-ip:port to access the port on Mac OS X.

Assumptions

• IP of Boot2Docker VM (Which run by Virtualbox) is 192.168.99.100
• IP of the docker container running the application is 172.17.0.2
• The application listen on address 0.0.0.0 and port 80

Read more »

Share on Facebook Twitter