add new user and password on CentOS for Amazon AWS EC2

$ useradd new_user_name
$ passwd new_user_name

how to install apache subversions on Amazon AWS EC2

take a look at README under /etc/httpd/conf.d

$ pwd
/etc/httpd/conf.d
[ec2-user@puppet conf.d]$ ls
notrace.conf  README  subversion.conf  welcome.conf
[ec2-user@puppet conf.d]$ cat subversion.conf

LoadModule dav_svn_module     modules/mod_dav_svn.so
LoadModule authz_svn_module   modules/mod_authz_svn.so
LoadModule dontdothat_module  modules/mod_dontdothat.so

#
# Example configuration to enable HTTP access for a directory
# containing Subversion repositories, "/var/www/svn".  Each repository
# must be both:
#
#   a) readable and writable by the 'apache' user, and
#
#   b) labelled with the 'httpd_sys_content_t' context if using
#   SELinux
#

#
# To create a new repository "http://localhost/repos/stuff" using
# this configuration, run as root:
#
#   # cd /var/www/svn
#   # svnadmin create stuff  
#   # chown -R apache:apache stuff
#   # chcon -R -t httpd_sys_content_t stuff
#

#<Location /repos>
#   DAV svn
#   SVNParentPath /var/www/svn
#
#   # Limit write permission to list of valid users.
#   <LimitExcept GET PROPFIND OPTIONS REPORT>
#      # Require SSL connection for password protection.
#      # SSLRequireSSL
#
#      AuthType Basic
#      AuthName "Authorization Realm"
#      AuthUserFile /path/to/passwdfile
#      Require valid-user
#   </LimitExcept>
#</Location>

What happen if I can't start my apache web server on Amazon aws ec2 ?

$ sudo service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using puppet.katdc.com for ServerName
(98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
                                                           [FAILED]
[ec2-user@puppet init.d]$ sudo !!
sudo sudo service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using puppet.katdc.com for ServerName
(98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
                                                           [FAILED]
[ec2-user@puppet init.d]$ sudo service nginx stop
Stopping nginx:                                            [  OK  ]
[ec2-user@puppet init.d]$ sudo sudo service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using puppet.katdc.com for ServerName
                                                           [  OK  ]

How to upload a whole directory of files to S3 in the command line.

s3cmd of course

How to install s3cmd? take a look at this URL.

http://ruentex.blogspot.tw/

How easy to install OpenVPN on an AWS EC2 CentOS instance.

## don't forget to open UDP 1194 port on your firewall.

$ wget https://github.com/viljoviitanen/setup-simple-openvpn/archive/master.zip
--2013-07-24 09:45:57--  https://github.com/viljoviitanen/setup-simple-openvpn/archive/master.zip
Resolving github.com (github.com)... 204.232.175.90
Connecting to github.com (github.com)|204.232.175.90|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/viljoviitanen/setup-simple-openvpn/zip/master [following]
--2013-07-24 09:45:58--  https://codeload.github.com/viljoviitanen/setup-simple-openvpn/zip/master
Resolving codeload.github.com (codeload.github.com)... 192.30.252.146
Connecting to codeload.github.com (codeload.github.com)|192.30.252.146|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘master.zip’

    [  <=>                                  ] 44,128       195KB/s   in 0.2s  

2013-07-24 09:45:59 (195 KB/s) - ‘master.zip’ saved [44128]

$ unzip master.zip
Archive:  master.zip
8c4c582d5a15a2e9a3bc2fac0691aae1bb942436
   creating: setup-simple-openvpn-master/
  inflating: setup-simple-openvpn-master/LICENSE.txt
  inflating: setup-simple-openvpn-master/OPENVPN-COPYING.txt
  inflating: setup-simple-openvpn-master/README.rst
   creating: setup-simple-openvpn-master/easy-rsa/
  inflating: setup-simple-openvpn-master/easy-rsa/Makefile
  inflating: setup-simple-openvpn-master/easy-rsa/README
  inflating: setup-simple-openvpn-master/easy-rsa/build-ca
  inflating: setup-simple-openvpn-master/easy-rsa/build-dh
  inflating: setup-simple-openvpn-master/easy-rsa/build-inter
  inflating: setup-simple-openvpn-master/easy-rsa/build-key
  inflating: setup-simple-openvpn-master/easy-rsa/build-key-pass
  inflating: setup-simple-openvpn-master/easy-rsa/build-key-pkcs12
  inflating: setup-simple-openvpn-master/easy-rsa/build-key-server
  inflating: setup-simple-openvpn-master/easy-rsa/build-req
  inflating: setup-simple-openvpn-master/easy-rsa/build-req-pass
  inflating: setup-simple-openvpn-master/easy-rsa/clean-all
  inflating: setup-simple-openvpn-master/easy-rsa/index.html
  inflating: setup-simple-openvpn-master/easy-rsa/inherit-inter
  inflating: setup-simple-openvpn-master/easy-rsa/list-crl
  inflating: setup-simple-openvpn-master/easy-rsa/openssl-0.9.6.cnf
  inflating: setup-simple-openvpn-master/easy-rsa/openssl.cnf
  inflating: setup-simple-openvpn-master/easy-rsa/pkitool
  inflating: setup-simple-openvpn-master/easy-rsa/revoke-full
  inflating: setup-simple-openvpn-master/easy-rsa/sign-req
  inflating: setup-simple-openvpn-master/easy-rsa/vars
  inflating: setup-simple-openvpn-master/easy-rsa/whichopensslcnf
  inflating: setup-simple-openvpn-master/setup.sh
  inflating: setup-simple-openvpn-master/template-client-config
  inflating: setup-simple-openvpn-master/template-server-config
[ec2-user@wiki2 install_vpn]$ ls
master.zip  setup-simple-openvpn-master
[ec2-user@wiki2 install_vpn]$ cd *master
[ec2-user@wiki2 setup-simple-openvpn-master]$ ls
easy-rsa             README.rst              template-server-config
LICENSE.txt          setup.sh
OPENVPN-COPYING.txt  template-client-config
[ec2-user@wiki2 setup-simple-openvpn-master]$ vi RE*
[ec2-user@wiki2 setup-simple-openvpn-master]$ sudo ./setup.sh
sudo: ./setup.sh: command not found
[ec2-user@wiki2 setup-simple-openvpn-master]$ chmod +x *.sh
[ec2-user@wiki2 setup-simple-openvpn-master]$ sudo ./setup.sh
Loaded plugins: priorities, security, update-motd, upgrade-helper
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package openvpn.x86_64 0:2.3.1-3.8.amzn1 will be installed
--> Processing Dependency: liblzo2.so.2()(64bit) for package: openvpn-2.3.1-3.8.amzn1.x86_64
--> Processing Dependency: libpkcs11-helper.so.1()(64bit) for package: openvpn-2.3.1-3.8.amzn1.x86_64
--> Running transaction check
---> Package lzo.x86_64 0:2.06-2.3.amzn1 will be installed
---> Package pkcs11-helper.x86_64 0:1.07-5.4.amzn1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package            Arch        Version                 Repository         Size
================================================================================
Installing:
 openvpn            x86_64      2.3.1-3.8.amzn1         amzn-updates      428 k
Installing for dependencies:
 lzo                x86_64      2.06-2.3.amzn1          amzn-main          60 k
 pkcs11-helper      x86_64      1.07-5.4.amzn1          amzn-main          55 k

Transaction Summary
================================================================================
Install       3 Package(s)

Total download size: 543 k
Installed size: 1.1 M
Downloading Packages:
(1/3): lzo-2.06-2.3.amzn1.x86_64.rpm                     |  60 kB     00:00    
(2/3): openvpn-2.3.1-3.8.amzn1.x86_64.rpm                | 428 kB     00:00    
(3/3): pkcs11-helper-1.07-5.4.amzn1.x86_64.rpm           |  55 kB     00:00    
--------------------------------------------------------------------------------
Total                                           598 kB/s | 543 kB     00:00    
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : lzo-2.06-2.3.amzn1.x86_64                                    1/3
  Installing : pkcs11-helper-1.07-5.4.amzn1.x86_64                          2/3
  Installing : openvpn-2.3.1-3.8.amzn1.x86_64                               3/3
  Verifying  : openvpn-2.3.1-3.8.amzn1.x86_64                               1/3
  Verifying  : pkcs11-helper-1.07-5.4.amzn1.x86_64                          2/3
  Verifying  : lzo-2.06-2.3.amzn1.x86_64                                    3/3

Installed:
  openvpn.x86_64 0:2.3.1-3.8.amzn1                                            

Dependency Installed:
  lzo.x86_64 0:2.06-2.3.amzn1       pkcs11-helper.x86_64 0:1.07-5.4.amzn1    

Complete!
NOTE: If you run ./clean-all, I will be doing a rm -rf on /etc/openvpn/easy-rsa/keys
Generating DH parameters, 1024 bit long safe prime, generator 2
This is going to take a long time
...........+.....................+............+............................+......................................................................................................................+................+..+....+..................+....+....................................+......................................................+........+...................+..............+.......................................................................................+.+...............................................+....+.....................................+.......................................+.....+............................................++*++*++*
Using CA Common Name: simpleopenvpn CA
Generating a 1024 bit RSA private key
......................................................................++++++
..............................++++++
writing new private key to 'ca.key'
-----
Generating a 1024 bit RSA private key
...++++++
................++++++
writing new private key to 'myserver.key'
-----
Using configuration from /etc/openvpn/easy-rsa/openssl.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName           :PRINTABLE:'US'
stateOrProvinceName   :PRINTABLE:'CA'
localityName          :PRINTABLE:'Simple OpenVPN server'
organizationName      :PRINTABLE:'simpleopenvpn'
commonName            :PRINTABLE:'myserver'
emailAddress          :IA5STRING:'me@myhost.mydomain'
Certificate is to be certified until Jul 22 09:48:03 2023 GMT (3650 days)

Write out database with 1 new entries
Data Base Updated
Generating a 1024 bit RSA private key
...................++++++
............................................++++++
writing new private key to 'client1-simpleopenvpn.key'
-----
Using configuration from /etc/openvpn/easy-rsa/openssl.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName           :PRINTABLE:'US'
stateOrProvinceName   :PRINTABLE:'CA'
localityName          :PRINTABLE:'Simple OpenVPN server'
organizationName      :PRINTABLE:'simpleopenvpn'
commonName            :PRINTABLE:'client1-simpleopenvpn'
emailAddress          :IA5STRING:'me@myhost.mydomain'
Certificate is to be certified until Jul 22 09:48:04 2023 GMT (3650 days)

Write out database with 1 new entries
Data Base Updated
Detecting external ip address
============================================================
Detected your server external ip address: 54.250.178.86
============================================================
Make sure it is correct before using the client configuration files!
  adding: simpleopenvpn.ovpn (deflated 55%)
  adding: ca-simpleopenvpn.crt (deflated 38%)
  adding: client1-simpleopenvpn.key (deflated 22%)
  adding: client1-simpleopenvpn.crt (deflated 47%)
Generated configuration files are in ./openvpn.JYc/ !
Starting openvpn:                                          [  OK  ]




## download you key for PVN client

scp -i ~/.ec2/babyplaykey.pem ec2-user@54.250.178.xx:/home/ec2-user/setup_vpn/setup-simple-openvpn-master/openvpn.JYc/simpleopenvpn-54.250.178.xx.zip .

simpleopenvpn-54.250.178.xx.zip               100% 5938     5.8KB/s   00:00

## unzip this file for as your keys required for VPN connection.

Troubleshooting on puppet agent fail to install mediawiki

## if you see following message during "puppet agent --test --debug"

err: /Stage[main]/Apache/Service[httpd]/ensure: change from stopped to running failed: Could not start Service[httpd]: Execution of '/sbin/service httpd start' returned 1:  at /etc/puppet/modules/apache/manifests/init.pp:77

## test it again manually.

# /sbin/service httpd start
Starting httpd: Syntax error on line 38 of /etc/httpd/conf/httpd.conf:
Invalid command 'LogFormat', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]


## manually remove following lines starting from line 38

$vi /etc/httpd/conf/httpd.conf

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent


## and try again, you will see another error message like

# /sbin/service httpd start
Starting httpd: Syntax error on line 26 of /etc/httpd/conf.d/15-default.conf:
Invalid command 'CustomLog', perhaps misspelled or defined by a module not included in the server configuration
                                                           [FAILED]


## edit /etc/httpd/conf.d/15-default.conf remove following line starting from line 26

CustomLog /var/log/httpd/access_log combined

## and try again

# /sbin/service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
                                                           [  OK  ]


# it works.

## DON'T RUN  "puppet agent ..." command again.

######

## check your contents under /var/www/html for customization.

$ ls -la
total 17528
drwxr-xr-x  3 root root     4096 Jul 18 02:28 .
drwxr-xr-x  6 root root     4096 Jul 17 09:50 ..
-rw-r--r--  1 root root      107 Jul 18 02:28 index.html
drwxr-xr-x 15 1226  550     4096 Jul 18 02:43 mediawiki-1.19.1
-rw-r--r--  1 root root 17929538 Jun 13  2012 mediawiki-1.19.1.tar.gz


[ec2-user@wiki2 html]$ pwd
/var/www/html

do you remember what your user data was when you create this AWS EC2 instance? (try this command)

curl http://169.254.169.254/latest/user-data

install a puppet agent (client) on AWS CE2 CentOS

$ sudo yum install puppet
$ puppet --version
2.7.22

## make sure your puppet server should install puppet version higher than this. (it's 3.2.3 on my puppet server)

## check your host name as in the /etc/puppet/manifests/nodes.pp file to make sure what you want to include.

$ puppet agent --server=puppet.katdc.com --debug --test --waitforcert=60

## on your puppet server side, to certify this hist

$ puppet cert list --all

## if you see a host name, such as AAA not signed.

$ puppet cert sign AAA

## back to your puppet agent, it will keep going and install all the categories.

install open vpn on Amazon CentOS EC2

https://github.com/viljoviitanen/setup-simple-openvpn

very good reference.

It works fine with Mac OS Tunnelblick.

to set ssh authorized_keys on CentOS is different from Ubuntu on EC2

cd ~/.ssh
chmod og-rw authorized_keys
chmod a-x authorized_keys

chmod 700 ~/.ssh

## append your local ~/.ssh/id_rsa.pub content to remote site's authorized_keys

install puppet master on an EC2 CentOS instance.

## both hostnames on master and agent are matter, not IP address.
## agent should be able to resolve puppet master's IP address, but agent (client) is not.
## therefore, puppet agents could be not on Internet, but the puppet master is on Internet.

   55  rpm -ivh http://yum.puppetlabs.com/el/5/products/i386/puppetlabs-release-5-6.noarch.rpm
   56  rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-7.noarch.rpm
   57  rpm -Uvh http://mirror01.idc.hinet.net/EPEL/6/x86_64/epel-release-6-8.noarch.rpm
   58  yum install puppet-server

   59  cd /etc/puppet/manifests
   60 vi site.pp

// add following lines

import "classes/*"
    node default {
        include sudo
}

   65  mkdir classes
   66  vi /etc/puppet/manifests/classes/sudo.pp

// add following lines

class sudo {
        file {
                "/etc/sudoers": owner => "root", group => "root", mode => 440,
        }
}

   68  service puppetmaster start
   69  chkconfig puppetmaster on

   72  puppetca --list

   77  vi /etc/puppet/autosign.conf

// add "*" in autosign.conf  if you don't want to bother with certification, use firewall to control your puppet client access permission.

mount a new file system on CentOS for EC2

1. create a new volume on AWS EC2 first. (use standard for example)

2. then attach it to the instance you want to mount on. (as a device name for example /dev/sdf)

3. ssh your instance and try to mount it.

of course, change to root using $sudo -s

4. $fdisk /dev/sdf

if there is no any partition, use n to create a new one.

then use w to sync and quit

5. $/sbin/mkfs.ext4 -L /backup /dev/sdf1

to format the whole partition to ext 4 format.

6. $mkdir /your_mount_point

make a mount point

7. $mount /dev/sdf1 /your_mount_point

8. $df

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda1              8256952   7359700    477824  94% /

none                    290848       116    290732   1% /dev

none                    325224         0    325224   0% /dev/shm

none                    325224        56    325168   1% /var/run

none                    325224         0    325224   0% /var/lock

none                    325224         0    325224   0% /lib/init/rw

/dev/sdf1             51605436    184136  48799896   1% /your_mount_point

9. to keep it auto mount when system boot

copy the following line in /etc/mtab

/dev/sdf1 /your_mount_point ext4 rw 0 0

to /etc/fstab

for example

proc                   /proc  proc  nodev,noexec,nosuid                      0  0  

LABEL=cloudimg-rootfs  /      ext3  defaults                                 0  0  

/dev/sdb /mnt auto defaults,nobootwait,comment=cloudconfig 0 0

/dev/sdf1 /your_mount_point ext4 rw 0 0

My new blog: ProgrammableWeb APIs for iOS

http://programmableWeb4iOS.blogspot.tw/