MetaMask Icon by Icon Mafia
Friday, June 3, 2022
Sunday, February 13, 2022
Thursday, July 30, 2020
Monday, May 11, 2020
Sunday, February 2, 2020
Thursday, January 25, 2018
Wednesday, November 8, 2017
Friday, October 2, 2015
"Generating Go from JSON data" idea from me (ming-der wang) is already public and start implement since Aug. 3, 2015 (at least as early)
I claim the following idea is already on public since I had share or public in any social networks.
"Generating Go from JSON data" is the key concept of gogetDB or my current database as service project (onion, demo-onion, ginger, pepper, chilli, and so on.)
since 2 month ago (10/1/2015) of https://hackpad.com/Project-2-Product-Description-9iPxzxUeewK
all source code are in github (before Oct. 1, 2015), start not to open in additional parts.
the concept is implemented and demo in https://prezi.com/0lrgdih96_aw/copy-of-932015-gcpugtw-free-talk-kubernetes/ on Sept. 3, 2015
also public the way to do it in https://hackpad.com/GCPUG.TW-932015-live-demo-mGYQk3BWynb
The presentation of this concept actually in Sept. 3, 2015 NTU hackathon, I already present it to public.
Also named it "Give me JSON, I will give you an EndPoint (back)"
All copyright belong to Ming-der Wang. log4analytics.com log4security.com
Friday, May 22, 2015
run your docker on google container engine.
Better reading on gist gist
$ kubectl config use-context gke_iron-potion-92209_us-central1-a_hello-world
$ kubectl create -f casandra-pod.yaml
$ kubectl create -f cassandra-service.yaml
$ kubectl get endpoints cassandra -o yaml
apiVersion: v1beta3
kind: Endpoints
metadata:
creationTimestamp: 2015-05-22T07:30:24Z
name: cassandra
namespace: default
resourceVersion: "53898"
selfLink: /api/v1beta1/endpoints/cassandra?namespace=default
uid: 694655e9-0054-11e5-815a-42010af013e9
subsets:
- addresses:
- IP: 10.40.1.5
targetRef:
kind: Pod
name: cassandra
namespace: default
resourceVersion: "53826"
uid: 2e9b73bb-0054-11e5-815a-42010af013e9
ports:
- port: 9042
protocol: TCP
$ kubectl get services
NAME LABELS SELECTOR IP PORT(S)
cassandra name=cassandra name=cassandra 10.43.254.112 9042/TCP
kube-dns k8s-app=kube-dns,kubernetes.io/cluster-service=true,name=kube-dns k8s-app=kube-dns 10.43.240.10 53/UDP
53/TCP
kubernetes component=apiserver,provider=kubernetes 10.43.240.2 443/TCP
kubernetes-ro component=apiserver,provider=kubernetes 10.43.240.1 80/TCP
## yaml files on https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/cassandra
Thursday, May 21, 2015
first gcloud command learning by example. HOW TO GET PASSWORD AND BEARERTOKEN FROM YOUR CLUSTER
$ gcloud alpha container clusters describe hello-world
clusterApiVersion: 0.17.0
containerIpv4Cidr: 10.40.0.0/14
creationTimestamp: '2015-05-21T09:23:24+00:00'
description: ming cluster 1
enableCloudLogging: true
endpoint: 23.251.150.58
masterAuth:
bearerToken: FRxE88jcslxxxtI
password: FRxE88jcslNxxxPtI
user: admin
name: hello-world
network: default
nodeConfig:
machineType: n1-standard-1
serviceAccounts:
- email: default
scopes:
- https://www.googleapis.com/auth/compute
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
sourceImage: https://www.googleapis.com/compute/v1/projects/google-containers/global/images/container-vm-v20150505
nodeRoutingPrefixSize: 24
numNodes: 1
selfLink: https://www.googleapis.com/container/v1beta1/projects/608411524514/zones/us-central1-a/clusters/hello-world
servicesIpv4Cidr: 10.43.240.0/20
status: running
zone: us-central1-a
Tuesday, January 13, 2015
elasticsearch commands (example) to see the logs from logstash
$ curl -gs -XGET http://localhost:9200/logstash-\*/_search\?pretty\=true
{
"took" : 104,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 6,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD71",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:33 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:33.792Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh3bkHn9XFZmqFmD75",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:56:51 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:56:51.780Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh26y1n9XFZmqFmD70",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:32 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:32.773Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD72",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:34 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:34.817Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD73",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:35 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:35.820Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh28f_n9XFZmqFmD74",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:44 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:44.841Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
} ]
}
}
{
"took" : 104,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 6,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD71",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:33 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:33.792Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh3bkHn9XFZmqFmD75",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:56:51 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:56:51.780Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh26y1n9XFZmqFmD70",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:32 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:32.773Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD72",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:34 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:34.817Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD73",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:35 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:35.820Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh28f_n9XFZmqFmD74",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:44 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:44.841Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
} ]
}
}
Monday, January 12, 2015
Kibana: This version of Kibana requires Elasticsearch 1.4.0 or higher on all nodes. I found the following incompatible nodes in your cluster: Elasticsearch 1.1.1 @ inet[/10.0.2.15:9301] (172.16.255.250)
If your Kibana 4.0 complains with your elasticsearch version 1.1.1. Even though you has install elasticsearch 1.4.2. It's because an elasticsearch running embedded in logstash 1.4.2 (under vendor/jar/elasticearch-1.1.1). You should disable it if you install another elasticsearch 1.4.2 on the same or a different node.
This is a way to verify it (as follow)
root@node1:~# netstat -plant | grep 9300
tcp 0 0 0.0.0.0:9300 0.0.0.0:* LISTEN 18907/java
tcp 0 0 10.0.2.15:48597 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48598 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48602 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48605 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48602 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48599 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48638 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48639 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48628 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48601 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48595 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48604 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48633 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48595 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48593 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48598 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48596 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48593 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48596 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48635 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48594 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48631 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48600 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48632 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48597 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48627 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48629 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48605 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48637 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48630 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48636 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48604 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48601 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48603 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48599 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48594 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48634 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48600 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48603 10.0.2.15:9300 ESTABLISHED 18907/java
tcp6 0 0 10.0.2.15:48639 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48636 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48634 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48633 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48637 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48631 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48628 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48629 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48632 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48627 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48630 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48635 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48638 10.0.2.15:9300 ESTABLISHED 15288/java
root@node1:~# ps wwwp 18907
PID TTY STAT TIME COMMAND
18907 ? SLl 0:30 /usr/bin/java -server -Djava.net.preferIPv4Stack=true -Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml -Xms1201m -Xmx1201m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.pidfile=/usr/local/var/run/node1.pid -Des.foreground=yes -Des.path.home=/usr/local/elasticsearch -cp :/usr/local/elasticsearch/lib/*:/usr/local/elasticsearch/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch
root@node1:~# ps wwwp 15288
PID TTY STAT TIME COMMAND
15288 ? Sl 1:41 /usr/bin/java -server -Xms400M -Xmx1201M -Djava.io.tmpdir=/opt/logstash/server/tmp/ -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/server/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/server/lib /opt/logstash/server/lib/logstash/runner.rb agent -f /opt/logstash/server/etc/conf.d --pluginpath /opt/logstash/server/lib -l /opt/logstash/server/log/logstash.log -w 1
This is a way to verify it (as follow)
root@node1:~# netstat -plant | grep 9300
tcp 0 0 0.0.0.0:9300 0.0.0.0:* LISTEN 18907/java
tcp 0 0 10.0.2.15:48597 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48598 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48602 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48605 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48602 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48599 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48638 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48639 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48628 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48601 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48595 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48604 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48633 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48595 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48593 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48598 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48596 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48593 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48596 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48635 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48594 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48631 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48600 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48632 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48597 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48627 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48629 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48605 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48637 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48630 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48636 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48604 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48601 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48603 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48599 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48594 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48634 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48600 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48603 10.0.2.15:9300 ESTABLISHED 18907/java
tcp6 0 0 10.0.2.15:48639 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48636 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48634 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48633 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48637 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48631 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48628 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48629 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48632 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48627 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48630 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48635 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48638 10.0.2.15:9300 ESTABLISHED 15288/java
root@node1:~# ps wwwp 18907
PID TTY STAT TIME COMMAND
18907 ? SLl 0:30 /usr/bin/java -server -Djava.net.preferIPv4Stack=true -Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml -Xms1201m -Xmx1201m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.pidfile=/usr/local/var/run/node1.pid -Des.foreground=yes -Des.path.home=/usr/local/elasticsearch -cp :/usr/local/elasticsearch/lib/*:/usr/local/elasticsearch/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch
root@node1:~# ps wwwp 15288
PID TTY STAT TIME COMMAND
15288 ? Sl 1:41 /usr/bin/java -server -Xms400M -Xmx1201M -Djava.io.tmpdir=/opt/logstash/server/tmp/ -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/server/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/server/lib /opt/logstash/server/lib/logstash/runner.rb agent -f /opt/logstash/server/etc/conf.d --pluginpath /opt/logstash/server/lib -l /opt/logstash/server/log/logstash.log -w 1
Adding ELK command line examples
ELK (Elasticsearch 1.4.2 + Logstash 1.4.2 + Kibana 4.0)
$ curl http://localhost:9200/_nodes?pretty=true
$ curl http://localhost:9200/_nodes?pretty=true
{
"cluster_name" : "elasticsearch",
"nodes" : {
"VJvlweg1TxWurCHeJAgNzA" : {
"name" : "node1",
"transport_address" : "inet[/10.0.2.15:9300]",
"host" : "node1",
"ip" : "172.16.255.250",
"version" : "1.4.2",
"build" : "927caff",
"http_address" : "inet[/10.0.2.15:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"settings" : {
"index" : {
"mapper" : {
"dynamic" : "true"
}
},
"bootstrap" : {
"mlockall" : "true"
},
"client" : {
"type" : "node"
},
"gateway" : {
"expected_nodes" : "1"
},
"pidfile" : "/usr/local/var/run/node1.pid",
"node" : {
"max_local_storage_nodes" : "1",
"name" : "node1"
},
"http" : {
"port" : "9200"
},
"name" : "node1",
"action" : {
"auto_create_index" : "true",
"disable_delete_all_indices" : "true"
},
"path" : {
"data" : "/usr/local/var/data/elasticsearch",
"home" : "/usr/local/elasticsearch",
"conf" : "/usr/local/etc/elasticsearch",
"logs" : "/usr/local/var/log/elasticsearch"
},
"cloud" : {
"node" : {
"auto_attributes" : "true"
}
},
"config" : "/usr/local/etc/elasticsearch/elasticsearch.yml",
"cluster" : {
"name" : "elasticsearch"
},
"discovery" : {
"zen" : {
"minimum_master_nodes" : "1",
"ping" : {
"multicast" : {
"enabled" : "true"
}
}
}
},
"foreground" : "yes"
},
"os" : {
"refresh_interval_in_millis" : 1000,
"available_processors" : 2,
"cpu" : {
"vendor" : "Intel",
"model" : "Core(TM) i5-4260U CPU @ 1.40GHz",
"mhz" : 2009,
"total_cores" : 2,
"total_sockets" : 1,
"cores_per_socket" : 2,
"cache_size_in_bytes" : 6144
},
"mem" : {
"total_in_bytes" : 2099187712
},
"swap" : {
"total_in_bytes" : 3137335296
}
},
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 18907,
"max_file_descriptors" : 64000,
"mlockall" : true
},
"jvm" : {
"pid" : 18907,
"version" : "1.7.0_65",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.65-b04",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1421130288311,
"mem" : {
"heap_init_in_bytes" : 1259339776,
"heap_max_in_bytes" : 1242955776,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 1242955776
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
},
"thread_pool" : {
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "200"
},
"bench" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"merge" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"bulk" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "50"
},
"optimize" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 6,
"max" : 6,
"queue_size" : "1k"
},
"listener" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"network" : {
"refresh_interval_in_millis" : 5000,
"primary_interface" : {
"address" : "10.0.2.15",
"name" : "eth0",
"mac_address" : "08:00:27:98:DC:AA"
}
},
"transport" : {
"bound_address" : "inet[/0.0.0.0:9300]",
"publish_address" : "inet[/10.0.2.15:9300]"
},
"http" : {
"bound_address" : "inet[/0.0.0.0:9200]",
"publish_address" : "inet[/10.0.2.15:9200]",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ ]
},
"oxvKUB4kTPiYkvAx_jiuRw" : {
"name" : "logstash-node1-15288-4072",
"transport_address" : "inet[/10.0.2.15:9301]",
"host" : "node1",
"ip" : "172.16.255.250",
"version" : "1.1.1",
"build" : "f1585f0",
"attributes" : {
"client" : "true",
"data" : "false"
},
"settings" : {
"path" : {
"logs" : "/opt/logstash/server/logs"
},
"cluster" : {
"name" : "elasticsearch"
},
"node" : {
"client" : "true",
"name" : "logstash-node1-15288-4072"
},
"discovery" : {
"zen" : {
"ping" : {
"unicast" : {
"hosts" : "localhost:9300,localhost:9301,localhost:9302,localhost:9303,localhost:9304,localhost:9305"
},
"multicast" : {
"enabled" : "false"
}
}
}
},
"http" : {
"enabled" : "false"
},
"name" : "logstash-node1-15288-4072"
},
"os" : {
"refresh_interval_in_millis" : 1000,
"available_processors" : 2
},
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 15288,
"max_file_descriptors" : 65550,
"mlockall" : false
},
"jvm" : {
"pid" : 15288,
"version" : "1.7.0_65",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.65-b04",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1421127217161,
"mem" : {
"heap_init_in_bytes" : 419430400,
"heap_max_in_bytes" : 506855424,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 506855424
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
},
"thread_pool" : {
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "200"
},
"get" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"merge" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"bulk" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "50"
},
"optimize" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 6,
"max" : 6,
"queue_size" : "1k"
},
"percolate" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"network" : {
"refresh_interval_in_millis" : 5000,
"primary_interface" : {
"address" : "",
"name" : "",
"mac_address" : ""
}
},
"transport" : {
"bound_address" : "inet[/0:0:0:0:0:0:0:0%0:9301]",
"publish_address" : "inet[/10.0.2.15:9301]"
},
"plugins" : [ ]
}
}
}
Thursday, September 25, 2014
CVE-2014-6271, CVE-2014-7169 Bash Vulnerability
more on https://shellshocker.net/
CVE-2014-6271 remote execution code through bash.
CVE-2014-7169 incomplete fix for CVE-2014-6271
- Ubuntu : http://people.canonical.com/~ubuntu-security/cve/2014/CVE-2014-7169.html (4.2-2ubuntu2.3, 4.3-7ubuntu1.2)
- Red Hat :https://access.redhat.com/security/cve/CVE-2014-7169
- Red Hat: https://access.redhat.com/articles/1200223
- Debian : https://security-tracker.debian.org/tracker/CVE-2014-7169
- CentOS: https://www.conoha.jp/blog/tech/3490.html (Japanese)
- CoreOS: https://coreos.com/blog/security-update-shellshock/
- Mac OS X: http://showterm.io/c96936f54f88c4475737d (need XCode)
FOR UBUNTU:
(9/26/2014) Ubuntu patch for CVE-2014-7169
for ubuntu, just use following command to upgrade bash
$ sudo apt-get update && sudo apt-get install bash
## check your bash version with
$ dpkg -s bash | grep Version
Version: 4.2-2ubuntu2.3
or
$ dpkg -s bash | grep Version
Version: 4.3-7ubuntu1.2
if your bash is patched, you will see no "echo" for 'vulnerable' as bellow;
$
env x='() { :;}; echo vulnerable' bash -c "echo this is a test"bash: warning: x: ignoring function definition attempt bash: error importing function definition for `x' this is a test
FOR CENTOS
CentOS7 bash-4.2.45-5.el7_0.4 CentOS6 bash-4.1.2-15.el6_5.2 CentOS5 bash-3.2-33.el5_11.4
Tuesday, August 19, 2014
after you install openstack on Ubuntu 14.04 server, you can use following nova commands
root@ssd:~# nova --os-username=admin --os-tenant-name=admin --os-auth-url="http://10.0.1.100:5000/v2.0/" usage
OS Password:
Usage from 2014-07-23 to 2014-08-21:
+---------+--------------+-----------+---------------+
| Servers | RAM MB-Hours | CPU Hours | Disk GB-Hours |
+---------+--------------+-----------+---------------+
| 2 | 556.80 | 1.09 | 1.09 |
+---------+--------------+-----------+---------------+
root@ssd:~# nova --os-username=admin --os-tenant-name=admin --os-auth-url="http://10.0.1.100:5000/v2.0/" --version
2.17.0
OS Password:
Usage from 2014-07-23 to 2014-08-21:
+---------+--------------+-----------+---------------+
| Servers | RAM MB-Hours | CPU Hours | Disk GB-Hours |
+---------+--------------+-----------+---------------+
| 2 | 556.80 | 1.09 | 1.09 |
+---------+--------------+-----------+---------------+
root@ssd:~# nova --os-username=admin --os-tenant-name=admin --os-auth-url="http://10.0.1.100:5000/v2.0/" --version
2.17.0
Tuesday, August 12, 2014
no password for CoreOS to login as username: core (on AWS EC2)
ming➜~/.ec2» ssh -i turboteam.pem root@54.64.18.203 [16:45:05]
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Connection closed by 54.64.18.203
ming➜~/.ec2» ssh -i turboteam.pem root@54.64.18.203 [16:54:17]
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Permission denied (publickey,password,keyboard-interactive).
ming➜~/.ec2» [16:54:27]
ming➜~/.ec2» ssh -i turboteam.pem root@54.64.18.203 [16:54:28]
root@54.64.18.203's password:
ming➜~/.ec2» ssh -i turboteam.pem core@54.64.18.203 [16:54:45]
CoreOS (beta)
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Connection closed by 54.64.18.203
ming➜~/.ec2» ssh -i turboteam.pem root@54.64.18.203 [16:54:17]
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Permission denied, please try again.
root@54.64.18.203's password:
Permission denied (publickey,password,keyboard-interactive).
ming➜~/.ec2» [16:54:27]
ming➜~/.ec2» ssh -i turboteam.pem root@54.64.18.203 [16:54:28]
root@54.64.18.203's password:
ming➜~/.ec2» ssh -i turboteam.pem core@54.64.18.203 [16:54:45]
CoreOS (beta)
A couple ways to launch an EC2 on AWS from remote.
- Vagrant (https://github.com/mitchellh/vagrant-aws)
- Apache Mesos (https://elastic.mesosphere.io/) 3-steps.
- Vagrant + Mesos (https://github.com/everpeace/vagrant-mesos#sec2)
- using jcloud library (https://jclouds.apache.org/guides/aws-ec2/)
- ???
Thursday, July 31, 2014
install VirtualBox on uBuntu server (no test on AWS EC2)
sudo sh -c "echo 'deb http://download.virtualbox.org/virtualbox/debian '$(lsb_release -cs)' contrib non-free' > /etc/apt/sources.list.d/virtualbox.list" && wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add - && sudo apt-get update && sudo apt-get install virtualbox-4.3 dkms
Wednesday, July 23, 2014
how to install docker on AWS EC2 CentOS instance.
<You can still despise git within a year, but not docker.>
sudo yum -y install docker-io
sudo service docker start
sudo chkconfig docker ontest to create an ubuntu container
sudo docker run -it ubuntu sh
<demo>
[ec2-user@ip-172-31-27-45 ~]$ sudo yum -y install docker-io
Loaded plugins: priorities, update-motd, upgrade-helper
amzn-main/latest | 2.1 kB 00:00
amzn-updates/latest | 2.3 kB 00:00
mongodb | 951 B 00:00
Package docker-1.0.0-1.15.amzn1.x86_64 already installed and latest version
Nothing to do
[ec2-user@ip-172-31-27-45 ~]$ sudo service docker start
Starting cgconfig service: [ OK ]
Starting docker: [ OK ]
[ec2-user@ip-172-31-27-45 ~]$ sudo chkconfig docker on
[ec2-user@ip-172-31-27-45 ~]$ sudo docker run -it ubuntu sh
Unable to find image 'ubuntu' locally
Pulling repository ubuntu
ba5877dc9bec: Download complete
511136ea3c5a: Download complete
9bad880da3d2: Download complete
25f11f5fb0cb: Download complete
ebc34468f71d: Download complete
2318d26665ef: Download complete
# uname -a
Linux d5d4e7e0ea4e 3.10.38-49.136.amzn1.x86_64 #1 SMP Mon May 5 18:28:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/docker-202:1-147559-d5d4e7e0ea4e50bff40e7e53a4106e2c238066811bbebbe86020f1c7cce987dd 10190136 228364 9421100 3% /
tmpfs 303460 0 303460 0% /dev
shm 65536 0 65536 0% /dev/shm
/dev/xvda1 8125880 7426032 599580 93% /etc/hosts
tmpfs 303460 0 303460 0% /proc/kcore
# exit
[ec2-user@ip-172-31-27-45 ~]$ uname -a
Linux ip-172-31-27-45 3.10.38-49.136.amzn1.x86_64 #1 SMP Mon May 5 18:28:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[ec2-user@ip-172-31-27-45 ~]$ df
df: ‘/var/lib/docker/devicemapper’: Permission denied
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 8125880 7426012 599600 93% /
devtmpfs 287060 52 287008 1% /dev
tmpfs 303460 0 303460 0% /dev/shm
Thursday, July 3, 2014
Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
MongoDB shell version: 2.6.3connecting to: test2014-07-01T15:34:46.151+0800 warning: Failed to connect to 127.0.0.1:27017, reason: errno:113 No route to host2014-07-01T15:34:46.153+0800 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146exception: connect failedSOLUTION:sudo iptables -I INPUT -p tcp -m state --state NEW,ESTABLISHED --dport 27017 -j ACCEPT
Subscribe to:
Posts (Atom)