Friday, October 2, 2015
"Generating Go from JSON data" idea from me (ming-der wang) is already public and start implement since Aug. 3, 2015 (at least as early)
I claim the following idea is already on public since I had share or public in any social networks.
"Generating Go from JSON data" is the key concept of gogetDB or my current database as service project (onion, demo-onion, ginger, pepper, chilli, and so on.)
since 2 month ago (10/1/2015) of https://hackpad.com/Project-2-Product-Description-9iPxzxUeewK
all source code are in github (before Oct. 1, 2015), start not to open in additional parts.
the concept is implemented and demo in https://prezi.com/0lrgdih96_aw/copy-of-932015-gcpugtw-free-talk-kubernetes/ on Sept. 3, 2015
also public the way to do it in https://hackpad.com/GCPUG.TW-932015-live-demo-mGYQk3BWynb
The presentation of this concept actually in Sept. 3, 2015 NTU hackathon, I already present it to public.
Also named it "Give me JSON, I will give you an EndPoint (back)"
All copyright belong to Ming-der Wang. log4analytics.com log4security.com
Friday, May 22, 2015
run your docker on google container engine.
Better reading on gist gist
$ kubectl config use-context gke_iron-potion-92209_us-central1-a_hello-world
$ kubectl create -f casandra-pod.yaml
$ kubectl create -f cassandra-service.yaml
$ kubectl get endpoints cassandra -o yaml
apiVersion: v1beta3
kind: Endpoints
metadata:
creationTimestamp: 2015-05-22T07:30:24Z
name: cassandra
namespace: default
resourceVersion: "53898"
selfLink: /api/v1beta1/endpoints/cassandra?namespace=default
uid: 694655e9-0054-11e5-815a-42010af013e9
subsets:
- addresses:
- IP: 10.40.1.5
targetRef:
kind: Pod
name: cassandra
namespace: default
resourceVersion: "53826"
uid: 2e9b73bb-0054-11e5-815a-42010af013e9
ports:
- port: 9042
protocol: TCP
$ kubectl get services
NAME LABELS SELECTOR IP PORT(S)
cassandra name=cassandra name=cassandra 10.43.254.112 9042/TCP
kube-dns k8s-app=kube-dns,kubernetes.io/cluster-service=true,name=kube-dns k8s-app=kube-dns 10.43.240.10 53/UDP
53/TCP
kubernetes component=apiserver,provider=kubernetes 10.43.240.2 443/TCP
kubernetes-ro component=apiserver,provider=kubernetes 10.43.240.1 80/TCP
## yaml files on https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/cassandra
Thursday, May 21, 2015
first gcloud command learning by example. HOW TO GET PASSWORD AND BEARERTOKEN FROM YOUR CLUSTER
$ gcloud alpha container clusters describe hello-world
clusterApiVersion: 0.17.0
containerIpv4Cidr: 10.40.0.0/14
creationTimestamp: '2015-05-21T09:23:24+00:00'
description: ming cluster 1
enableCloudLogging: true
endpoint: 23.251.150.58
masterAuth:
bearerToken: FRxE88jcslxxxtI
password: FRxE88jcslNxxxPtI
user: admin
name: hello-world
network: default
nodeConfig:
machineType: n1-standard-1
serviceAccounts:
- email: default
scopes:
- https://www.googleapis.com/auth/compute
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
sourceImage: https://www.googleapis.com/compute/v1/projects/google-containers/global/images/container-vm-v20150505
nodeRoutingPrefixSize: 24
numNodes: 1
selfLink: https://www.googleapis.com/container/v1beta1/projects/608411524514/zones/us-central1-a/clusters/hello-world
servicesIpv4Cidr: 10.43.240.0/20
status: running
zone: us-central1-a
Tuesday, January 13, 2015
elasticsearch commands (example) to see the logs from logstash
$ curl -gs -XGET http://localhost:9200/logstash-\*/_search\?pretty\=true
{
"took" : 104,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 6,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD71",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:33 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:33.792Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh3bkHn9XFZmqFmD75",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:56:51 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:56:51.780Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh26y1n9XFZmqFmD70",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:32 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:32.773Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD72",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:34 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:34.817Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD73",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:35 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:35.820Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh28f_n9XFZmqFmD74",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:44 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:44.841Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
} ]
}
}
{
"took" : 104,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 6,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD71",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:33 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:33.792Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh3bkHn9XFZmqFmD75",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:56:51 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:56:51.780Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh26y1n9XFZmqFmD70",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:32 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:32.773Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD72",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:34 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:34.817Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh27PSn9XFZmqFmD73",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:35 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:35.820Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
}, {
"_index" : "logstash-2015.01.13",
"_type" : "nginx_access",
"_id" : "AUrh28f_n9XFZmqFmD74",
"_score" : 1.0,
"_source":{"message":"172.16.255.1 - - [13/Jan/2015:05:54:44 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0\"","@version":"1","@timestamp":"2015-01-13T05:54:44.841Z","host":"node1","path":"/var/log/nginx/access.log","type":"nginx_access","tags":["_grokparsefailure"]}
} ]
}
}
Monday, January 12, 2015
Kibana: This version of Kibana requires Elasticsearch 1.4.0 or higher on all nodes. I found the following incompatible nodes in your cluster: Elasticsearch 1.1.1 @ inet[/10.0.2.15:9301] (172.16.255.250)
If your Kibana 4.0 complains with your elasticsearch version 1.1.1. Even though you has install elasticsearch 1.4.2. It's because an elasticsearch running embedded in logstash 1.4.2 (under vendor/jar/elasticearch-1.1.1). You should disable it if you install another elasticsearch 1.4.2 on the same or a different node.
This is a way to verify it (as follow)
root@node1:~# netstat -plant | grep 9300
tcp 0 0 0.0.0.0:9300 0.0.0.0:* LISTEN 18907/java
tcp 0 0 10.0.2.15:48597 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48598 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48602 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48605 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48602 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48599 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48638 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48639 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48628 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48601 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48595 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48604 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48633 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48595 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48593 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48598 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48596 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48593 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48596 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48635 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48594 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48631 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48600 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48632 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48597 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48627 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48629 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48605 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48637 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48630 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48636 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48604 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48601 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48603 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48599 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48594 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48634 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48600 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48603 10.0.2.15:9300 ESTABLISHED 18907/java
tcp6 0 0 10.0.2.15:48639 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48636 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48634 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48633 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48637 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48631 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48628 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48629 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48632 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48627 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48630 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48635 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48638 10.0.2.15:9300 ESTABLISHED 15288/java
root@node1:~# ps wwwp 18907
PID TTY STAT TIME COMMAND
18907 ? SLl 0:30 /usr/bin/java -server -Djava.net.preferIPv4Stack=true -Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml -Xms1201m -Xmx1201m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.pidfile=/usr/local/var/run/node1.pid -Des.foreground=yes -Des.path.home=/usr/local/elasticsearch -cp :/usr/local/elasticsearch/lib/*:/usr/local/elasticsearch/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch
root@node1:~# ps wwwp 15288
PID TTY STAT TIME COMMAND
15288 ? Sl 1:41 /usr/bin/java -server -Xms400M -Xmx1201M -Djava.io.tmpdir=/opt/logstash/server/tmp/ -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/server/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/server/lib /opt/logstash/server/lib/logstash/runner.rb agent -f /opt/logstash/server/etc/conf.d --pluginpath /opt/logstash/server/lib -l /opt/logstash/server/log/logstash.log -w 1
This is a way to verify it (as follow)
root@node1:~# netstat -plant | grep 9300
tcp 0 0 0.0.0.0:9300 0.0.0.0:* LISTEN 18907/java
tcp 0 0 10.0.2.15:48597 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48598 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48602 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48605 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48602 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48599 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48638 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48639 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48628 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48601 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48595 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48604 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48633 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48595 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48593 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48598 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48596 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48593 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48596 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48635 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48594 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48631 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48600 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48632 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48597 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48627 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48629 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48605 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48637 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48630 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48636 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48604 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48601 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48603 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48599 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48594 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:9300 10.0.2.15:48634 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48600 10.0.2.15:9300 ESTABLISHED 18907/java
tcp 0 0 10.0.2.15:48603 10.0.2.15:9300 ESTABLISHED 18907/java
tcp6 0 0 10.0.2.15:48639 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48636 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48634 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48633 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48637 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48631 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48628 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48629 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48632 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48627 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48630 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48635 10.0.2.15:9300 ESTABLISHED 15288/java
tcp6 0 0 10.0.2.15:48638 10.0.2.15:9300 ESTABLISHED 15288/java
root@node1:~# ps wwwp 18907
PID TTY STAT TIME COMMAND
18907 ? SLl 0:30 /usr/bin/java -server -Djava.net.preferIPv4Stack=true -Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml -Xms1201m -Xmx1201m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.pidfile=/usr/local/var/run/node1.pid -Des.foreground=yes -Des.path.home=/usr/local/elasticsearch -cp :/usr/local/elasticsearch/lib/*:/usr/local/elasticsearch/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch
root@node1:~# ps wwwp 15288
PID TTY STAT TIME COMMAND
15288 ? Sl 1:41 /usr/bin/java -server -Xms400M -Xmx1201M -Djava.io.tmpdir=/opt/logstash/server/tmp/ -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/server/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/server/lib /opt/logstash/server/lib/logstash/runner.rb agent -f /opt/logstash/server/etc/conf.d --pluginpath /opt/logstash/server/lib -l /opt/logstash/server/log/logstash.log -w 1
Adding ELK command line examples
ELK (Elasticsearch 1.4.2 + Logstash 1.4.2 + Kibana 4.0)
$ curl http://localhost:9200/_nodes?pretty=true
$ curl http://localhost:9200/_nodes?pretty=true
{
"cluster_name" : "elasticsearch",
"nodes" : {
"VJvlweg1TxWurCHeJAgNzA" : {
"name" : "node1",
"transport_address" : "inet[/10.0.2.15:9300]",
"host" : "node1",
"ip" : "172.16.255.250",
"version" : "1.4.2",
"build" : "927caff",
"http_address" : "inet[/10.0.2.15:9200]",
"attributes" : {
"max_local_storage_nodes" : "1"
},
"settings" : {
"index" : {
"mapper" : {
"dynamic" : "true"
}
},
"bootstrap" : {
"mlockall" : "true"
},
"client" : {
"type" : "node"
},
"gateway" : {
"expected_nodes" : "1"
},
"pidfile" : "/usr/local/var/run/node1.pid",
"node" : {
"max_local_storage_nodes" : "1",
"name" : "node1"
},
"http" : {
"port" : "9200"
},
"name" : "node1",
"action" : {
"auto_create_index" : "true",
"disable_delete_all_indices" : "true"
},
"path" : {
"data" : "/usr/local/var/data/elasticsearch",
"home" : "/usr/local/elasticsearch",
"conf" : "/usr/local/etc/elasticsearch",
"logs" : "/usr/local/var/log/elasticsearch"
},
"cloud" : {
"node" : {
"auto_attributes" : "true"
}
},
"config" : "/usr/local/etc/elasticsearch/elasticsearch.yml",
"cluster" : {
"name" : "elasticsearch"
},
"discovery" : {
"zen" : {
"minimum_master_nodes" : "1",
"ping" : {
"multicast" : {
"enabled" : "true"
}
}
}
},
"foreground" : "yes"
},
"os" : {
"refresh_interval_in_millis" : 1000,
"available_processors" : 2,
"cpu" : {
"vendor" : "Intel",
"model" : "Core(TM) i5-4260U CPU @ 1.40GHz",
"mhz" : 2009,
"total_cores" : 2,
"total_sockets" : 1,
"cores_per_socket" : 2,
"cache_size_in_bytes" : 6144
},
"mem" : {
"total_in_bytes" : 2099187712
},
"swap" : {
"total_in_bytes" : 3137335296
}
},
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 18907,
"max_file_descriptors" : 64000,
"mlockall" : true
},
"jvm" : {
"pid" : 18907,
"version" : "1.7.0_65",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.65-b04",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1421130288311,
"mem" : {
"heap_init_in_bytes" : 1259339776,
"heap_max_in_bytes" : 1242955776,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 1242955776
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
},
"thread_pool" : {
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "200"
},
"bench" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"get" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"merge" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"bulk" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "50"
},
"optimize" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 6,
"max" : 6,
"queue_size" : "1k"
},
"listener" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"percolate" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"network" : {
"refresh_interval_in_millis" : 5000,
"primary_interface" : {
"address" : "10.0.2.15",
"name" : "eth0",
"mac_address" : "08:00:27:98:DC:AA"
}
},
"transport" : {
"bound_address" : "inet[/0.0.0.0:9300]",
"publish_address" : "inet[/10.0.2.15:9300]"
},
"http" : {
"bound_address" : "inet[/0.0.0.0:9200]",
"publish_address" : "inet[/10.0.2.15:9200]",
"max_content_length_in_bytes" : 104857600
},
"plugins" : [ ]
},
"oxvKUB4kTPiYkvAx_jiuRw" : {
"name" : "logstash-node1-15288-4072",
"transport_address" : "inet[/10.0.2.15:9301]",
"host" : "node1",
"ip" : "172.16.255.250",
"version" : "1.1.1",
"build" : "f1585f0",
"attributes" : {
"client" : "true",
"data" : "false"
},
"settings" : {
"path" : {
"logs" : "/opt/logstash/server/logs"
},
"cluster" : {
"name" : "elasticsearch"
},
"node" : {
"client" : "true",
"name" : "logstash-node1-15288-4072"
},
"discovery" : {
"zen" : {
"ping" : {
"unicast" : {
"hosts" : "localhost:9300,localhost:9301,localhost:9302,localhost:9303,localhost:9304,localhost:9305"
},
"multicast" : {
"enabled" : "false"
}
}
}
},
"http" : {
"enabled" : "false"
},
"name" : "logstash-node1-15288-4072"
},
"os" : {
"refresh_interval_in_millis" : 1000,
"available_processors" : 2
},
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 15288,
"max_file_descriptors" : 65550,
"mlockall" : false
},
"jvm" : {
"pid" : 15288,
"version" : "1.7.0_65",
"vm_name" : "OpenJDK 64-Bit Server VM",
"vm_version" : "24.65-b04",
"vm_vendor" : "Oracle Corporation",
"start_time_in_millis" : 1421127217161,
"mem" : {
"heap_init_in_bytes" : 419430400,
"heap_max_in_bytes" : 506855424,
"non_heap_init_in_bytes" : 24313856,
"non_heap_max_in_bytes" : 224395264,
"direct_max_in_bytes" : 506855424
},
"gc_collectors" : [ "ParNew", "ConcurrentMarkSweep" ],
"memory_pools" : [ "Code Cache", "Par Eden Space", "Par Survivor Space", "CMS Old Gen", "CMS Perm Gen" ]
},
"thread_pool" : {
"generic" : {
"type" : "cached",
"keep_alive" : "30s",
"queue_size" : -1
},
"index" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "200"
},
"get" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"snapshot" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"merge" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"suggest" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"bulk" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "50"
},
"optimize" : {
"type" : "fixed",
"min" : 1,
"max" : 1,
"queue_size" : -1
},
"warmer" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"flush" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
},
"search" : {
"type" : "fixed",
"min" : 6,
"max" : 6,
"queue_size" : "1k"
},
"percolate" : {
"type" : "fixed",
"min" : 2,
"max" : 2,
"queue_size" : "1k"
},
"management" : {
"type" : "scaling",
"min" : 1,
"max" : 5,
"keep_alive" : "5m",
"queue_size" : -1
},
"refresh" : {
"type" : "scaling",
"min" : 1,
"max" : 1,
"keep_alive" : "5m",
"queue_size" : -1
}
},
"network" : {
"refresh_interval_in_millis" : 5000,
"primary_interface" : {
"address" : "",
"name" : "",
"mac_address" : ""
}
},
"transport" : {
"bound_address" : "inet[/0:0:0:0:0:0:0:0%0:9301]",
"publish_address" : "inet[/10.0.2.15:9301]"
},
"plugins" : [ ]
}
}
}
Subscribe to:
Comments (Atom)