Bellow design is to monitor 6 application services ( Services A, B, C, D, E,F ) distributed via 6 virtual machines and each service is load balanced via 2 nodes.
We will be able to monitor below.
1. JVM Memory, Thread and CPU utilization graphs
2. API tracing and monitoring
3. API hit rate, success rate, error rates
Figure 1 - Integrating Apache Skywalker APM tool
1. Download latest Apache Skywalking : https://skywalking.apache.org/downloads/
2. Install Apache Skywalking Tool in your preferred VM node.
3. Open webapp.yml in the folder called "webapp" in the installation directory
4. Change the skywalker web-ui startup port accordingly. I have changed the port to 9090
server:
port: 9090
collector:
path: /graphql
ribbon:
ReadTimeout: 10000
# Point to all backend's restHost:restPort, split by ,
listOfServers: 127.0.0.1:12800
5. Make sure latest Java is installed in the VM where we going to run Skywalker tool
6. Start the Skywalker by below command.
cd bin
./startup.sh
7. Verify Skywalker is up and running. http://localhost:9090/
Figure 2 - Apache Skywalker is up and running.
8. Now we have to configure Skywalker agent in each and every distributed services.
You will be able to find folder called "agent" in installation location.
9. Copy the folder "agent" to each and every VM node.
Ex : /data/skywalker/client/agent
10. This client agents need to be configured in each and every app server.
Client agents will send services statistics to Skywalker server that we have installed.
11. Open agent.config file located in /data/skywalker/client/agent/config
12. Change the agent config properties in order to mention service group names.
Skywalker web-ui will group service statistics based on these naming conversion.
In Node 1
agent.service_name=${SW_AGENT_NAME:node-1}
agent.sample_n_per_3_secs=10
In Node 2
agent.service_name=${SW_AGENT_NAME:node-2}
agent.sample_n_per_3_secs=10
In Node 3
agent.service_name=${SW_AGENT_NAME:node-3}
agent.sample_n_per_3_secs=10
In Node 4
agent.service_name=${SW_AGENT_NAME:node-4}
agent.sample_n_per_3_secs=10
In Node 5
agent.service_name=${SW_AGENT_NAME:node-5}
agent.sample_n_per_3_secs=10
In Node 6
agent.service_name=${SW_AGENT_NAME:node-6}
agent.sample_n_per_3_secs=10
12. Now we have to configure application servers ( I am using Tomcat servers )
with Skywalker client library. Once servers are started, then Skywalker clients will start
sending services statistics. Integrate skywalking-agent.jar in each Tomcat server by adding
bellow configurations to <Tomcat-Server-Path>/bin/catalina.sh
In Node 1
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/data/skywalker/client/agent/skywalking-agent.jar"
export CATALINA_OPTS
In Node 2
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/data/skywalker/client/agent/skywalking-agent.jar"
export CATALINA_OPTS
In Node 3
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/data/skywalker/client/agent/skywalking-agent.jar"
export CATALINA_OPTS
In Node 4
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/data/skywalker/client/agent/skywalking-agent.jar"
export CATALINA_OPTS
In Node 5
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/data/skywalker/client/agent/skywalking-agent.jar"
export CATALINA_OPTS
In Node 6
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/data/skywalker/client/agent/skywalking-agent.jar"
export CATALINA_OPTS
13. Now start each and every tomcat and start sending request to each and every services deployed in each tomcat
14. Set monitoring date range in Skywaling web-ui. http://localhost:9090/
Figure 3 - Apache Skywalker date range selection
15. View JVM Memory, CPU and Thread utilization over the time
Go to dashboard
Select instances
Select Current Service ( The Node name given to VM )
Select Current Instance
Figure 4 - JVM Memory / JVM CPU / JVM Thread utilization
Superb..!
ReplyDelete