I have Logstash and Elasticsearch running on an EC2 (8GB ram), ArchLinux.
I can not clear the cached memory (I understand kernel should be able to claim this memory back if needed). from the meminfo, I understand that the most of the cached memory is inactive, which means no process is using that,so I should be able to clear it, right?.
I am doing sync && echo 3 > /proc/sys/vm/drop_caches to clear the memory. Only system reboot is clearing this memory. There could be a memory leak, how can I see which process is using this cache memory.
cat /proc/meminfo
MemTotal: 7661268 kB
MemFree: 613144 kB
MemAvailable: 561144 kB
Buffers: 8580 kB
Cached: 2271144 kB
SwapCached: 0 kB
Active: 165424 kB
Inactive: 2244356 kB
Active(anon): 152528 kB
Inactive(anon): 2226088 kB
Active(file): 12896 kB
Inactive(file): 18268 kB
Unevictable: 4429248 kB
Mlocked: 4429248 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 164 kB
Writeback: 0 kB
AnonPages: 4559304 kB
Mapped: 33916 kB
Shmem: 2226220 kB
Slab: 54908 kB
SReclaimable: 33764 kB
SUnreclaim: 21144 kB
KernelStack: 1176 kB
PageTables: 11832 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 3830632 kB
Committed_AS: 6861504 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 19132 kB
VmallocChunk: 34359707084 kB
HardwareCorrupted: 0 kB
AnonHugePages: 4329472 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 10240 kB
DirectMap2M: 7985152 kB
Output of a top:
top - 06:04:57 up 16 days, 17:12, 1 user, load average: 0.00, 0.04, 1.39
Tasks: 76 total, 1 running, 75 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.2 us, 0.3 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 7661268 total, 7051140 used, 610128 free, 9772 buffers
KiB Swap: 0 total, 0 used, 0 free. 2272468 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19958 elastic+ 20 0 9746648 4.273g 22760 S 0.3 58.5 1:11.36 java
19970 root 20 0 1008496 89912 3876 S 0.3 1.2 0:07.13 node
19932 root 20 0 28256 6728 6496 S 0.0 0.1 0:01.29 systemd-journal
1 root 20 0 110052 3316 1596 S 0.0 0.0 1:45.84 systemd
19856 root 20 0 16136 2100 1460 S 0.0 0.0 0:02.05 bash
19851 root 20 0 82472 2040 1136 S 0.0 0.0 0:01.39 sshd
245 haproxy 20 0 28872 1996 624 S 0.0 0.0 4:01.48 haproxy
19854 root 20 0 157700 1980 0 S 0.0 0.0 0:00.00 (sd-pam)
244 haproxy 20 0 28596 1728 768 S 0.0 0.0 0:00.00 haproxy
221 systemd+ 20 0 23496 1476 1056 S 0.0 0.0 0:04.99 systemd-network
20197 root 20 0 17544 1464 1064 R 0.0 0.0 0:00.02 top
149 root 20 0 31016 1456 768 S 0.0 0.0 0:00.10 systemd-udevd
233 root 20 0 39980 1448 860 S 0.0 0.0 0:01.03 sshd
217 root 20 0 13204 1344 796 S 0.0 0.0 0:02.55 crond
219 dbus 20 0 24748 1236 900 S 0.0 0.0 0:18.18 dbus-daemon
229 systemd+ 20 0 18164 1196 980 S 0.0 0.0 0:00.92 systemd-resolve
19853 root 20 0 26764 1168 700 S 0.0 0.0 0:00.01 systemd
218 root 20 0 15252 1144 876 S 0.0 0.0 0:07.26 systemd-logind
194 root 16 -4 105484 1136 764 S 0.0 0.0 0:03.15 auditd
214 root 20 0 19240 1024 776 S 0.0 0.0 1:08.62 irqbalance
226 root 20 0 8124 820 700 S 0.0 0.0 0:00.00 agetty
225 root 20 0 8124 816 700 S 0.0 0.0 0:00.00 agetty
224 root 20 0 8124 792 680 S 0.0 0.0 0:00.01 agetty
242 root 20 0 23896 624 292 S 0.0 0.0 0:00.00 haproxy-systemd
213 root 20 0 9324 612 420 S 0.0 0.0 9:44.04 rngd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.16 kthreadd
3 root 20 0 0 0 0 S 0.7 0.0 65:18.76 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
7 root 20 0 0 0 0 S 0.0 0.0 10:08.87 rcu_sched
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.36 migration/0
10 root rt 0 0 0 0 S 0.0 0.0 0:01.17 watchdog/0
11 root rt 0 0 0 0 S 0.0 0.0 0:00.91 watchdog/1
12 root rt 0 0 0 0 S 0.0 0.0 0:00.37 migration/1
