Adding cluster nodes, expanding storage - Elasticsearch
This is a copy/paste/censor of our wiki doc (that I wrote, and have permission to publish; the censoring might make stuff look off, sorry). I left in the generic stuff because maybe we're doing something horribly wrong and a nice person will point it out. Or maybe we're doing something that someone else will find inspiration in. Who knows! Gives some context anyways. Hope this helps someone...
Overview of adding a new node to the ES cluster.
1. Deploy the current CentOS template
2. Assign an ip address from IPadmin
3. Create the A record in AD DNS now
4. Add a 500GB disk, located on one of the VMFS-ES-x datastores
5. Change the networking:
§ /etc/sysconfig/network-scripts/ifcfg-eth0 (change IP)
§ /etc/sysconfig/network (change hostname)
6. Run updates: yum update
ES Big Disk config
If VM is already in place
This will allow you to add the disk without rebooting:
Continue the disk setup
New node - add plugins
Set the heap size
At this point, just watch for errors, it should join and be happy.
Issues w. plugins loading
If you have timeouts loading plugins, but the cluster health is ok - it's most likely a client-side thing (i.e. cache). Had some problems with Chrome that magically cleared...