| No comment yet

Controlling linux/unix server load


Load spikes are results of some resource consuming processes running at that instant. Locating
that process, stopping it and preventing it from happening again it is the key to maintain a healthy load.
These resource consumers can be due to the following and many more:

•scripts being run in background by any user
•incoming spams
•mass mailing (outgoing) from our server
•mysql databases consuming resources
•high apache traffic

How to locate them? Use command tools such as:

$top -c
$ps -aux
$mysqladmin process
prints mysql databases that are being queried
$netstat -plan
prints out all network connections and from there one can grep out and count the no: of
connections per service/port
$/usr/local/cpanel/bin/dcpumonview
in cPanel servers will list each users with cpu usage
Once you figured out program is causing load, try restarting the associated service,
Eg:- From 'top -c' it will list the processes that are having high cpu usage. If that process is say httpd,
try restarting the web service
service httpd restart
If this doesn't control it, locate which user in the server is using that apache traffic or from which IP
most of the apache connections are coming from, you can get that count from each IP but customizing
'netstat -plan' output slightly
If 'top -c' shows it is mysql that is taking load, next step is to determine which database or user is that,
you can have a idea of that from 'mysqladmin process'
If 'top -c' shows a third party perl or bash script running, just locate the PID and kill it.
If it is the mail service in 'top -c', do a 'tail -f /var/log/maillog', you should be able to see if there is a lot
of incoming mails or outgoing ones. If it is incoming ones from a single location, block that IP in
firewall, if outgoing ones, one will have to find out which user or program in the server is sending out
mails.
Please note that locating the exact reason behind any load spike is important, but there can't be single
best fix that can be used in all such cases. A 'Fix' should be determined by considering many things
like the server nature, its clients etc..
For eg:-, disabling a database that is consuming load in a shared server should be ok, but doing the
same in a dedicated server that runs this single one database does no good.

| No comment yet

Securing cPanel server

There is no guarantee you can make your server 100% secure. But you may prevent the worst of if pay attention to following 
I have earlier posted tips on securing web server methods in general.All the above Securing methods are applicable in cPanel servers as well, however, cPanel being

a much more common panel for webhosting, there is a few additional tips as below:

• securing /tmp
   just use the script /scripts/securetmp

• RDNS
   Since cPanel is a mail server as well, make sure that the mail outgoing IP has a RDNS set. It
  will prevent the server IP from being blacklisted by third party spam controllers.

• Spf records and domainkeys.
   These if added will improve the email authentication for outgoing mails.

• exim extended logging
   This adds valuable logging information to your exim_mainlog file so that you can determine
  where messages are coming from, who's sending the message and from what directory in server
 the mails are originating from, if your seeing mail leaving as nobody. In addition, it adds very
useful information to exim_mainlog to help you decipher email coming and going.
Just add the below to the exim.conf
log_selector = +address_rewrite +all_parents +arguments +connection_reject +delay_delivery
+delivery_size +dnslist_defer +incoming_interface +incoming_port +lost_incoming_connection
+queue_run +received_sender +received_recipients +retry_defer +sender_on_delivery +size_reject
+skip_delivery +smtp_confirmation +smtp_connection +smtp_protocol_error +smtp_syntax_error
+subject +tls_cipher +tls_peerdn

Enable SuPHP
Enable open_basedir
Disable system compilers
Enable cphulk protection
Enable sell fork bomb protection
Limit outgoing mails for accounts
in WHM > Tweak settings, there are many options which can be enabled/disabled depending
upon the server nature.

| No comment yet

Server securing methods

On the top of everthing when it comes to setting up webserver our priority would be Security.Here I will describe the basic steps to secure server.

Strong password
Always keep a strong password for servers, don't use any default ones or simple dictionary
words. Also make it a habit to change the password frequently.

Change ssh port
Change it to anything other than default 22, so that hackers keep guessing. Alter the option 'Port'
in sshd_config file.
Eg:- Port 24825

Denying root user login for ssh
Use 'PermitRootLogin no' in sshd_config. Now direct root login via ssh is denied. Instead just
create a normal user say 'admin' using which you can login through ssh and then later 'su -' to become
root. For this make sure that the user 'admin' has ssh access.

Allow access to only the limited users via ssh
Use 'AllowUsers and DenyUsers options in sshd_config.
Eg:
AllowUsers admin admin2
DenyUsers user2

TCP wrappers
Use host.allow and/or hosts.deny wisely to allow and deny access to particular hosts/IPs.

Secure /tmp
/tmp is world writable (chmod 777) so anyone can get a file uploaded to it and execute it. To
prevent this, /tmp must be mounted with noexec; just make sure that fstab entries are as follows and
remount /tmp :
/dev/sdax
/tmp
ext3
noexec,nosuid,rw
0
0
However, this is possible only if /tmp is a separate partition. If not, one will have to go ahead and create
a new loopback device for /tmp; This is how it is:
dd if=/dev/zero of=/dev/TmpFS bs=1024 count=1000000
mkfs.ext3 /dev/TmpFS
cp -rp /tmp /tmp.bak
mount -o loop,noexec,nosuid,rw /dev/TmpFS /tmp
chmod 1777 /tmp
cp -rp /tmp.bak/* /tmp
cp -p /etc/fstab /etc/fstab.bak
; creates a loop device of 1Gb
; format it as ext3
; backup the present /tmp
; mount the new loop device to /tmp
; make its permissions correct
; restore the contents from backup
; backup fstab
add the necessary entry in fstab now, will look as follows:
/dev/TmpFS /tmp
ext3
loop,noexec,nosuid,rw
00

Configure firewall (iptables) rules
After adding all the required ACCEPT rules for required services/ports in the server, at the end
add a rule to DROP out all the the connections.
iptables -A INPUT -j DROP
; add this rule
This will DROP all those connections except for the ACCEPT rules. So recheck to make sure your
ACCEPT rules are correct before adding this, or else you may lock out yourselves.

Shutdown unwanted services
Any unwanted services that the server in not going to host can be stopped. This will save
resource usage as well.

| No comment yet

Important linux configuration files


Main Linux files:
/etc/aliases
This file describes user ID aliases used by sendmail. The file resides in /etc and is formatted as
a series of lines, each alias per line.
Eg:- support: postmaster
This line states that all mails addressed to user 'support' will be send to user 'postmaster' .
/etc/crontab
This file specifies the running times for system wide tasks, not that of individual users. A typical
entry in this file is as follows:
-----------
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
-----------
/etc/fstab
The file fstab contains descriptive information about the various file systems. It is the duty of
the system administrator to properly create and maintain this file. Each filesystem is described on a
separate line; fields on each line are separated by tabs or spaces.
This consists of the
• filesystem device, say a Hard disk partition
• its mount point, say /boot , /home etc.
• the filesystem type, say vfat, ext3, ntfs
• mount options for the fs, such as ro, rw, for a particular user or nouser
• the fsck options, of how often the checks must be done etc.
/etc/group
This holds all the usergroups and the respective group ID (GID).
group_name:x:GID:user1,user2
group_name: It is the name of group. If you run ls -l command, you will see this name printed in the
group field.
x is Password field: Generally password is not used, hence it is empty/blank. It can store encrypted
password. This is useful to implement privileged groups.
Group ID (GID): Each user must be assigned a group ID. You can see this number in your /etc/passwd
file.
Group List: It is a list of user names of users who are members of the group. The user names, must be
separated
/etc/hosts
This file is a simple text file that associates IP addresses with hostnames, one line per IP
address. For each host a single line should be present with the following information:
IP_address canonical_hostname [aliases...]
hosts_access files /etc/hosts.allow & /etc/hosts.deny
A level of fire-walling can be implemented using the two above files for specific services.
In each file, one can specify an IP or HOST that should be allowed access to or denied.
Eg:- Sshd : 192.168.1.56
The above line if added to allow file will grant access to that IP for service ssh, on the other
hand if added to hosts.deny, then that IP will be denied by ssh service.
/etc/inittab
This describes the default runlevel the system will be on at its boot.
The different runlevels are:
0 - halt (Do NOT set initdefault to this)
1 - Single user mode
2 - Multiuser, without NFS (The same as 3, if you do not have networking)
3 - Full multiuser mode
4 - unused
5 - X11
6 - reboot (Do NOT set initdefault to this)
Eg:- id:3:initdefault:
/etc/passwd
This file contains the account information, such as username, password, UID, GID, home
directory, and default shell. All normal users on the system can read this file, however, the passwords
are encrypted, so even though everyone can read the file, it doesn't automatically guarantee access to
the system.
Eg:-
admin:x:508:508::/home/admin:/bin/bash
/etc/shadow
This file stores actual password in encrypted format for user's account with additional properties
related to user password i.e. it stores secure user account information. All fields are separated by a
colon (:) symbol. It contains one entry per line for each user listed in /etc/passwd .
It contains:-
User name : login name
Password: In encrypted format.
Last password change (lastchanged): Days since Jan 1, 1970 that password was last changed
Minimum: The minimum number of days required between password changes
Maximum: The maximum number of days the password is valid
Warn : The number of days before password is to expire that user is warned that his/her password must
be changed
Inactive : The number of days after password expires that account is disabled
Expire : days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login
may no longer be used
/etc/rc.local
This script will be executed after all the other init scripts. You can put your own initialization
stuff in here.
/etc/redhat-release
This file will be there in redhat based systems, and will hold the exact OS that the system is
installed with. Just try 'cat /etc/redhat-release'.
/etc/resolv.conf
It contains information that determines the operational parameters of the DNS resolver. The
DNS resolver allows applications running in the operating system to translate human-friendly domain
names into the numeric IP addresses that are required for access to resources on the local area network
or the Internet. The process of determining IP addresses from domain names is called resolving. The
resolv.conf file typically contains directives with the IP addresses of nameservers available to a host.
Eg:-
nameserver 192.168.1.1
/etc/sysctl.conf
The sysctl command is used to modify kernel parameters at runtime. /etc/sysctl.conf is a text
file containing sysctl values to be read in and set by sysctl at boot time.
sysctl -a ; To view current values.
sysctl -p ; To reload any changes settings.
Eg:- To turn on IP forwarding, one would use the following line:
net.inet.ip.forwarding=1

| No comment yet

Commonly used linux commands/tools in this industry


ls
list contents in preset folder
ls -l : list with more details

cd
change to a different directory

cp <SOURCE> <DESTINTION>
copy contents to a different location
-r for directories

mv
rename a file/folder OR move it to a different location

rm
delete a file
-r in case of directories
NOTE : Always think twice before using 'rm' command

file <FILENAME>
displays the file type

grep <PATTERN> <FILENAME>
searches and prints if a particular PATTERN is present in a FILENAME

cat <FILENAME>
prints the textual contents in FILENAME
more & less
unlike cat for viewing contents in a FILENAME page-wise

tail
outputs last few lines of a FILENAME
-f used to get the real-time output of a file, say a logfile eg:- tail -f /var/log/httpd/error_log

head
opposite of tail , displays top few lines

ln
create links to an existing file
-s used to create symbolic links eg:- ln -s <TARGET> <LINKNAME>

touch <FILENAME>
change time-stamps of FILENAME to that of present, and if FILENAME doesn't exists an empty one
with present time-stamps is created.

mkdir
create a directory

df
outputs disk space usage
-f outputs a much friendly output

du
outputs the file/folder space usage
-sch results a grand total space usage in friendly format

chmod <BITS> <FILE/FOLDER>
alter permissions for a file/folder
BITS -> r w e r w e r w e
----- ----- -----
U G O
each 'r' 'w' and 'e' can be 1 to enable OR 0 to disable

chown USER:GROUP <FILE/FOLDER>
alter ownerships to that for a particular user and group.

find
search for a particular file/folder

which <COMMAND>
shows the exact location of a shell command

locate <FILE>
prints all files with name FILE in the system

updatedb
'locate' needs a database which stores all the files and its location, 'updatedb' updates this database, by
default it will be in system daily crons.

history
pulls out a list of all the commands that are entered in a shell

mount <DEVICE> <MOUNTPOINT>
mounts the DEVICE (a hard disk partition, CD-ROM, external drive etc.) to a folder MOUNTPOINT.

umount <DEVICE or MOUNTPOINT>

su <USER>
switches to a different USER

sudo
executes a command with another user privileges.
However a user needs permission to execute sudo which can be given by editing the /etc/sudoers file
or executing 'visudo' in redhat based systems.

ifconfig
configure a network interface, without any arguments it lists all the network interfaces in the system
eg:- ifconfig eth0 192.168.1.50
- this assigns the IP 192.168.1.50 to the primary network card

ssh HOST -l USER -p PORT
a remote login program to a SSH server

scp
remote copying of files through ssh protocol
eg:- scp user1@server1.com:/file1 filecopy
use -r for directories

rsync
a remote/local file copying and syncing program
eg:- rsync -avz user1@server1.com: /home/user1
rsync -avz /home/user1/ /backup/user1/

vim
a text based file editor in linux/unix
for vim insert mode : INSERT or I key
for vim escape mode : ESC key
:w - save changes
:q - quit
:wq - save & quit
:q! - quit without saving

ftp
a text based FTP client from shell

mail
mail program from shell
mail -s SUB TO_ADDRESS -b BCC -c CC -a ATTACHMENT < FILENAME (with contents)

telnet
connect to remote hosts using telnet protocol
telnet <HOST> <PORT>

tar -c <ARCHIVE.tar> <FILE1 FILE2>
archive files together to a single file
-c : create archive
tar -x <ARCHIVE.tar>
-x : extract archive

screen
opens a shell session and any program started here will run as long as the screen in running, even if the
user logs out.

w
gives a list of who all are logged in and what each of them are doing
uptime
shows how long the system has been up and running

ps
list current running processes

ps aux : lists all running processes in the system.

free
shows memory usage in the system

free -m : mostly used

uname
prints system information such as OS, architecture, kernel version, hostname etc.
eg:- uname -a

kill
terminates a particular PID in the system
eg:- kill -9 <PID>
killall -9 httpd

ping <HOST or IP>
send ICMP requests to HOSTS

dig <HOSTNAME>
DNS lookup utility
dig <HOSTNAME> <NAMESERVER> : DNS lookup for HOSTNAME in NAMESERVER

host <HOST or IP>
DNS lookup utility to convert hosts to IP and vice-versa
eg:- host google.com

top
displays running processes in real-time.
top -c : mostly used, shows top results with full commands

nice
assign priorities for a program, default value is 10.
with option -n the range can be from -20 (most favorable scheduling) to 19 (least favorable).
eg:- nice top -c

whois <DOMAIN or IP>
provides domain name registration details OR IP ownership details.
eg:- whois google.com

netstat
outputs all present network connections to server.
netstat -plan : commonly used.

iptables
default firewall in linux
-A
Append rule to end of a chain
-I
Insert rule
-F
Flush. Deletes all the rules in the selected table
-p <protocol-type> protocol types include icmp, tcp, udp, and all
--sport <port> source port
--dport <port> destination port
-s <ip-address>
source IP address
-d <ip-address>
destination IP address
-j <target>
target can be ACCEPT, REJECT or DROP. select depending on what you wish to
do with the new rule.
-nL Lists the present rules