VideoRanchCloudServerConfiguration
Videoranch Cloud Server Configuration.
The purpose of this document is provide information on how gihon.orgs cloud server is currently configured and basic guidelines for maintaining it.
| # Date | # Author | # Comments | |
|---|---|---|---|
| 28MAY16 | Donald Delmar Davis | don@suspectdevices.com | Initial document |
| ### Background | |||
| We were asked to convert a 15 year old internet server running freebsd to the cloud. We started by setting up a staging server running Ubuntu 14.04 and migrating the users data and log files from the old server. This provided a backup of the original data and a place where we could work without having to pay for disk or bandwidth before deploying the final product. After a long process of porting all of the users and web sites that the server had served over the decades we began identifying which services, users, and domains were needed on the server. Given a much smaller set of users and web sites that were actually needed, we deployed an AWS image based on the AMI provided by the commercial entity which maintains Ubuntu. The active users users and web content have been installed on this server and the remainder has been archived to an external disk. |
The Base Image
We chose to deploy an image provided by Canonical specifically for AWS "ubuntu-trusty-14.04-amd64-server-20150325 (ami-5189a661)" http://cloud-images.ubuntu.com/releases/trusty/release-20150325/
Adjustments to the image
The ubuntu user which provides a back door through which AWS allows users that it has authenticated to have root access to the instance. Unfortunately the ubuntu UID(1000) was already taken (jess) so it was moved to 999 and files owned by it were migrated as well.
chown --from=1000:1000 999:999 /. -Rv
Also the mail spool was somewhere new (/var/spool/mail) so I linked the new location back to /var/mail
Additions to the image
a lamp stack was added to the image using the "tasksel" package which bundles most services into supported configurations and deploys them along with all of their dependencies. (Note that the Ubuntu Cloud Image was already installed)
# tasksel
Package configuration
┌─────────────┤ Software selection ├─────────── ───────┐
│ You can choose to install one or more of the following predefined collections of software. │
│ │
│ Choose software to install:
│
│ [*] Basic Ubuntu server
│ [*] OpenSSH server
│ [ ] DNS server
│ [*] LAMP server
│ [*] Mail server
│ [*] PostgreSQL database
│ [ ] Print server
│ [ ] Samba file server
│ [ ] Tomcat Java server
│ [*] Ubuntu Cloud Image (instance)
│ [ ] Virtual Machine host
...
│ <Ok> │
│ │
└─────────────────────────────────────┘
users and superusers
The following users were added to the system.
jess:x:1000:1000:Jessica Kent:/home/jess:/bin/csh
gepr:x:1053:1053:Glen E Ropella:/home/gepr:/bin/bash
don:x:1054:1054:Donald Delmar Davis:/home/don:/bin/bash
vic:x:1002:1002:Victoria Kennedy:/home/vic:/bin/bash
nez:x:1003:1003:Michael Nesmith:/home/nez:/bin/bash
vranch:x:1004:1004:Videoranch User:/home/vranch:/bin/bash
foreman:x:1005:1005:Videoranch Foreman:/home/foreman:/bin/tcsh
navajoslim:x:1007:1007:Navajo Slim:/home/navajoslim:/bin/bash
gihon:x:1017:1017:Gihon Foundation:/home/gihon:/bin/bash
vk:x:1021:1021:Victoria Kennedy:/home/vk:/bin/bash
vrresume:x:1024:1024:videoranch resume:/home/vrresume:/bin/bash
vak:x:1027:1027:victoria kennedy:/home/vak:/bin/tcsh
nezrays:x:1031:1031:nezrays:/usr/home/vranch/nezrays/www:/bin/sh
vr3d:x:1035:1035:VR3D:/home/vr3d:/bin/sh
staging:x:1041:1041:staging:/home/staging:/bin/bash
nesmith:x:1042:1042:nesmith:/home/nesmith:/bin/bash
director:x:1045:1045:Jessica Kent:/home/director:/bin/bash
petetest:x:1048:1048:petetest:/home/petetest:/bin/bash
mn:x:1022:1022:Michael Nesmith:/home/mn:/bin/bash
This had to be done manually as some of the origional passwords were so old that their encryption methods were no longer supported. In cases where the users were less than a few years old the users passwords transferred to the new system seamlessly. In other cases the passwords will have to be reset by someone with root access.
ubuntu@cloud # passwd vranch
Their mail spools (/var/mail/
sudo privileges were enabled for members of the sudo group.
ubuntu@cloud # vigr
...
sudo:x:27:ubuntu,jess,foreman,don,gepr
...
Apache Configuration
In addition to the home directories of the remaining users the /home/vranch directory tree and /home/gihon were copied to the new server. The server configurations were ported to be as close to the originals as possible. (exceptions noted below)
The default server is set to www.gihon.com and is configured based on the original virtual-host. The php information and much about the apache server can be queried directly at http://videoranch.com/test.php
#ServerName www.gihon.com
<VirtualHost *:80>
ServerName www.gihon.com
ServerAlias gihon.com www.gihon.org gihon.org cloud.gihon.com
ServerAdmin info@digitaloffspring.com
DocumentRoot /home/gihon/www
<Directory '/home/gihon'>
AllowOverride All
</Directory>
ScriptAlias /cgi-bin/ /home/gihon/cgi-bin
CustomLog /home/gihon/logs/gihon-access_log common
ErrorLog /home/gihon/logs/gihon-error_log
</VirtualHost>
- Note that the log files are left in user space (off of /home) this allows clients to pull and view the log files in the same way that they update the content of their web site (ftp etc)
- Some configuration directives are no longer supported and are commented out.
- Extremely dangerous statements such as AllowOverides for the root directory were modified.
All other servers are named virtualhosts. The first of which is www.videoranch.com defined in /etc/apache2/sites-enabled/www.videoranch.com.conf
<VirtualHost *:80>
ServerName www.videoranch.com
ServerAlias videoranch.com www.videoranch.com
# Header append p3p 'CP=\"OTI DSP COR CUR UNI\" polyref=\"/w3c/p3policy.xml\"'
ServerAdmin info@digitaloffspring.com
DocumentRoot /home/vranch/videoranch/www
ScriptAlias /cgi-bin/ /home/vranch/videoranch3d/cgi-bin/
ErrorLog /home/vranch/logs/www.videoranch.com-error_log
CustomLog /home/vranch/logs/www.videoranch.com-access_log common
<Directory /home/vranch/videoranch/www>
Options Indexes FollowSymLinks
AllowOverride All
</Directory>
</VirtualHost>
Pro-ftpd Configuration
We configured proftpd (which we vetted as a viable and secure ftp daemon) as closely as possible to the original configuration on the old server. Because AWS instances are in their own private network and access has to be explicitly allowed you must specify the PASV ports in /etc/proftpd/proftpd.conf. These ports must be opened up in the "Security Group" configuration as well.
# In some cases you have to specify passive ports range to by-pass
# firewall limitations. Ephemeral ports can be used for that, but
# feel free to use a more narrow range.
PassivePorts 49152 49153
Ftp in its native form is insecure and so we would prefer to have configured an SSL certificate and require TLS for all ftp requests. We were able to verify that SFTP (ftp provided by ssh).
Network and "Security Group" configuration
The AWS instance is placed in a private network. This network provides the instance a private ip through dhcp. For this reason the main interface is configured as follows in /etc/networks/interfaces.d/eth0
# The primary network interface
auto eth0
iface eth0 inet dhcp
This address is attached to the outside world via an "Elastic" ip (52.34.143.142). To connect the external traffic to the private address you have to create a "Security group" and define the rules which allow traffic in and out of the private network.
- INBOUND RULES * || |# protocol|# family|# port|# allow from| |----------|--------|------|------------| | HTTP | TCP | 80 | 0.0.0.0/0 | | SSH | TCP | 22 | 0.0.0.0/0 | | SMTP | TCP | 25 | 0.0.0.0/0 | | Custom TCP Rule | TCP | 20 - 21 | 0.0.0.0/0 | | IMAP | TCP | 143 | 0.0.0.0/0 | | Custom TCP Rule | TCP | 49152 - 49153 | 0.0.0.0/0 | | HTTPS | TCP | 443 | 0.0.0.0/0 |
Outbound rules allow all outgoing traffic.
Unused Capabilities
MySQL and PostgresSQL
While the M in LAMP is MySQL, Many developers prefer Postgres which is much more standards oriented and robust. Both databases are available and PHP is configured for them. At one point mysql was on the old server however neither gihon nor the model files served by videoranch.com seemed to use it. _ Note that if either database is used a mechanism to back up the data must also be implimented_
Postfix and Dovecot
The standard SMTP (email) server for most current operating systems is Postfix. The Mail server task also includes Dovecot which provides both POP and IMAP servers for clients to download any mail still on the server. To use the pop server will require the addition of the ports for pop (110) to be added to the security group configuration. _These servers are not currently configured. _
Log Rotation Configuration
On the previous server most log files were larger than the content being provided. Ubuntu provides a log rotation utility designed to compress and delete logs in a reasonable manner preventing them from consuming system resources over time. Since the apache logs on this system are in "user space" and not under /var/log/apache2 their location needed to be configured.
Here is the section added to /etc/logrotate.d/apache2 for the gihon.com
/home/gihon/logs/*_log {
weekly
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if /etc/init.d/apache2 status > /dev/null ; then \
/etc/init.d/apache2 reload > /dev/null; \
fi;
endscript
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
}
unattended upgrades (security only)
The system is configured to automatically install security upgrades as released by the operating system. In the event that an error occurs mail is sent to the foreman account.
Operations Guide
Given the state of the previous system the soundest approach is to automate as much of the systems upkeep as possible. Log rotation and unattended system upgrades along with other minor adjustments (turning on apt's auto-remove for instance) should enable us to think of the box more as an appliance.
Backing up Server work with Live Snapshots
AWS allows a server to be backed up while running. These snapshots can be run up as separate servers (for development or to do a major release upgrade) Or they can be reattached to an existing instance (in the case of disaster or compromise). Please make a snapshot of the server whenever significant work has been done to it.
Backing up your data
Since the servers web content is in the user space. Log files, websites and other data served should be copied to a local server preferably one behind a firewall. In particular Gihon should take care to keep updated copies of /home/gihon and /home/vranch
Accessing the server
Privileged access can be granted through AWS to the Ubuntu user. For instructions on how to do this see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html. The server has been configured to allow ssh access directly.
$ ssh www.videoranch.com
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-85-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Mon Apr 11 18:12:10 UTC 2016
System load: 0.0 Processes: 139
Usage of /: 69.8% of 29.39GB Users logged in: 1
Memory usage: 28% IP address for eth0: 172.31.16.108
Swap usage: 0%
Graph this data and manage this system at:
https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
You have new mail.
Last login: Mon Apr 11 16:11:57 2016 from 71-34-91-188.ptld.qwest.net
don@cloud:~$
References
- why ubuntu? https://insights.ubuntu.com/2014/04/15/ubuntu-14-04-lts-the-cloud-platform-of-choice/
- https://www.digitalocean.com/community/tutorials/how-to-configure-logging-and-log-rotation-in-apache-on-an-ubuntu-vps
- https://help.ubuntu.com/lts/serverguide/automatic-updates.html
- https://anturis.com/linux-server-maintenance-checklist/