Blue Flower

Bill

Bill

Monday, 06 July 2015 01:07

How to backup vpopmail with rsync/ssh

Setting up vpopmail with rsync and ssh

The reason why vpopmail just doesn't work with rsync is mainly because vpopmail has a chmod of 700 which means only the owner, vpopmail, can read it or root if you're logged in that way. Here is how to rsync vpopmail over a ssh connection.

Part of this documentation was taken from http://jms1.net/ssh.shtml and also http://www.qmailinfo.org/index.php/ExampleRsyncScripts and https://wiki.archlinux.org/index.php/SSH_keys#Ed25519

Setting up the ssh keys

On the server, Open up /etc/ssh/sshd_config and add PermitRootLogin without-password. Then you will want to restart sshd by running killall -HUP sshd. You will get kicked out of your terminal. If all is well, You should be able to login again. Root is now allowed to login only via a ssh key. Don't worry, there is some added security in the document as well.

The first thing we will want to do is generate the key on the client, or backup, machine. Run the following command:

ssh-keygen -t ed25519 -b 1024 -f id_dsa_vpopmail -C 'Some comment'

When you see 'Enter passphrase (empty for no passphrase):' Just hit enter and then when you see the confirmation that says 'Enter same passphrase again:' just hit enter again. After a few seconds, it should give you an output similar to the following:

Your identification has been saved in id_dsa_vpopmail.
Your public key has been saved in id_dsa_vpopmail.pub.
The key fingerprint is:
88:da:e8:4c:50:5d:c5:95:b9:1e:6e:8f:96:82:c0:19 Some Comment

Now, the id_dsa_vpopmail.pub is the file we need to ssh over to your server. Here are the steps you will want to follow to get this over to your server to allow root logins via ssh:

# scp id_dsa_vpopmail.pub user@server:
user@server's password:
id_dsa_vpopmail.pub |********************| 0 0:00
# ssh user@server:
user@server's password:
(You have to be root at this point to do the following steps)
# cat id_dsa_vpopmail.pub >> ~root/.ssh/authorized_keys2
# chmod 600 ~root/.ssh/authorized_keys2
# exit

This is not the only way to get the public key file into place. You could also copy it on a floppy disk, or email it to the system administrator and have them install it for you (remember this is the PUBLIC key, there is no security risk in sending it via normal email.)

Note that you can have multiple keys listed in the .ssh/authorized_hosts2 file- this is why the public keys have comments at the end, so you can easily tell which line in the file corresponds to which key.

To add a bit more security on the server, will will want to change the /root/.ssh/authorized_keys2 to look like so:

command="/root/.ssh/rsync-key" ssh-dss AAAAB3NzaAAA4fobEeQMoC6vRInbeNy5PukQ5fAkCc+Vr...

Then this is what is in the /root/.ssh/rsync-key file:

#!/bin/sh
logger -t ssh-command "$SSH_ORIGINAL_COMMAND"
echo $SSH_ORIGINAL_COMMAND > /tmp/work.$$
if ! grep -q '^rsync --server ' /tmp/work.$$
then
logger -t rsync-key INVALID COMMAND ""$SSH_ORIGINAL_COMMAND""
exit 1
fi
rm /tmp/work.$$
exec $SSH_ORIGINAL_COMMAND

Then we need to chmod it properly so it runs:

chmod 755 /root/.ssh/rsync-key

Backing up vpopmail

Now that we have the keys in place for rsync via ssh to work, we can now setup a script to automate this for us and then we can put it into cron. Here is my script for this. I will call this vpopmail-backup.sh:

#!/bin/sh
echo `date` starting >> /var/log/vpopmail-backup.log
rsync -aS --delete -e /path/to/backup-vpopmail-ssh root@HOST:/usr/home/vpopmail/ /backup/vpopmail/location
echo `date` done >> /var/log/vpopmail-backup.log

The '/path/to/backup-vpopmail-ssh' in the above file has this in it:

#!/bin/sh
unset SSH_AUTH_SOCK
exec /usr/bin/ssh -i /path/to/id_dsa_vpopmail $*

Now we want to setup the chmod for the following files:

chmod 755 vpopmail.sh
chmod 755 backup-vpopmail-ssh

Now lets give it a test!

./vpopmail.sh

If all goes well, It should pause there for a minute and then it will come back to the prompt. Check your vpopmail log in /var/log/vpopmail-backup.log and see if it started and stopped correctly.

If you have installed courier, stop all the courier services and remove startup scripts, config files, etc. Then, Follow the Slackware method for installing imap and it should work fine. You will need to add --with-mysql to the configure command in order for mysql authentication to work.

Monday, 06 July 2015 01:05

Setting up SMTP with TLS

Copied with permission from:
John Simpson
http://qmail.jms1.net/smtp-service.shtml
http://qmail.jms1.net/tls-auth.shtml

This service only accepts mail from authorized clients- it requires the AUTH command before accepting any messages. It also requires STARTTLS before the AUTH command may be entered. This makes an ideal "SMTP relay service" for your authorized users.

You must be using John Simpsons qmail patch in order for this to function properly.

# cd /var/qmail/supervise
# mkdir -m 1755 qmail-smtpd-tls
# cd qmail-smtpd-tls
# fetch http://goodcleanemail.com/files/run.smtp.sslserver
# mv run.smtp.sslserver run
# vi run

This will start up a text editor on the script. I prefer nano, but you are free to use pico, vi, emacs, or any other text editor you like. Set the options as needed for your service. The file itself contains documentation on the options you can set.

You should set the following values:

IP=1.2.3.4  Substitute your own IP address. Do not leave this set to 0 without a good reason.
PORT=587  Set the port number we will be listening on.
SSL=0  Do not run an SSL-only service.
FORCE_TLS=1  Refuse to accept mail from clients who have not done STARTTLS.
DENY_TLS=0  Do not refuse to process the STARTTLS command.
AUTH=1  Allow the AUTH command after STARTTLS has been completed.
REQUIRE_AUTH=1  Refuse to accept mail from clients who have not done AUTH.

Once you are finished editing and have saved the file...

# chmod 700 run
# mkdir -m 755 log
# cd log
# fetch http://goodcleanemail.com/files/service-any-log-run
# mv run.log run
# chmod 700 run

Setting up the tcpserver access files

If you have already setup the Makefile, skip down to Creating the smtp file

I do something a little different with my tcpserver access control files than most other people. Instead of calling the files /etc/tcp.smtp and /etc/tcp.smtp.cdb (the files are in /etc/ and have names which start with "tcp.") I call them /etc/tcp/smtp and /etc/tcp/smtp.cdb. The idea is that /etc/tcp is a directory containing all of my tcpserver access control files, along with a Makefile which rebuilds any out-of-date cdb files whenever their source text files have been updated.

If you use the "run" scripts from my web site, you will find them written this way. Of course you can edit the scripts and change the filenames if you like, but I have found this to be a much easier way to administer the control files (I use tcpserver for a lot more than just qmail.)

To set this up on your own server...

These commands should be run as root.
# mkdir -m 755 /etc/tcp
# cd /etc/tcp
# fetch http://goodcleanemail.com/files/etc-tcp-makefile
# mv etc-tcp-makefile Makefile

Creating the smtp file

At this point it should be ready to go- all you need to do is create the "smtp" file, containing the normal access control list. It may look something like this:

127.:allow,RELAYCLIENT=""
:allow

run

# gmake

or if you're not using FreeBSD

# make

Finally:

# cd ~vpopmail/bin
# chown vpopmail:vchkpw vchkpw
# chmod 6711 vchkpw

The final step is to start the service running.

# ln -s /var/qmail/supervise/qmail-smtpd-tls /service/

Wait about ten seconds, and then make sure the service is running correctly.

# svstat /service/qmail-smtpd-tls
/service/qmail-smtpd-tls: up (pid 25183) 6 seconds

The number of seconds should be two or greater, and if you re-run the same command again, you should see the count going up rather than cycling back to zero. If the count never passes three, or if the service is not listed as "up" to start with, check the logs to see what's going on.

# tail log/main/current

Monday, 06 July 2015 01:03

SMTP to Smarthosts

John Simpsons patch already includes the patch for using smtproutes. Just skip the next section and continue with editing the smtproutes file:

# cd /usr/src/qmail/qmail-1.03
# wget http://www.goodcleanemail.com/files/makefile-remote-auth-qmr.patch
# wget http://www.goodcleanemail.com/files/qmail-remote-auth-qmr.patch
# patch -p0 < qmail-remote-auth-qmr.patch
# patch -p0 < makefile-remote-auth-qmr.patch
# make

If it makes cleanly, run

# make setup check

to install the binaries.

Then, in the smtproutes file you can add a couple of extra arguments
of a username and password after the host, .e.g.

:my.default.smarthost.com user password

then it will authenticate with the remote host to send email. Btw,
with just the colon ':' at the start of the line that's the 'default'
route for mail. If it was just for a particular domain, then put that
in front of the colon.

Special thanks to Derek for documenting this.

There are a few reasons why you would want to upgrade your qmailrocks package.  The first is security by being able to use TLS or SSL encryption. The default qmailrocks setup does authentication via plain text passwords.  It's not that difficult for a malicious user to sniff out that text as it passes between your clients and the server.  

Before you continue on, There are 2 things you need to know about. First I think you should check out Johns "Upgrading from qmailrocks" page which is at

http://qmail.jms1.net/upgrade-qmr.shtml

Second, When you do the upgrade from qmailrocks to Johns combined patch, smtp-auth will not work with plain text passwords. It will only work with TLS as setup in this doc or via SSL which is explained at the bottom of this page. I would suggest having a migration plan setup ahead of time so this doesn't upset anyone. If you really need a smtp server setup with plain-text passwords, setup another box to do smtp only.

Probably the biggest reason to switch is John Simpson's validrcptto patch. It drops invalid recipients at the smtp level by checking a database of valid email addresses on your server before the email is processed.  This causes less load on the server and can make a tremendous difference if your domain is getting bombarded with spam to invalid users.

Lastly, When I setup new clients for qmail, I do it via the freebsdrocks.net site. Basically what you do when you set it up is it only allows incoming connections on port 25 and outbound connections on ssl (port 465) or TLS (Port 585). This will allow you to enable things on port 25 like jgreylist, rbls and validrcptto. What most people don't understand is you cannot enable clients to send mail on the same port you're accepting mail. Incoming mail won't have a problem but when clients try to send mail on port 25, they're going to timeout because of all the checks (like jgreylisting, rbls, validrcptto)

Plus there are many other patches included in Johns Combined patch that will enhance your server's performance and security.  For details go to:

http://qmail.jms1.net/patches/combined-details.shtml

The first step is to download the new qmail-smtpd/run file so we can get it prepped and ready to go. PLEASE NOTE: We are *NOT* going to replace the run file until AFTER we patch the system first. Changing the file won't take very long after we run the patch.

# cd /service/qmail-smtpd/
# cp run bak_run
# fetch http://qmail.jms1.net/scripts/service-qmail-smtpd-run
# vi service-qmail-smtpd-run

Now the list of available options for service-qmail-smtpd-run is listed below

http://qmail.jms1.net/tls-auth.shtml

If you ONLY want to accept messages on port 25 and not enable smtp-auth, use the following commands. Once this is done, you can turn on jgreylisting, rbls and validrcptto.

IP=1.2.3.4 Substitute your own IP address. Do not leave this set to 0 without a good reason.
PORT=25 Set the port number we will be listening on.
SSL=0 Do not run an SSL-only service.
FORCE_TLS=0 Refuse to accept mail from clients who have not done STARTTLS.
DENY_TLS=0 Refuse to process the STARTTLS command.
AUTH=0 DENY the AUTH command after STARTTLS has been completed.
REQUIRE_AUTH=0 Refuse to accept mail from clients who have not done AUTH.

However, if you HAVE to have auth on port 25, use the following commands to enable TLS. I would NOT RECOMMEND turning on jgreylisting, rbls or validrcptto.

IP=1.2.3.4 Substitute your own IP address. Do not leave this set to 0 without a good reason.
PORT=25 Set the port number we will be listening on.
SSL=0 Do not run an SSL-only service.
FORCE_TLS=0 Refuse to accept mail from clients who have not done STARTTLS.
DENY_TLS=0 Do not refuse to process the STARTTLS command.
AUTH=1 Allow the AUTH command after STARTTLS has been completed.
REQUIRE_AUTH=0 Refuse to accept mail from clients who have not done AUTH.

I would suggest setting the IP to your private, internal IP (If applicable). Then tell your firewall/router to pass port 25(smtp) and 110(pop) to your FreeBSD box internally.

Now that we have the qmail-smtp/run file prepared, we need to download the current version of the patch from:

http://qmail.jms1.net/patches/combined-details.shtml

For example:

# cd ~root
# fetch http://qmail.jms1.net/patches/qmail-1.03-jms1.7.06.patch

Now lets download the qmail source and extract it

# cd ~root
# wget http://cr.yp.to/software/qmail-1.03.tar.gz
# tar xvzf qmail-1.03.tar.gz
# cd qmail-1.03
# patch < ../qmail-1.03-jms1.7.06.patch

Don't worry about all the code flying by as long as it says 'Done' at the end.  Once the patch is complete we are ready to compile qmail with all the new enhancements. Please make sure there are no messages in the qmail queue when you stop qmail below. The output of qmailctl stat will tell you if there are any local/remote messages in the queue. THIS IS VERY IMPORTANT!! When you stop qmail and run make or make setup check, it may tell you if something is running. If it is, it is safe to kill it.

# make
# qmailctl stop
# ps ax | grep qmail-send
(if it's still running, wait a few seconds and try it again)
# ps ax | egrep qmail-send
...
# make setup check

Before we start qmail again, We now need to copy over the new qmail-smtpd/run file:

# cd /service/qmail-smtpd/
# cp service-qmail-smtpd-run run
# chmod 755 run

We will also replace the qmail-smtpd/log/run file with a new one as well

# cd /service/qmail-smtpd/log
# fetch http://qmail.jms1.net/scripts/service-any-log-run
# cp run bak_run
# cp service-any-log-run run
# chmod 0600 bak_run
# chmod 755 run

Before we start qmail, we need to make sure TLS works with vpopmail so lets run the following:

# chmod 6711 ~vpopmail/bin/vchkpw

Setting up the tcpserver access files

If you have already setup the Makefile, skip down to Creating the smtp file

I do something a little different with my tcpserver access control files than most other people. Instead of calling the files /etc/tcp.smtp and /etc/tcp.smtp.cdb (the files are in /etc/ and have names which start with "tcp.") I call them /etc/tcp/smtp and /etc/tcp/smtp.cdb. The idea is that /etc/tcp is a directory containing all of my tcpserver access control files, along with a Makefile which rebuilds any out-of-date cdb files whenever their source text files have been updated.

If you use the "run" scripts from my web site, you will find them written this way. Of course you can edit the scripts and change the filenames if you like, but I have found this to be a much easier way to administer the control files (I use tcpserver for a lot more than just qmail.)

To set this up on your own server...

These commands should be run as root.
# mkdir -m 755 /etc/tcp
# cd /etc/tcp
# fetch http://goodcleanemail.com/files/etc-tcp-makefile
# mv etc-tcp-makefile Makefile

Creating the smtp file

At this point it should be ready to go- all you need to do is create the "smtp" file, containing the normal access control list. It may look something like this:

192.168.0.:allow,RELAYCLIENT=""
:allow

run

# gmake

or if you're not using FreeBSD

# make

Ok, Everything is set to start so lets start qmail.

# qmailctl start
# svc -u /service/*
# qmailctl stat

Just make sure here that the /service/qmail-smtpd and /service/qmail-smtpd/log ones are up for more than one second. If not, take a look at the log file

# tai64nlocal < /service/qmail-smtpd/log/main/current

Other than that, you are good to go! You now need to tell your users to send smtp via TLS. If you want to setup smtp via SSL, You need to setup a separate service. Take a look at

http://freebsdrocks.net/index.php/11-installing-qmail/28-setting-up-smtp-with-ssl

or setup SMTP with TLS on a seperate service:

http://freebsdrocks.net/index.php/13-useful-qmail-utilities/85-setting-up-smtp-with-tls

If you want to enable validrcptto, take a look at the following URL. FYI this is part of my qmail guide.

http://freebsdrocks.net/index.php/11-installing-qmail/34-installing-vpopmail-with-onchange

If you would like to enable jgreylist, follow this website:

http://qmail.jms1.net/scripts/jgreylist.shtml

Have you ever looked at the output of ps axwww (on Linux for example) and
wondered what the readproctitle sericve errors were for?

Did you ever wonder if those messages were current, or 100 days old?

Well, normally there is no way to tell since that is just FIFO a buffer that
does not change unless a new error message gets written to it.

Wouldn't it be nice to either rotate errors out of that buffer, or to timestamp
it so that you can be sure any errors are current?

Here we have a sample listing of a readproctitle service errors buffer:

--[snip]--
readproctitle service errors: ...s not exist?softlimit: fatal: unable to run  :
file does not exist?softlimit: fatal: unable to run  : file does not
exist?softlimit: fatal: unable to run  : file does not exist?softlimit: fatal:
unable to run  : file does not exist?softlimit: fatal: unable to run  : file
does not exist?softlimit: fatal: unable to run  : file does not exist?softlimit:
fatal: unable to run  : file does not exist?
--[snip]

We know that the file softlimit exists, and we know that it is being properly
called by full path in our script. So, are these old messages or are they new?

Do we really still have a problem with softlimit, or can we ignore this error
and look elsewhere for the cause of our problem?

- Start off by creating a new supervise directory.
eg: /var/qmail/supervise/readproctitle

- Create an executable script called "run" in that directory. "run" should
look something like this:

#!/bin/sh
echo -n  ..`date "+%D-%R"`..  >&2

- Touch the file /var/qmail/supervise/readproctitle/down so that daemontools
does  not try to automatically start this process.

- Create the symlink in the service directory: 
ln -s /var/qmail/supervise/service

- Run the script and check the output:
svc -o /service/readproctitle
ps axwww | grep readproctitle

You should now see that a current timestamp has been pushed into the end of the
FIFO buffer. Now we are ready to automate this process.

- Now create a cronjob to run this script at any interval you desire. Keep in
mind though, we are going to be putting a timestamp into this FIFO buffer so
the shorter the duration of your cronjob, the sooner actual error messages
will be pushed out of the buffer by timestamps. Here is our sample cronjob
entry:

0 * 0 0 0 root /usr/local/bin/svc -o /service/readproctitle

This will put a timestamp into the readproctitle service errors buffer once
every hour, on the hour.

**note**  You may not want ugly timestamps cluttering up yout readproctitle
service errors buffer. You may have a script that parses that buffer expecting
to only see periods and will notify you if there is anything else in that
buffer. In either of these cases, you may simply replace the string in the
readproctitle/run file with one single period.

Special thanks to
William Arlofski
mtnbktATrevpolDOTcom

 How can I disable qmail from conducting reverse DNS lookups on SMTP connections?

 This can be done by adding a "-H" flag to the tcpserver call within the qmail-smtpd supervise script. That file is located at /var/qmail/supervise/qmail-smtpd/run and you would do the following:

Find the line that starts with the tcpserver call:

 /usr/local/bin/tcpserver -v -R -l "$LOCAL" -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \

 and add the "-H" flag:

 /usr/local/bin/tcpserver -v -R -H -l "$LOCAL" -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \

use dyndns for resolution but alas, that doesn't help with dynamic address range BLs.

I set this up as a temporary fix while I wait for my static address block to
be implemented. For anyone else out there who would like to do this for
whatever reason, whether it's like me or you just need to relay all your mail
to an ISP's smtp server which only supports SSL connections, here's how you
do it.

This assumes you have followed the freebsdrocks qmail install or another guide
but installed either jms' combined patches or at least qmail-remote and auth.

Step 1 - Install stunnel
----------------------------------------
# cd /usr/ports/security/stunnel
# make install clean

Accept the default threading mech.

Step 2 - Configure stunnel
----------------------------------------
# cd /usr/local/etc/stunnel
# vi stunnel.conf

There is a sample included in this directory with the options and you can
check the man page for additional info. For the purposes of this, we'll just
stick with creating a blank conf and adding only the lines we need.

Add the following lines:

client=yes
[smtps]
accept = 2525
connect = my.smarthost.name.xxx:465

Obviously, change the name and any port numbers. The accept port is where you
want to listen on your box for connections to tunnel to the host in the
connect value. Most SSL smtp servers should use the default of 465 as the
port number. If they use something else, change it.

Step 3 - rc.conf & starting stunnel
----------------------------------------
# echo 'stunnel_enable="YES"' >> /etc/rc.conf
# /usr/local/etc/rc.d/stunnel.sh start

To verify that it's working, you should be able to see your accept port in a
netstat.

# netstat -anf inet

You should now be able to telnet to your accept port and use their SSL enabled
sever as if you were connecting without SSL.

# telnet localhost 2525

This should return a standard telnet session to smtp.

Step 4 - Configuring qmail
----------------------------------------
# echo ':localhost:2525 -user pass' > /var/qmail/control/smtproutes

The above line tells qmail to send all outbound mail to localhost:2525 and use
user and pass to auth. The - before the user is required. By default,
qmail-remote performs a security check on the connection to ensure that it is
TLS secured prior to sending any auth. We're using SSL here. The - instructs
qmail-remote to bypass this check and send the auth anyway.

You can also do selective smtp relaying by prepending a domain name before the
first colon. Like:

# echo 'domain.com:localhost:2525 -user pass' > /var/qmail/control/smtproutes

This will tell qmail to only route email to domain.com to the tunnel.

Now restart qmail

# qmailctl stop
# qmailctl start

Step 5 - Verifying that it works
----------------------------------------
Using your smtp server, send an email to a remote address. To send a quick
email from the local system, use the mail command.

# echo "Testing 1 2 3" | mail -s "Testing stunnel" This email address is being protected from spambots. You need JavaScript enabled to view it.

Now tail the log to see what the remote server said.

# tail /var/log/qmail/qmail-send/current

If it worked, you should see some messages like

new msg 24007
info msg 24007: bytes 2341 from < This email address is being protected from spambots. You need JavaScript enabled to view it. > qp 66734 uid 1003
starting delivery 37: msg 24007 to remote This email address is being protected from spambots. You need JavaScript enabled to view it.
status: local 0/10 remote 1/120
delivery 37: success:
127.0.0.1_accepted_message./Remote_host_said:_250_ok_1172761202_qp_2377/
status: local 0/10 remote 0/120
end msg 24007

Special thanks to Nick Holder

The original author only included the line to change.

# cd /usr/ports/sysutils/ucspi-tcp/work/ucspi-tcp-0.88
# vi rblsmtpd.c

Find the line (should be line 128):

if (text.len > 200) text.len = 200;

and change it to:

if (text.len > 500) text.len = 500;

Special thanks to Nick Holder and Michael Bowe at

http://www.bowe.id.au/michael/isp/webmail-server.htm

:allow
This will allow the specified IP range to make a connection

:deny
This will deny the specified IP range to make a connection

RELAYCLIENT=""
When set with :allow, this will accept relay messages from the specified IP
range

RBLSMTPD=""
When set with :allow or :deny, this will instruct smtpd to skip RBL checks for
the specified IP range

RBLSMTPD="my temp error message"
When set, this will skip RBL lookups but return "my temp error messsage" as a
4xx temp error for the specified IP range

RBLSMTPD="-my perm error message"
When set, this will skip RBL lookups and return "my perm error message" as a
5xx perm error for the specified IP range
 
Special thanks to Nick Holder
Page 8 of 14