Archiving Stopped
In most cases, when archiving stops, there is a misconfiguration of either MailArchiva or the mail server; alternatively, something has changed in the environment (for example, the journal account password has changed or the server has run out of disk space). It is not normal behaviour for the server to stop all by itself. Stoppages should be extremely rare events.
Archiving ok
If the server is archiving correctly, blue and green bars will appear in Status->Charts. This behaviour indicates messages are being received from the mail server (depicted by blue bars) and archived correctly (denoted by green bars). Its perfectly fine for there to be more emails received than archived for a short period. During periods of low mail server activity (for example, late in the evening), the server will have the opportunity to catch up.
Archiving ignored
Red bars will appear if incoming traffic is ignored. A common mistake is to create an incorrectly defined route in Configuration->Routes. To resolve, go to Configuration->Routes and delete all routes. If this doesn't help, go to Configuration->Archive rules and verify that there is not an archiving rule defined that could cause all incoming traffic to be ignored.
Not receiving or archiving
If no bars are appearing at all, the server is not receiving any mails from the mail server.
Possible reasons for the server not receiving mail traffic are as follows:
- Everything's ok - There is simply no mail passing through your mail server at that moment.
- Legacy volume format in use - MailArchiva v8.12.21+ and MailArchiva v9+ no longer allow writing in legacy V1 and V2 volume formats.
- General misconfiguration - MailArchiva and the mail server are not set up correctly
- IMAP journaling issues - Resolutions for archiving problems related to IMAP journaling
- SMTP/Milter archiving issues - Resolutions to SMTP/Milter archiving problems
- No disk space left - One or more of your disks has run out of disk space.
The server will not archive any data if the ACTIVE volume is in legacy V1 or V2 formats. If you are still using legacy V1 and V2 volume formats, it will be necessary to switch to V3 volume format since V1 and V2 volume formats are now marked as read-only. To switch the volume format, go to Configuration→General→Archive, select volume format to V3. Save. Visit Configuration→Volumes, create a new volume and close the existing Active one.
MailArchiva/Mail server msconfiguration
Refer to Mail Servers for general configuration instructions.
First establish whether journaling traffic is being sent to MailArchiva. Examine the charts in Status->Charts. You should see blue bars for incoming traffic and green for archiving. If only blue bars appear, then ensure that an UNUSED or ACTIVE volume is configured in Configuration->Volumes. If red bars appear, go to Configuration->Routes and delete all routes and check that there is not an archiving rule in effect configured to ignore all incoming traffic. If no bars appear, verify the following:
- The SMTP listener is enabled in Configuration->Listeners from the MailArchiva web console. If not, enable it.
- There is no firewall present running on the MailArchiva server, blocking port 25 (e.g. Windows firewall). If so, the resolution is to open port 25 for incoming traffic on the firewall.
- There is not another application running on the MailArchiva server bound to port 25 (e.g. Postfix). If so, the resolution is to disable/uninstall the 3rd party application (e.g. Postfix) and restart MailArchiva or the SMTP listener so that it can bind to SMTP port 25.
- An SMTP listener was created to listen on port 25 and not an SMTP client connection. In this case, go to Configuration->Listeners and create an SMTP listener.
- If archiving from a mail service/server residing outside your network..
a) On your company firewall, ensure that a port forward rule is defined to forward port 25 traffic correctly from an external Ip address (of your choosing) to the MailArchiva server.
b) Using your DNS manager, ensure that an A record is defined for the MailArchiva server and points to the external Ip address referenced in a) above.
c) Some mail services, such as Microsoft 365, require that a TLS connection is enabled. For this purpose, ensure that a certificate and a key is imported into MailArchiva. - A journaling rule exists on the mail server. If the rule exists, verify the accuracy of the journal rule IP address +/ email routing address (for example, archive@mailarchiva.company.com (where mailarchiva.company.com resolves to the IP address of the MailArchiva server). The address must be in the format archive@[FQDN of the MailArchiva server] (i.e. do not enter an email address here!) FYI: Exchange resolves the [FQDN of the MailArchiva server] to the IP address of MailArchiva, and forwards SMTP traffic on port 25 to that IP address.
- That the fully qualified domain name (FQDN) of MailArchiva server is resolvable to the IP address of MailArchiva server from the mail server/service.
It is relatively easy to test the connection. If using an external mail service such as Microsoft 365, telnet to FQDN of the MailArchiva server (e.g. mailarchiva.company.com) on port 25 from any machine residing outside your local area network (LAN). If your mail server is running from within your LAN, telnet to the MailArchiva server FQDN from the machine running your mail server or alternatively from another machine running on your LAN.
root@:~# telnet mailarchiva.company.com 25
Trying 135.137.132.1...
If all is well, an MailArchiva SMTP header should be outputted on the console. If not... revisit the above troubleshooting steps, while taking further note of the following:
- Check that the firewall running on the MailArchiva machine is not blocking incoming traffic on port 25.
- Verify that the fully qualified domain name (FQDN) resolves to the correct IP address of the MailArchiva server. If an SMTP header response is received but has the wrong branding, there could be another service bound to port 25 or the FQDN could be resolving to the IP address of another server (i.e. not the MailArchiva server).
- If using an external mail service such as Microsoft 365, double check that the port forward rule is defined correctly on the firewall. Port 25 must be forwarded from an outside IP address (the one referenced in your DNS A record) to MailArchiva residing on your internal network.
Finally, one other thing to try. Clear your queues. Rename C:\ProgramData\MailArchiva\ROOT\queue\receive to C:\ProgramData\MailArchiva\ROOT\queue\restore_receive, Rename C:\ProgramData\MailArchiva\ROOT\queue\index to C:\ProgramData\MailArchiva\ROOT\queue\restore_index. Start the MA service.
- An IMAP Client Connection is not created and connected to the Exchange Journal Account, or the connection is disabled.
a) Ensure the IMAP Connection is created in Configuration->Connections and enabled. See Client Connections for more information.
b) Test connectivity to the mail server, by clicking in Test Connection in Configuration->Connections->IMAP Connection.
- The Journal Account password is expired/changed.
a) Reset the journal account password in Exchange
b) Ensure the journal account is not disabled
c) Ensure the account password has not expired
d) Ensure the journal account force password to change off
- The Exchange IMAP service is stopped or frozen.
Restart the IMAP Service on the mail server.
- There are too many messages (>20,000) in the journal account. See IMAP Journal Account Full.
a) Uncheck all checkboxes in Connections>IMAP (except enabled), lower the polling interval to 10 msec. Check to see if archiving speeds up. OR
b) Worst case scenario - Recreate the Journal account and reconfigure envelope journaling to forward mails to the new account. Use Exchange import to import old mail.
- The polling interval specified in Client Connections->IMAP may be set to a figure that is either too high or too low.
The polling interval should never be lower than 10 msec. If it is too high, it may not keep up with the mail flow in your organization and emails will build up the journal account. Once the journal account is too full, Exchange may find it difficult to deal with the large number of emails in the journal account.
- IMAP Idle is enabled - some mail servers, such as Microsoft Exchange, fail to deliver messages when IMAP Idle is enabled after a long period.
Disable IMAP Idle in IMAP Client Connection
- Lost connectivity to your mail server
a) Telnet to port 143 or 443 to the mail server address from your MailArchiva server. If there is no response (output), there is likely to be a basic connectivity issue.
b) Ensure there is no firewall anywhere along the network path to the mail server (including those running on both servers) blocking communications on port 143 or 443
c) Ensure there are no DNS issues at play by telnetting to the IP address of the Mail Server (i.e. not using the fully qualified domain name)
d) Verify that general network connectivity is available to both servers (e.g. check that the green link light on each network card is visible)
One or more of your disks has run out of space. Please check the disk space on all volume index and store locations, including the appropriate paths listed in File System Layout. Refer to system requirements for information on how much disk space is needed and for guidance on how your disks should be partitioned.
Receiving, yet not archiving
If only blue bars appear in Status->Charts, emails are received from the mail server, but cannot be archived and indexed. The unprocessed messages will reside in a local receive queue until the problem is resolved, whereupon the messages will be requeued for archiving.
Possible explanations for the server receiving mail but not archiving are as follows:
- Misconfiguration - Archiving may not occur due to a simple misconfiguration or misunderstanding of the product.
- Corrupted queue database - The queue database is corrupt. The resolution is to rename the queue database directory and restart the service.
- Corrupted graph database - The internal graph database is corrupted. The resolution is to rename the graph database directory and restart the service.
- Operating system or file permissions - If archiving to a remote storage device, the server may have insufficient permissions to write to it.
- Corrupted audit index - The audit index is corrupted. This can cause system instability and archiving may stop.
- Corrupted archive file - ZIP file spanning/splitting is not supported!!
- License troubles - The server may stop archiving due to an invalid license (only applicable to v2)
- Insufficient amount of memory allocated - Insufficient memory is allocated to the server.
- Not enough disk space Left - Insufficient disk space remains.
- Remote SMB/NFS share locking - The server is experiencing locking problems while archiving to remote storage
Server misconfigurations include:
- Encryption password not set.
Set an encryption password in Volumes.
- There is no UNUSED/ACTIVE volume configured. There must be at least one ACTIVE or UNUSED volume, in Configuration->Volumes.
Define a new UNUSED volume in Volumes.
- Insufficient permissions to write to the remote disk.
Refer to Network Attached Storage for further instructions.
- Invalid routes are specified in Configuration->Routes
Delete/adjust routing configurations in Configuration->Routes (note: try deleting all routes, as the server will be default archive everything)
- Unintended archive rules specified in Configuration->Archive Rules
Ensure that the defined archive rules have the desired consequences and are specified correctly.
Queue database corruption can occur in the event of sudden server power loss or due to unexpected/unscheduled termination of the MailArchiva process.
- Stop the MailArchiva service
- Rename [application data]\queue\receive [application data]\queue\restore_receive ([application data]/queue/receive to [application data]/queue/restore_receive)
- Rename [application data]\queue\index [application data]\queue\restore_index ([application data]/queue/index to [application data]/queue/restore_index)
- Start the MailArchiva service
The Orient DB graph database may be corrupted if sudden power loss occurs or the process is killed unexpectedly.
Examining the debug.log file in Configuration-Logs. If there are Orient DB related errors present:
- Stop the MailArchiva service
- Rename [application data]/database/archiva to [application data]/database/archiva.bak
- Start the MailArchiva service
Operating system & file permissions
If archiving to a network attached storage device, MailArchiva may not have sufficient privileges to write to the remote storage location.
The following error may be outputted in the Alerts:
The root cause of the error is revealed in the debug.log file (see Configuration->Logs). In most cases, the root cause is insufficient writing permissions on store path.
The resolution is one of the following:
- The MailArchiva email archiving server runs under the USER account and not the ADMINISTRATOR account.
Change the MailArchiva service logon account.
- Write permissions are not enabled in the parent directory of the store
You are sharing the store directory. Please share the parent directory of the store rather (Windows querk)
- MailArchiva does not run under an account with sufficient access rights.
Please make sure the MailArchiva service runs under an Administrator/root account
- The email archive server does not have write access to the NAS or SAN disk
The NAS or SAN disk is not connected or permissions are set incorrectly.
Refer to Network Attached Storage for further instructions.
The server will stop archiving or will become unstable if the audit log is corrupted.
In the event that an audit log is corrupted, an alert may appear in Status->Alerts. Alternatively, indexing related errors as described below will appear in the debug.log file.
java.lang.NullPointerException
at com.stimulus.archiva.index.LuceneIndex.commit(LuceneIndex.java:259)
at com.stimulus.archiva.index.LuceneIndex$IndexCommit.run(LuceneIndex.java:277)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
To resolve the audit corruption problem, simply reindex the audit log by clicking the Reindex button in Configuration->Logs. Once the reindexing process is complete, it is a good idea to restart the server.
When the alert below is received in Status->Alerts, one or more volume archive files may be corrupted:
Corruption may occur due to physical disk errors, or network problems caused by use of TCP/IP based network storage protocols (e.g. SMBFS / NFS). Experience suggests that the use of Network Attached Storage (NAS) devices that rely on TCP/IP protocols are error/corruption prone. It is rather recommended to use local storage or SAN's connected via Fibre Channel.
Under normal circumstances, when archiving, MailArchiva opens archive files in append mode only. In append mode, it is impossible for MailArchiva to overwrite and corrupt existing archive data. Data will only be rewritten if compaction is enabled in Configuration->Archive (which by default is disabled for safety reasons).
To recover from corruption, it is necessary to run a Zip Repair on the affected volume store archive files (.ZZ files). First identify the problem archive files and their location by examining the stack traces outputted in the debug.log file (accessible from Configuration->Logs). The affected archive files (i.e. ZZ file/s) will be located inside one of the volume store paths. Next, backup the original corrupted file for safe keeping. Thereafter, use a utility such as Sys Internals Zip Repair to repair the file and replace the original. Finally, update the server to the latest and start the server. The problem should be resolved.
In MailArchiva v2 (not MailArchiva v3), archiving may stop due to an invalid license. A license may be invalid due to one of the following:
- Your mailbox limit has exceeded the license quota. If this is the case, an appropriate alert will appear in Configuration->Alerts
Upgrade your license by logging to the MailArchiva website, select Licenses, choose License, click Extend, purchase, and install it in About.
- A corrupted index can lead to an invalid license. When MailArchiva conducts a license check, internally, it executes a search across all volumes. If an index is corrupt, the license check may fail.
Reindex the troublesome volume index by clicking on reindex in Configuration->Volumes. - The license file has been accidentally modified during transit
Obtain a new license file by logging into the MailArchiva website and downloading your license.
The server operating environment may not meet the minimum requirements, or the server may be configured to use an unsufficient amount of memory. Refer to Out of Memory for possible resolutions.
If archiving is stopped/slowed due to file locking issues, an error such as "The process cannot access the file because another process has locked a portion of the file" will appear in the server's debug log file.
Some poorly implemented NAS'es are incompatible with MailArchiva V2 format due to sophisticated locking requirements. To solve this, switch to using the volume V3 format. Alternatively, disable strict locking in the SMB or NFS drivers.
For example, for SMB shares, disable strict locking in smb.conf as follows:
strict locking = no
Found this information useful? Visit mailarchiva.com to learn more about MailArchiva.