Email Import

MailArchiva archives email data from the point at which the product is deployed. If there is the need to archive historical (pre-deployment) data, it can be imported separately using the product's import facility. MailArchiva has inbuilt facilities for importing messages from .EML (RFC2822), MBOX, PST, OST, and direct import from Microsoft Office 365, MS Exchange and Google Workspace. 

It is important to note that during import MailArchiva may temporarily fill up the index receive queue until it reaches a maximum configurable size. This can happen when importing data from very fast import sources (such as EML import from disk), as the time taken to move the import data to the receive queue is much faster than the time it takes to index, compress, encrypt and store the data in the volume store. I It is good practice to place the queues on a separate disk partition so that the server can continue to function effectively in the event the amount of data in the queues fill up all available disk space on the partition.

Preparation

1. Before importing data, ensure that an UNUSED or ACTIVE volume is available in Configuration->Volumes.
2. Ensure there is plenty of disk space on the partition where MailArchiva queues are located (/etc/opt/mailarchiva/ROOT/queues/receive or C:\ProgramData\MailArchiva\ROOT\queue\receive).
3. If there is limited disk space on the partition where MailArchiva's queues are located (see above for default location), move the queue location to a separate partition/disk with more space by editing bootstrap.conf file in /etc/opt/mailarchiva/ROOT or C:\ProgramData\MailArchiva\ROOT\conf and restart MA service.

Import From File

The import dialog is accessible by clicking the Import Data button in Configuration->Volumes, or from the top right User menu. Click the Import Data button in the Configuration->Volumes. Select EML, MBOX or PST as the import source.

Import From Client

To import emails from your workstation, simply drag 'n drop the emails from File Explorer onto the "Drop file here" location. MailArchiva streams the data from your client machine to a temporary location on the MailArchiva server. If importing a large number of files, zip them up and drop a zip file onto the drop location. Upon receiving a zip file, MailArchiva will automatically unpack it and import each file in the archive file automatically. 

Import From Server

The Import From Server option allows data to be imported from the local server. To import the data from the server running MailArchiva, copy the files to a directory on the server using File Explorer. Click the down arrow beside the source directory control, and select the folder where the files are located. To import recursively, enter “*” in the Include Directories field. Click Import Data to start the import process.

Note For Administrators: To prevent users from writing data to any path on the file system, it is essential that an Import path restriction is defined in Configuration->General->Limits. This setting constrains the import path such that it is specified relative to the restriction folder (i.e. CHROOT).

Post Import Action

When importing a large number of files, some files may not be imported successfully due to issues ranging from data corruption, unrecognized format to undetected mailbox owner email address (in the case of a PST file). To assist in identifying which files were processed successfully, and which were not, there is the option to either delete or add a prefix to those files that were successfully processed.

Import Sources

MailArchiva supports a variety of file import sources; they are:

• EML / MBOX 
• PST / OST
• Exchange Direct Import
• GSuite Import

After starting an import process, the import task may monitored and will run in the background until the task is complete or it is stopped.

EML / MBOX

EML (short for electronic mail), is a file extension for an email message saved to a file in the Internet Message Format protocol for electronic mail messages.  MBOX (short for mailbox) is a mail format used for organizing and storing multiple email messages in a single text file. Both these formats are commonly used for storing and exchanging email data between systems.

PST / OST

PST and OST files are Outlook Data Files. They contain not only email data, but also Outlook folders, calendar entries and contact entries. To import PST data from PST, visit Configuration->Volumes, click Import Data, and select PST as the Import Source. Drag 'n drop a PST file onto the "Drop here" box, and click Import Data. MailArchiva streams the data from the client machine to a temporary location on the MailArchiva server. 

PST Owner Primary Email Address

When presenting the tree view in the search interface, all imported PST folder entries link to the PST owner's primary email address. There are three ways that the PST owner's primary email address is determined:

• Automatically (default) - PST files are scanned to determine the primary email automatically. Sometimes, it is not possible for MailArchiva to determine the owner of the PST. For example, when a PST file contains data associated with multiple users (not a single user).
• Manually in the import dialog - Manually enter the owner's primary email address by clicking Advanced in the Import dialog, unchecking "Auto detect primary email address", and entering the owner's email address.
• Manually in pst_owner.txt - Specify a list of PST file names and their respective primary email address owner in the pst_owner.txt file (see Linking PST Data To Primary Mailbox Address below

Determing the PST owner automatically may take time, since it is necessary to scan the entire PST file to determine the owner. With "auto detect primary email address" unchecked, import is considerably faster as there is no need to scan the entire PST.  If the auto-detect primary email address option is disabled, then the specified owner's email will be used. If auto detect is enabled, and the system is unable to detect the owner, then the specified owner email will be applied as the default, unless it is not specified, in which case, the import of the PST will fail. 

Linking PST Data To Primary Mailbox Address

The folder view in the search interface needs to be able to assign imported folders and data from a PST file to a mailbox. MailArchiva assigns the contents of a PST to the mailbox owner's primary address in the following manner:

1. MailArchiva looks for a file called pst_owner.txt inside the uploaded ZIP file or in the root of the server's import path. The contents of the pst_owner.txt file should be as follows:
   
   [filename]=[mailbox emailaddress]
   
   For example:
   
   allen.pst=allen@company.com
   peter.pst=peter@company.com

The above assigns allen.pst, peter.pst to allen@company.com and peter@company.com, respectively.

2. Assuming no mapping in (1) above is available and the "auto detect primary email address" option is checked, MailArchiva will attempt to automatically detect the user's primary email address by analyzing the contents of the PST file.
3. If a primary email address can not be established in step (1) and (2) above, assuming it is specified, the fallback owner mailbox specified in the import dialog is used.
4. If neither of the above applies, import of the PST will fail altogether.

Exchange Direct Import.

 
MailArchiva is capable of performing direct imports of messages from Microsoft Exchange (2003 to 2019). Before an import is attempted, it is necessary to define an impersonation account with sufficient rights to access all mailboxes.
 

Prerequisites

1. Follow the steps Logins to setup Active Directory authentication
2. Setup an impersonation account in Exchange Impersonation
3. Create an Exchange client connection in Connections. Save changes.
   In the server address field, enter the fully qualified doman name (FQDN) of the Exchange server.  In the Impersonate Account field, enter the full User Principal Name of the impersonation account (e.g. user@company.local).
4. Run the Test Connection button in the Exchange connection. If the test fails, refer to Cannot Establish Exchange Connection.

Import Procedure

1. Be sure that a volume is configured in the MailArchiva console configuration
2. Click the Import Data button in the Volumes section
3. Select the newly created Exchange Connection as the Import source
    
   Note: The Exchange import source will only appear once the Exchange Client Connection connection has been created in Connections.
4. Enter the fully qualified domain name of the Exchange server
5. Press the import button.

GSuite Import

Prior to performing a GSuite import, the GSuite Archiving & Sync configuration steps must be performed.

Thereafter, perform the following steps:

1. In Configuration->Volumes click the Import Data button
2. Select the Google Connection created in GSuite Archiving & Sync as the source
3. Click the Import Button
4. Monitor the process of the Import, in System Status->Tasks

Monitoring the Import Task

After initiating the import process as described earlier, the import task may be monitored by either clicking on Status->Tasks in the top menu, or clicking Tasks in the top right menu. To view the output of the import process, click the magnifying glass. To stop the import process, click the stop icon. The results of an import process are also available in the import log accessible from Configuration->Logs.

Once an import process is started, it is safe to logout of the console. The import task will continue to run in the background. If system alerts are enabled, an alert will be sent via email when the import task is complete.