Tag Archives: attachments

mailReader.php — Parse E-mail and Save Attachments PHP, Version 2

by stuporglue Posted on April 14, 2012

UPDATE:

This script can now be found on GitHub. https://github.com/stuporglue/mailreader

One of most popular pages of all time is Recieve E-mail and Save Attachments with a PHP script. What was meant to be a quick hack that was only ever tested with Gmail ended up generating lots of support requests.

At first I suggested that people needed more robust email parsing use a dedicated library. But no one seemed to want to do the coding for that, so I ended up writing a new version which uses the PEAR mimeDecode.php library to do the parsing.

Without further ado, here’s mailReader.php!

E-mail Processing Script Features:

  1. Saves the e-mail sender, subject and body to a database
  2. Saves any attachments as files and creates an entry for those files in the database, associated with the e-mail info in #1
  3. Sends  a response back to the sender telling them what files were received and their file sizes
  4. Checks a list of allowed senders to make sure we only take files from specified addresses.

Database Setup:

If you’re going to use the database features, you’ll need a database. Here’s the SQL to create an identical setup to the one I have:

-- Here's my DB structureCREATE TABLE IF NOT EXISTS `emails` (
  `id` int(255) NOT NULL AUTO_INCREMENT,
  `from` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `subject` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `body` text COLLATE utf8_unicode_ci NOT NULL,
  `date` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1 ;

CREATE TABLE IF NOT EXISTS `files` (
  `id` int(255) NOT NULL AUTO_INCREMENT,
  `email_id` int(255) NOT NULL,
  `filename` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `size` varchar(20) COLLATE utf8_unicode_ci NOT NULL,
  `mime` varchar(100) COLLATE utf8_unicode_ci NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1 ;

Security

Make sure that your upload directory is out of your webroot. If someone emails you a malicious PHP script (eg. Virus.php) and can access it via the web, they could infect your server or your visitors. Many servers are configured to automatically treat .pl and .cgi as CGI scripts and run them as well. You do not want to create a way for untrusted users to upload files to your webroot!

With the file names in the database you can use Readfile to pass files down to users.

The Script: mailReader.php

Download it here.

#!/usr/bin/php -q
<?php
//  Use -q so that php doesn't print out the HTTP headers

/*
 * mailReader.php
 *
 * Recieve mail and attachments with PHP
 *
 * Usage:
 * This script expects to recieve raw emails via STDIN.
 *
 * Configure your mail server to pipe emails to this script. (See
 * http://stuporglue.org/add-an-email-address-that-forwards-to-a-script/
 * for instructions).  Make this script executable, and edit the
 * configuration options to suit your needs. Change permissions
 * of the directories so that the user executing the script (probably the
 * mail user) will have write permission to the file upload directory.
 *
 * By default the script is configured to save pdf, zip, jpg, png and gif files.
 * Edit the switch statements around line 200 to change this.
 *
 * Requirements:
 * You will need mimeDecode.php from http://pear.php.net/package/Mail_mimeDecode/
 * I used version 1.5.5
 *
 * Copyright 2012, Michael Moore
 * Licensed under the same terms as PHP itself. You are free to use this script
 * for personal or commercial projects. Use at your own risk. No guarantees or
 * warranties.
 *
 * Contact:
 * <[email protected]>
 * http://stuporglue.org
 *
 * Support:
 * Limited free support available in the comments on the webpage for this script
 * or via email. Contracted support available for specific projects.
 * http://stuporglue.org/mailreader-php-parse-e-mail-and-save-attachments-php-version-2/
 *
 * Thanks:
 * Many thanks to forahobby of www.360-hq.com for testing this script and helping me find
 * the initial bugs.
 * Thanks to Craig Hopson of twitterrooms.co.uk for help tracking down an iOS email handling bug.
 */

global $save_directory,$saved_files,$debug,$body;

/*
 *
 * 	Configuration Options
 *
 */

// What's the max # of seconds to try to process an email?
$max_time_limit = 600; 

// A safe place for files WITH TRAILING SLASH
// Malicious users could upload a php or executable file,
// so keep this out of your web root
$save_directory = "/a/safe/save/directory/";

// Allowed senders is now just the email part of the sender (no name part)
$allowed_senders = Array(
    '[email protected]',
    '[email protected]',
); 

// Send confirmation e-mail back to sender?
$send_email = FALSE; 

// Save e-mail message and file list to DB?
$save_msg_to_db = FALSE; 

// Configure your MySQL database connection here
$db_host = 'localhost';
$db_un = 'db_un';
$db_pass = 'db_pass';
$db_name = 'db_name';

$debug = FALSE;

/*
 *
 * 	End of Configuration Options
 *
 */

//Anything printed to STDOUT will be sent back to the sender as an error!
//error_reporting(-1);
//ini_set("display_errors", 1);

// Initialize the other global, set PHP options, load email library
$saved_files = Array();
set_time_limit($max_time_limit);
ini_set('max_execution_time',$max_time_limit);
require_once('mimeDecode.php');

// Some functions we'll use
function formatBytes($bytes, $precision = 2) {
    $units = array('B', 'KB', 'MB', 'GB', 'TB');

    $bytes = max($bytes, 0);
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024));
    $pow = min($pow, count($units) - 1);

    $bytes /= pow(1024, $pow);

    return round($bytes, $precision) . ' ' . $units[$pow];
} 

// Find a happy place! Find a happy place!
function saveFile($filename,$contents,$mimeType){
    global $save_directory,$saved_files,$debug;
    $filename = preg_replace('/[^a-zA-Z0-9_-]/','_',$filename);

    $unlocked_and_unique = FALSE;
    while(!$unlocked_and_unique){
	// Find unique
	$name = time() . "_" . $filename;
	while(file_exists($save_directory . $name)) {
	    $name = time() . "_" . $filename;
	}

	// Attempt to lock
	$outfile = fopen($save_directory.$name,'w');
	if(flock($outfile,LOCK_EX)){
	    $unlocked_and_unique = TRUE;
	}else{
	    flock($outfile,LOCK_UN);
	    fclose($outfile);
	}
    }

    fwrite($outfile,$contents);
    fclose($outfile);

    // This is for readability for the return e-mail and in the DB
    $saved_files[$name] = Array(
	'size' => formatBytes(filesize($save_directory.$name)),
	'mime' => $mimeType
    );
}

function decodePart($body_part){
    global $body,$debug;
    if(array_key_exists('name',$body_part->ctype_parameters)){ // everyone else I've tried
	$filename = $body_part->ctype_parameters['name'];
    }else if($body_part->ctype_parameters && array_key_exists('filename',$body_part->ctype_parameters)){ // hotmail
	$filename = $body_part->ctype_parameters['filename'];
    }else{
	$filename = "file";
    }

    if($debug){
	print "Found body part type {$body_part->ctype_primary}/{$body_part->ctype_secondary}\n";
    }

    $mimeType = "{$body_part->ctype_primary}/{$body_part->ctype_secondary}"; 

    switch($body_part->ctype_primary){
    case 'text':
	switch($body_part->ctype_secondary){
	case 'plain':
	    $body = $body_part->body; // If there are multiple text/plain parts, we will only get the last one.
	    break;
	}
	break;
    case 'application':
	switch ($body_part->ctype_secondary){
	case 'pdf': // save these file types
	case 'zip':
	case 'octet-stream':
	    saveFile($filename,$body_part->body,$mimeType);
	    break;
	default:
	    // anything else (exe, rar, etc.) will faill into this hole and die
	    break;
	}
	break;
    case 'image':
	switch($body_part->ctype_secondary){
	case 'jpeg': // Save these image types
	case 'png':
	case 'gif':
	    saveFile($filename,$body_part->body,$mimeType);
	    break;
	default:
	    break;
	}
	break;
    case 'multipart':
	if(is_array($body_part->parts)){
	    foreach($body_part->parts as $ix => $sub_part){
		decodePart($sub_part);
	    }
	}
	break;
    default:
	// anything else isn't handled
	break;
    }
}

//
// Actual email handling starts here!
// 

// Process the e-mail from stdin
$fd = fopen('php://stdin','r');
$raw = '';
while(!feof($fd)){ $raw .= fread($fd,1024); }

// Uncomment this for debugging.
// Then you can do
// cat /my/saved/file.raw | ./mailReader.php
// for testing
//file_put_contents("$save_directory/" . time() . "_email.raw",$raw);

// Now decode it!
// http://pear.php.net/manual/en/package.mail.mail-mimedecode.decode.php
$decoder = new Mail_mimeDecode($raw);
$decoded = $decoder->decode(
    Array(
	'decode_headers' => TRUE,
	'include_bodies' => TRUE,
	'decode_bodies' => TRUE,
    )
);

// Set $from_email and check if it's allowed
$from = $decoded->headers['from'];
$from_email = preg_replace('/.*<(.*)>.*/',"$1",$from);
if(!in_array($from_email,$allowed_senders)){
    die("$from_email not an allowed sender");
}

// Set the $subject
$subject = $decoded->headers['subject'];

// Find the email body, and any attachments
// $body_part->ctype_primary and $body_part->ctype_secondary make up the mime type eg. text/plain or text/html
if(is_array($decoded->parts)){
    foreach($decoded->parts as $idx => $body_part){
	decodePart($body_part);
    }
}

// $from_email, $subject and $body should be set now. $saved_files should have
// the files we captured

// Put the results in the database if needed
if($save_msg_to_db){
    mysql_connect($db_host,$db_un,$db_pass);
    mysql_select_db($db_name);

    $q = "INSERT INTO `emails` (`from`,`subject`,`body`) VALUES ('" .
	mysql_real_escape_string($from_email) . "','" .
	mysql_real_escape_string($subject) . "','" .
	mysql_real_escape_string($body) . "')";

    mysql_query($q) or die(mysql_error());

    if(count($saved_files) > 0){
	$id = mysql_insert_id();
	$q = "INSERT INTO `files` (`email_id`,`filename`,`size`,`mime`) VALUES ";
	$filesar = Array();
	foreach($saved_files as $f => $data){
	    $filesar[] = "('$id','" .
		mysql_real_escape_string($f) . "','" .
		mysql_real_escape_string($data['size']) . "','" .
		mysql_real_escape_string($data['mime']) . "')";
	}
	$q .= implode(', ',$filesar);
	mysql_query($q) or die(mysql_error());
    }
}

// Send response e-mail if needed
if($send_email && $from_email != ""){
    $to = $from_email;
    $newmsg = "Thanks! I just uploaded the following ";
    $newmsg .= "files to your storage:\n\n";
    $newmsg .= "Filename -- Size\n";
    foreach($saved_files as $f => $s){
	$newmsg .= "$f -- $s\n";
    }
    $newmsg .= "\nI hope everything looks right. If not,";
    $newmsg .=  "please send me an e-mail!\n";

    mail($to,$subject,$newmsg);
}

if($debug){
    print "From : $from_email\n";
    print "Subject : $subject\n";
    print "Body : $body\n";
    print "Saved Files : \n";
    print_r($saved_files);
}

Thanks

Many thanks to forahobby for testing this script and helping me squash a bunch of little bugs. Thanks to Craig Hopson for his help finding a problem handling emails from iOS devices.

Posted in Computers, Programming, Something Interesting | Tagged , , , | 89 Comments

Recieve E-mail and Save Attachments with a PHP script

by stuporglue Posted on March 24, 2010

EDIT 4/14/2012 — There’s a new version of this script available which uses PEAR’s mimeDecode library to decode the email instead of the cobbled mess you see on this page. Go there now instead of wasting your time on this outdated page!

Here’s something fun! If you can tell your server to send e-mail to a script, you can send e-mails to PHP. Once you are processing the e-mail with PHP you can save attachments, automatically respond to the e-mail, save it to a database, make a webpage from it…really whatever you want!

Here’s a script I am currently using. My brother is on a mission for our Church, in Peru for two years and has near weekly e-mail access but can’t do much more than e-mail. He wanted a way to easily send photos to the server via e-mail; this script is the results of his wishes.

E-mail Processing Script Features:

  1. Saves the e-mail sender, subject and body to a database
  2. Saves any attachments as files and creates an entry for those files in the database, associated with the e-mail info in #1
  3. Sends  a response back to the sender telling them what files were received and their file sizes
  4. Checks a list of allowed senders to make sure we only take files from specified addresses.

Database Setup:

If you’re going to use the database features, you’ll need a database. Here’s the SQL to create an identical setup to the one I have:

-- Here's my DB structure

SET SQL_MODE="NO_AUTO_VALUE_ON_ZERO";
-- Table structure for table `emails`

CREATE TABLE IF NOT EXISTS `emails` (
  `id` int(100) NOT NULL AUTO_INCREMENT,
  `from` varchar(250) NOT NULL,
  `subject` text NOT NULL,
  `body` text NOT NULL,
  `date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1;

-- --------------------------------------------------------
-- Table structure for table `files`

CREATE TABLE IF NOT EXISTS `files` (
  `id` int(100) NOT NULL AUTO_INCREMENT,
  `email_id` int(100) NOT NULL,
  `filename` varchar(255) NOT NULL,
  `size` varchar(100) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1;

Pretty straightforward I would say. The size field in the files table stores a user friendly “100Mb” type description of the size. You will need to know your database name, username, password and host name in the next step.

The Email Handling Script:

The max_time_limit variable is how long you want the script to be allowed to run. The default max for your server might be too small to handle 20Mb of attachments (the max you can send with Google).

#!/usr/bin/php -q
<?php
//  Use -q so that php doesn't print out the HTTP headers
//  Anything printed to STDOUT will be sent back to the sender as an error!

//  Config options

$max_time_limit = 600; // in seconds
// A safe place for files with trailing slash (malicious users could upload a php or executable file!)
$save_directory = "/some/folder/path";
$allowedSenders = Array('[email protected]',
    'Bob the Builder <[email protected]>'); // only people you trust!

$send_email = TRUE; // Send confirmation e-mail?
$save_msg_to_db = TRUE; // Save e-mail body to DB?

$db_host = 'localhost';
$db_un = 'db_un';
$db_pass = 'password';
$db_name = 'db_name';

// ------------------------------------------------------

set_time_limit($max_time_limit);
ini_set('max_execution_time',$max_time_limit);

global $from, $subject, $boundary, $message, $save_path,$files_uploaded;
$save_path = $save_directory;
$files_uploaded = Array();

function formatBytes(&$bytes, $precision = 2) {
    $units = array('B', 'KB', 'MB', 'GB', 'TB');

    $bytes = max($bytes, 0);
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024));
    $pow = min($pow, count($units) - 1);

    $bytes /= pow(1024, $pow);

    return round($bytes, $precision) . ' ' . $units[$pow];
} 

function process_part(&$email_part){
    global $message;

    // Max two parts. The data could have more than one \n\n in it somewhere,
    // but the first \n\n should be after the content info block
    $parts = explode("\n\n",$email_part,2);

    $info = split("\n",$parts[0]);
    $type;
    $name;
    $encoding;
    foreach($info as $line){
	if(preg_match("/Content-Type: (.*);/",$line,$matches)){
	    $type = $matches[1];
	}
	if(preg_match("/Content-Disposition: attachment; filename=\"(.*)\"/",
	    $line,$matches)){
	    $name = time() . "_" . $matches[1];
	}
	if(preg_match("/Content-Transfer-Encoding: (.*)/",$line,$matches)){
	    $encoding = $matches[1];
	}
    }

    // We don't know what it is, so we don't know how to process it
    if(!isset($type)){ return FALSE; }

    switch($type){
    case 'text/plain':
	// "But if you get a text attachment, you're going to overwrite
	// the real message!" Yes. I don't care in this case...
	$message = $parts[1];
	break;
    case 'multipart/alternative':
	// Multipart comes where the client sends the data in two formats so
	// that recipients who can't read (or don't like) fancy content
	// have another way to read it. Eg. When sending an html formatted
	// message, they will also send a plain text message
	process_multipart($info,$parts[1]);
	break;
    default:
	if(isset($name)){ // the main message will not have a file name...
	    // text/html messages won't be saved!
	    process_data($name,$encoding,$parts[1]);
	}elseif(!isset($message) && strpos($type,'text') !== FALSE){
	    $message = $parts[1]; // Going out on a limb here...capture
	    // the message
	}
	break;
    }
}

function process_multipart(&$info,&$data){
    global $message;

    $bounds;
    foreach($info as $line){
	if (preg_match("/boundary=(.*)$/",$line,$matches)){
	    $bounds = $matches[1];
	}
    }

    $multi_parts = split("--" .$bounds,$data);
    for($i = 1;$i < count($multi_parts);$i++){
	process_part($multi_parts[$i]);
    }
}

function process_data(&$name,&$encoding = 'base64' ,&$data){
    global $save_path,$files_uploaded;

    // find a filename that's not in use. There's a race condition
    // here which should be handled with flock or something instead
    // of just checking for a free filename

    $unlocked_and_unique = FALSE;
    while(!$unlocked_and_unique){
	// Find unique
	$name = time() . "_" . $name;
	while(file_exists($save_path . $name)) {
	    $name = time() . "_" . $name;
	}

	// Attempt to lock
	$outfile = fopen($save_path.$name,'w');
	if(flock($outfile,LOCK_EX)){
	    $unlocked_and_unique = TRUE;
	}else{
	    flock($outfile,LOCK_UN);
	    fclose($outfile);
	}
    }

    if($encoding == 'base64'){
	fwrite($outfile,base64_decode($data));
    }elseif($encoding == 'uuencode'){
	// I haven't actually seen this in an e-mail, but older clients may
	// still use it...not 100% sure that this will work correctly as is
	fwrite($outfile,convert_uudecode($data));
    }
    flock($outfile,LOCK_UN);
    fclose($outfile);

    // This is for readability for the return e-mail and in the DB
    $files_uploaded[$name] = formatBytes(filesize($save_path.$name));
}

// Process the e-mail from stdin
$fd = fopen('php://stdin','r');
$email = '';
while(!feof($fd)){ $email .= fread($fd,1024); }

// Headers hsould go till the first \n\n. Grab everything before that, then
// split on \n and process each line
$headers = split("\n",array_shift(explode("\n\n",$email,2)));
foreach($headers as $line){
    // The only 3 headers we care about...
    if (preg_match("/^Subject: (.*)/", $line, $matches)) {
	$subject = $matches[1];
    }
    if (preg_match("/^From: (.*)/", $line, $matches)) {
	$from = $matches[1];
    }
    if (preg_match("/boundary=(.*)$/",$line,$matches)){
	$boundary = $matches[1];
    }
}

// Check $from here and exit if it's blank or
// not someone you want to get mail from!
if(!in_array($from,$allowedSenders)){
    die("Not an allowed sender");
}

// No boundary was in the e-mail sent to us. We don't know what to do!
if(!isset($boundary)){
    die("I couldn't find an e-mail boundary. Maybe this isn't an e-mail");
}

// Split the e-mail on the found boundary
// The first part will be the header (hence $i = 1 in our loop)
// Each other chunk should have some info on the chunk,
// then \n\n then the chunk data
// Process each chunk
$email_parts = split("--" . $boundary,$email);
for($i = 1;$i < count($email_parts);$i++){
    process_part($email_parts[$i]);
}

// Put the results in the database if needed
if($save_msg_to_db){
    mysql_connect($db_host,$db_un,$db_pass);
    mysql_select_db($db_name);

    $q = "INSERT INTO `emails` (`from`,`subject`,`body`) VALUES ('" .
	mysql_real_escape_string($from) . "','" .
	mysql_real_escape_string($subject) . "','" .
	mysql_real_escape_string($message) . "')";

    mysql_query($q) or die(mysql_error());

    if(count($files_uploaded) > 0){
	$id = mysql_insert_id();
	$q = "INSERT INTO `files` (`email_id`,`filename`,`size`) VALUES ";
	$filesar = Array();
	foreach($files_uploaded as $f => $s){
	    $filesar[] = "('$id','" .
		mysql_real_escape_string($f) . "','" .
		mysql_real_escape_string($s) . "')";
	}
	$q .= implode(', ',$filesar);
	mysql_query($q) or die(mysql_error());
    }
}

// Send response e-mail if needed
if($send_email && $from != ""){
    $to = $from;
    $newmsg = "Thanks! I just uploaded the following ";
    $newmsg .= "files to your storage:\n\n";
    $newmsg .= "Filename -- Size\n";
    foreach($files_uploaded as $f => $s){
	$newmsg .= "$f -- $s\n";
    }
    $newmsg .= "\nI hope everything looks right. If not,";
    $newmsg .=  "please send me an e-mail!\n";

    mail($to,$subject,$newmsg);
}

Testing The Script:

Save an e-mail, headers and all, and upload it to your server. Cat the saved e-mail to your script to test it.

cat 'saved_email.txt' | ./process_email.php

You should get an e-mail response, see new entries in your DB and see your saved attachments. If you don’t, you can print debugging statements or use your usual PHP debugging techniques with this easy testing method.

Security:

Don’t let anonymous people upload files to your server without appropriate precautions! If someone uploaded a PHP file they could run it and easily gain access to your server that way. Protect the upload destination directory and limit who can send e-mail to this script.  There are probably a bunch of glaring security holes in this script. In my limited use situation where only two people knows the only address that can send e-mail to this script and the address to send it to, and where only I have access to the uploaded files (outside of the server root!) I’m comfortable with the level of security. For your situation, you might need to be more cautious.

EDIT: Sending Email to the Script

Initially I didn’t have any instructions on how to forward email to the script you just made. I have now posted those instructions here: Add an Email Address That Forwards to a Script. It has instructions for CPanel and a link for Exim, Sendmail and Qmail users.

Enjoy!

Posted in Programming, Projects | Tagged , , , , , , | 57 Comments