mailReader.php — Parse E-mail and Save Attachments PHP, Version 2

EDIT 4/14/2012 — There’s a new version of this script available which uses PEAR’s mimeDecode library to decode the email instead of the cobbled mess you see on this page. Go there now instead of wasting your time on this outdated page!

UPDATE:

This script can now be found on GitHub. https://github.com/stuporglue/mailreader

One of most popular pages of all time is Recieve E-mail and Save Attachments with a PHP script. What was meant to be a quick hack that was only ever tested with Gmail ended up generating lots of support requests.

At first I suggested that people needed more robust email parsing use a dedicated library. But no one seemed to want to do the coding for that, so I ended up writing a new version which uses the PEAR mimeDecode.php library to do the parsing.

Without further ado, here’s mailReader.php!

E-mail Processing Script Features:

  1. Saves the e-mail sender, subject and body to a database
  2. Saves any attachments as files and creates an entry for those files in the database, associated with the e-mail info in #1
  3. Sends  a response back to the sender telling them what files were received and their file sizes
  4. Checks a list of allowed senders to make sure we only take files from specified addresses.

Database Setup:

If you’re going to use the database features, you’ll need a database. Here’s the SQL to create an identical setup to the one I have:

-- Here's my DB structureCREATE TABLE IF NOT EXISTS `emails` (
  `id` int(255) NOT NULL AUTO_INCREMENT,
  `from` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `subject` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `body` text COLLATE utf8_unicode_ci NOT NULL,
  `date` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1 ;

CREATE TABLE IF NOT EXISTS `files` (
  `id` int(255) NOT NULL AUTO_INCREMENT,
  `email_id` int(255) NOT NULL,
  `filename` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `size` varchar(20) COLLATE utf8_unicode_ci NOT NULL,
  `mime` varchar(100) COLLATE utf8_unicode_ci NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1 ;

Security

Make sure that your upload directory is out of your webroot. If someone emails you a malicious PHP script (eg. Virus.php) and can access it via the web, they could infect your server or your visitors. Many servers are configured to automatically treat .pl and .cgi as CGI scripts and run them as well. You do not want to create a way for untrusted users to upload files to your webroot!

With the file names in the database you can use Readfile to pass files down to users.

The Script: mailReader.php

Download it here.

#!/usr/bin/php -q
<?php
//  Use -q so that php doesn't print out the HTTP headers

/*
 * mailReader.php
 *
 * Recieve mail and attachments with PHP
 *
 * Usage:
 * This script expects to recieve raw emails via STDIN.
 *
 * Configure your mail server to pipe emails to this script. (See
 * http://stuporglue.org/add-an-email-address-that-forwards-to-a-script/
 * for instructions).  Make this script executable, and edit the
 * configuration options to suit your needs. Change permissions
 * of the directories so that the user executing the script (probably the
 * mail user) will have write permission to the file upload directory.
 *
 * By default the script is configured to save pdf, zip, jpg, png and gif files.
 * Edit the switch statements around line 200 to change this.
 *
 * Requirements:
 * You will need mimeDecode.php from http://pear.php.net/package/Mail_mimeDecode/
 * I used version 1.5.5
 *
 * Copyright 2012, Michael Moore
 * Licensed under the same terms as PHP itself. You are free to use this script
 * for personal or commercial projects. Use at your own risk. No guarantees or
 * warranties.
 *
 * Contact:
 * <stuporglue@gmail.com>
 * http://stuporglue.org
 *
 * Support:
 * Limited free support available in the comments on the webpage for this script
 * or via email. Contracted support available for specific projects.
 * http://stuporglue.org/mailreader-php-parse-e-mail-and-save-attachments-php-version-2/
 *
 * Thanks:
 * Many thanks to forahobby of www.360-hq.com for testing this script and helping me find
 * the initial bugs.
 * Thanks to Craig Hopson of twitterrooms.co.uk for help tracking down an iOS email handling bug.
 */

global $save_directory,$saved_files,$debug,$body;

/*
 *
 * 	Configuration Options
 *
 */

// What's the max # of seconds to try to process an email?
$max_time_limit = 600; 

// A safe place for files WITH TRAILING SLASH
// Malicious users could upload a php or executable file,
// so keep this out of your web root
$save_directory = "/a/safe/save/directory/";

// Allowed senders is now just the email part of the sender (no name part)
$allowed_senders = Array(
    'myemail@example.com',
    'whatever@example.com',
); 

// Send confirmation e-mail back to sender?
$send_email = FALSE; 

// Save e-mail message and file list to DB?
$save_msg_to_db = FALSE; 

// Configure your MySQL database connection here
$db_host = 'localhost';
$db_un = 'db_un';
$db_pass = 'db_pass';
$db_name = 'db_name';

$debug = FALSE;

/*
 *
 * 	End of Configuration Options
 *
 */

//Anything printed to STDOUT will be sent back to the sender as an error!
//error_reporting(-1);
//ini_set("display_errors", 1);

// Initialize the other global, set PHP options, load email library
$saved_files = Array();
set_time_limit($max_time_limit);
ini_set('max_execution_time',$max_time_limit);
require_once('mimeDecode.php');

// Some functions we'll use
function formatBytes($bytes, $precision = 2) {
    $units = array('B', 'KB', 'MB', 'GB', 'TB');

    $bytes = max($bytes, 0);
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024));
    $pow = min($pow, count($units) - 1);

    $bytes /= pow(1024, $pow);

    return round($bytes, $precision) . ' ' . $units[$pow];
} 

// Find a happy place! Find a happy place!
function saveFile($filename,$contents,$mimeType){
    global $save_directory,$saved_files,$debug;
    $filename = preg_replace('/[^a-zA-Z0-9_-]/','_',$filename);

    $unlocked_and_unique = FALSE;
    while(!$unlocked_and_unique){
	// Find unique
	$name = time() . "_" . $filename;
	while(file_exists($save_directory . $name)) {
	    $name = time() . "_" . $filename;
	}

	// Attempt to lock
	$outfile = fopen($save_directory.$name,'w');
	if(flock($outfile,LOCK_EX)){
	    $unlocked_and_unique = TRUE;
	}else{
	    flock($outfile,LOCK_UN);
	    fclose($outfile);
	}
    }

    fwrite($outfile,$contents);
    fclose($outfile);

    // This is for readability for the return e-mail and in the DB
    $saved_files[$name] = Array(
	'size' => formatBytes(filesize($save_directory.$name)),
	'mime' => $mimeType
    );
}

function decodePart($body_part){
    global $body,$debug;
    if(array_key_exists('name',$body_part->ctype_parameters)){ // everyone else I've tried
	$filename = $body_part->ctype_parameters['name'];
    }else if($body_part->ctype_parameters && array_key_exists('filename',$body_part->ctype_parameters)){ // hotmail
	$filename = $body_part->ctype_parameters['filename'];
    }else{
	$filename = "file";
    }

    if($debug){
	print "Found body part type {$body_part->ctype_primary}/{$body_part->ctype_secondary}\n";
    }

    $mimeType = "{$body_part->ctype_primary}/{$body_part->ctype_secondary}"; 

    switch($body_part->ctype_primary){
    case 'text':
	switch($body_part->ctype_secondary){
	case 'plain':
	    $body = $body_part->body; // If there are multiple text/plain parts, we will only get the last one.
	    break;
	}
	break;
    case 'application':
	switch ($body_part->ctype_secondary){
	case 'pdf': // save these file types
	case 'zip':
	case 'octet-stream':
	    saveFile($filename,$body_part->body,$mimeType);
	    break;
	default:
	    // anything else (exe, rar, etc.) will faill into this hole and die
	    break;
	}
	break;
    case 'image':
	switch($body_part->ctype_secondary){
	case 'jpeg': // Save these image types
	case 'png':
	case 'gif':
	    saveFile($filename,$body_part->body,$mimeType);
	    break;
	default:
	    break;
	}
	break;
    case 'multipart':
	if(is_array($body_part->parts)){
	    foreach($body_part->parts as $ix => $sub_part){
		decodePart($sub_part);
	    }
	}
	break;
    default:
	// anything else isn't handled
	break;
    }
}

//
// Actual email handling starts here!
// 

// Process the e-mail from stdin
$fd = fopen('php://stdin','r');
$raw = '';
while(!feof($fd)){ $raw .= fread($fd,1024); }

// Uncomment this for debugging.
// Then you can do
// cat /my/saved/file.raw | ./mailReader.php
// for testing
//file_put_contents("$save_directory/" . time() . "_email.raw",$raw);

// Now decode it!
// http://pear.php.net/manual/en/package.mail.mail-mimedecode.decode.php
$decoder = new Mail_mimeDecode($raw);
$decoded = $decoder->decode(
    Array(
	'decode_headers' => TRUE,
	'include_bodies' => TRUE,
	'decode_bodies' => TRUE,
    )
);

// Set $from_email and check if it's allowed
$from = $decoded->headers['from'];
$from_email = preg_replace('/.*<(.*)>.*/',"$1",$from);
if(!in_array($from_email,$allowed_senders)){
    die("$from_email not an allowed sender");
}

// Set the $subject
$subject = $decoded->headers['subject'];

// Find the email body, and any attachments
// $body_part->ctype_primary and $body_part->ctype_secondary make up the mime type eg. text/plain or text/html
if(is_array($decoded->parts)){
    foreach($decoded->parts as $idx => $body_part){
	decodePart($body_part);
    }
}

// $from_email, $subject and $body should be set now. $saved_files should have
// the files we captured

// Put the results in the database if needed
if($save_msg_to_db){
    mysql_connect($db_host,$db_un,$db_pass);
    mysql_select_db($db_name);

    $q = "INSERT INTO `emails` (`from`,`subject`,`body`) VALUES ('" .
	mysql_real_escape_string($from_email) . "','" .
	mysql_real_escape_string($subject) . "','" .
	mysql_real_escape_string($body) . "')";

    mysql_query($q) or die(mysql_error());

    if(count($saved_files) > 0){
	$id = mysql_insert_id();
	$q = "INSERT INTO `files` (`email_id`,`filename`,`size`,`mime`) VALUES ";
	$filesar = Array();
	foreach($saved_files as $f => $data){
	    $filesar[] = "('$id','" .
		mysql_real_escape_string($f) . "','" .
		mysql_real_escape_string($data['size']) . "','" .
		mysql_real_escape_string($data['mime']) . "')";
	}
	$q .= implode(', ',$filesar);
	mysql_query($q) or die(mysql_error());
    }
}

// Send response e-mail if needed
if($send_email && $from_email != ""){
    $to = $from_email;
    $newmsg = "Thanks! I just uploaded the following ";
    $newmsg .= "files to your storage:\n\n";
    $newmsg .= "Filename -- Size\n";
    foreach($saved_files as $f => $s){
	$newmsg .= "$f -- $s\n";
    }
    $newmsg .= "\nI hope everything looks right. If not,";
    $newmsg .=  "please send me an e-mail!\n";

    mail($to,$subject,$newmsg);
}

if($debug){
    print "From : $from_email\n";
    print "Subject : $subject\n";
    print "Body : $body\n";
    print "Saved Files : \n";
    print_r($saved_files);
}

Thanks

Many thanks to forahobby for testing this script and helping me squash a bunch of little bugs. Thanks to Craig Hopson for his help finding a problem handling emails from iOS devices.

This entry was posted in Computers, Programming, Something Interesting and tagged , , , . Bookmark the permalink.

94 Responses to mailReader.php — Parse E-mail and Save Attachments PHP, Version 2

  1. Pingback: Recieve E-mail and Save Attachments with a PHP script » Stuporglue.org

  2. Craig Hopson says:

    Awsome script so simple to use and setup FULL credit to Michael and his hard work…..Keep it up

  3. Craig Hopson says:

    Hey Michael if i wanted to save the attachments in say “public_html/uploads/” could i do this

    $subject = holiday // This value would be set by the email recived

    “public_html/uploads/$subject” if the directory is already there read

    • stuporglue says:

      Yes, but this can be very dangerous.

      If I sent you an email with the subject “../” and a file named “index.php”?

      It would go to public_html/uploads/../index.php, in other words, it would wipe out your homepage’s default index.php.

      When uploading user files you nearly always want to avoid putting them in the public_html directory or a subdirectory. If I upload “myspampage.php”, I can make you host my spam, and use your trustworthy URL for my nefarious purposes.

      In the end it’s your server and you know the audience, security settings and other factors that will lead your decision, but please be warned!

      • Craig Hopson says:

        Ok i understand that but what about this

        case 'application':
        switch ($body_part->ctype_secondary){
        case 'pdf': // save these file types
        case 'zip':
        case 'octet-stream':
        saveFile($filename,$body_part->body,$mimeType);
        break;
        default:
        // anything else (exe, rar, etc.) will faill into this hole and die
        break;
        }
        break;

        would this not stop and i quote “anything else (exe, rar, etc.) will faill into this hole and die”
        Also why we are here is it possible to put a custom error email before the break saying such.

        Thanks

      • stuporglue says:

        It’s all about good security practices. You can, and should, sanitize user input with regexes, switch cases and white lists. If you can also add another layer of security by making it impossible to upload files to the webroot then, in my opinion, it’s worth doing.

        If you’re comfortable without that last layer of protection, then that’s up to you.

      • Craig Hopson says:

        how about this

        // Set the $subject
        $subjectB = $decoded->headers['subject'];
        $subject = ereg_replace("[^0-9]", "", $subjectB);
        // Find the email body, and any attachments

  4. Craig Hopson says:

    Hi once again big fan of this script using on my site but yet another question is it possible to have custom emails for errors like IF someone trys to upload a .exe or .php it will send back a sorry not allowed!

  5. Ed Blackwood says:

    Sorry to be such a newbie, but what do I do with the mimeDecode.php file? Do I have to do something other than put it in the script directory where the mailreader.php is?

  6. Ed Blackwood says:

    Thanks. Now, if I browse to the PHP file I get some output in the error log and it fails (of course) when checking the “from” list since I am not calling it from an email. And, I have set up the forwarder. But, when I send an email I get no output in the error_log nor an attachment in the save_directory. Any suggestions?

    Thanks,
    Ed

    • stuporglue says:

      If you have SSH to your server, I would connect to your server and verify that you can run the PHP script from the command line. You can do something as simple as:

      echo “Make an error” | ./mailReader.php

      If that doesn’t work, check that your PHP is located at /usr/bin/php (line 1 of the script) and that the script is executable.

  7. David says:

    hi, wondering if you can help…

    unfortunately i don’t have the option within my 1and1 control panel to forward email to a script. only to another email address.

    is there some way i can connect to the mailbox at the start of your script and get the latest email to then use with the script?

    many thanks

    • stuporglue says:

      It certainly can be done, but I’m afraid that’s outside the scope of this script.

      I haven’t done this before, so I don’t have a specific class or script to recommend, but you’re going to be looking for either an IMAP class/script or a POP3 class/script. This is probably a good place to start http://www.php.net/manual/en/refs.remote.mail.php

      You would use POP or IMAP to fetch the emails, then you could use my script to do the processing/saving to the database.

  8. erron says:

    im getting an error piping mine, i get the following as a reply email:

    This message was created automatically by mail delivery software.
    A message that you sent could not be delivered to one or more of its
    recipients. This is a permanent error. The following address(es) failed:
    pipe to |/home/xxx/public_html/mailReader.php
    generated by testpipe@mydomain.com.au
    local delivery failed

    any thoughts?

  9. David says:

    Hi there
    This looks like a brilliant piece of code — I’m getting an error notice and I wonder if you had any thoughts?

    Notice: Undefined property: stdClass::$d_parameters in parse.php on line 152

    Beyond that, it works as far as it extracts the header and plain text body, and will insert these in the SQL DB, but any file attachment or HTML email content is not captured.

    Anything I can check?

    many thanks

    David

    • stuporglue says:

      David,

      My apologies, I’m not sure how that happened. I had a typo. The 3 instances of ->d_parameters should have been ->ctype_parameter. I have fixed it here, you can re-download or just change it in your code.

      Thanks,
      Michael

  10. Craig Hopson says:

    Hi Michael,

    Been using this script from day 1 just getting round the tweeking it i’ve added this case (below) but i just wont work it say’s “Thanks! I just uploaded the following files to your storage” where am i going wrong?


    case 'NON SAVE':
    switch ($body_part->ctype_secondary){
    case 'php':
    case 'txt':
    case 'html':
    mail('$from_email','error','file incorrect');
    die("not an allowed file");
    break;
    default:

    • stuporglue says:

      If that’s going inside of the outer switch block, then it’s because ‘NON SAVE’ is never a ctype_primary. ctype_primary and ctype_secondary are the two parts of the mime-type.

      switch($body_part->ctype_primary){
      ...
      case 'NON SAVE':
      switch ($body_part->ctype_secondary){
      case 'php':
      case 'txt':
      case 'html':
      mail('$from_email','error','file incorrect');
      die("not an allowed file");
      break;
      default:
      ...
      }

      Wikipedia has some of the more common mime types listed here: http://en.wikipedia.org/wiki/Internet_media_type

      If that’s not where your switch statement is going, then I’m not sure where it fits in.

      • Craig Hopson says:

        Basically i want a error email to be sent if file type is not supported
        (png, jpeg, gif) ect

      • stuporglue says:

        Files that are not supported fall into the “default” sections of the switch cases. You’ve got a couple of options depending on your desired behavior.

        If you want it to fail/email immediately, then you’re going to run into a couple of problems. If someone emails two files and the first is supported, but the second isn’t, then you’ll send out an email after the first file is already saved. If the first is unsupported, then the supported file will not be saved. From a user perspective this sort of behavior will seem inconsistent (why should it matter to them which order they’re attached in?).

        If you want to not upload anything if any unsupported files are present, then you’ll need to first check the mime types of all files before saving any of them.

        If you want to upload the supported types and then tell the user which file(s) weren’t supported, you’ll need to track the unsupported files and include them in the email confirmation. This is the easiest option. If you want to do this, then you’d add a new global named $not_saved_files and in each default case in the switch statement, you’d add the filename to the $not_saved_files global. When you get to the block of code (starting with the comment “// Send response e-mail if needed”) that sends the response email you would just loop through $not_saved_files and include that in the email body.

        Regardless of which way you decide to go, you’ll need to add your code to each default handler in the switch statements.

  11. Roes Wibowo says:

    I can’t access PHP Pear from server because server in Safe Mode. Any solution for this? Thank you in advance.

  12. Alex says:

    Hi Michael,
    Is there a way to only get the “reply” part of the message because Outlook is messing around like usual ?
    For example, with Outlook Office the “reply message” seems to bo in parts[1] instead of parts[0] (gmail, etc…)
    Great script, thank you.

  13. Chad says:

    I love this script but was wondering why it wont work for audio or video files?

    i want to be able to have someone email me with a audio for video file

    • stuporglue says:

      It should work just fine for audio and video files, you would just need to add the appropriate mime type(s) in the decodePart function.

      function decodePart($body_part){
      ...
          $mimeType = "{$body_part->ctype_primary}/{$body_part->ctype_secondary}"; 
          switch($body_part->ctype_primary){
      ... add more mime types in here ...
      
      • Chad says:

        I added this but didnt work, did i do something wrong?

        case ‘video':
        switch($body_part->ctype_secondary){
        case ‘.3g2′: // Save these video types
        case ‘.3gp':
        case ‘.3gpp':
        case ‘.3gp2′:
        case ‘.dv':
        case ‘.mp4′:
        case ‘.H.263′:
        case ‘.mov':
        case ‘.MPEG4′:
        saveFile($filename,$body_part->body,$mimeType);
        break;
        default:
        break;
        }
        break;
        case ‘audio':
        switch($body_part->ctype_secondary){
        case ‘.MP3′: // Save these video types
        case ‘.AAC':
        case ‘.OGG':
        case ‘.WAV':
        case ‘.M4A':
        case ‘.WMA':
        case ‘.amr':

      • stuporglue says:

        Mime types are actually different than file extensions, although there is some overlap for some types.

        3gp for example doesn’t match the extension, it would be audio/3gpp or video/3gpp (for audio or video files respectively).

        You may want to check out Wikipedia for some more info on them : http://en.wikipedia.org/wiki/Internet_media_type#Type_video

      • Chad says:

        wait now i see what it could have been, DUHHHH

        i have “.” in front… let me test with out it

      • Chad says:

        very nice and one other question maybe it is covered here but why does it take the .3gp or .jpg and change it to a text/generic?

        i would like to be able to keep it as the regular file

      • stuporglue says:

        It shouldn’t be changing it to text/generic.

        function saveFile($filename,$contents,$mimeType){

        is the function that saves the file. It takes the $filename, removes any potentially unsafe characters, then saves it. If it’s naming it as whatever.mpg.txt (or something) you’ll want to look at where saveFile() is being called and what arguments are being passed.

      • Chad says:

        and strange this is the 3gp was working because it came to my email as “octet-stream” so now i am going to email myself as many files as i can to see what they say

      • stuporglue says:

        If 3gp is being detected as octet-stream, then something along the way isn’t setting the mime-type correctly for that file type. “octet-stream” basically just means “This is a bunch of binary data that I don’t have a more specific type for”.

        pdf and zip come this way, for example.

        If your 3gp are coming out as text/generic, I’d have to look at a sample email to figure out why.

      • Chad says:

        ok this isnt working here for example if i have this case:

        case ‘image':
        switch($body_part->ctype_secondary){
        case ‘jpeg': // Save these image types
        case ‘png':
        case ‘gif':

        i get the files saved in my directory with these names and says they are text/generic

        1340589988_server_error_png
        1340589359_catskiing-tv-logo_jpg
        video like this:
        case ‘video':
        switch($body_part->ctype_secondary){
        case ‘3g2′: // Save these video types
        case ‘3gp':
        case ‘3gpp':
        case ‘quicktime':

        1340578242_V29-01-12_13-08_3gp
        1340590252_IMG_0218_MOV

        so as you can see, it is taking the “.” and replacing with a “_”

        the save file is like this
        saveFile($filename,$body_part->body,$mimeType);

        I am at my wits end here

      • stuporglue says:

        That looks right.

        $filename = preg_replace('/[^a-zA-Z0-9_-]/','_',$filename);
        

        Anything not in a-z, A-Z, 0-9, _ or – is replaced with an underscore. You can remove the preg_replace line to use the filename as-is, or add ‘.’ to the regular expression to allow periods.

        Removing the preg_replace completely could let people do things like upload a file named “../../../../../../usr/bin/ls” which would end up in /usr/bin (if the permissions on your server were setup incorrectly).

        The file data should be correct. Using the “file” command (on *nix) should yield the correct file type once again. Eg. when I run:

        ~/www/downloads$ file airplane_jpg
        airplane_jpg: JPEG image data, JFIF standard 1.01
        

        On Linux the file extension doesn’t really matter, and doubly so if you’re serving up the files to a web browser doing some sort of PHP passthrough. On Windows where it looks at the file extension you’ll need to rename it before it knows which type the file really is.

      • Chad says:

        that makes sense and you know what it works that way to example i took the .mov file for example

        http://…..com/uploadedFiles/1340590252_IMG_0218_MOV

        and player it directly in VLC with out the “.Mov” extension…. and i was worrying to death for nothing, i never knew that thought it had to have the extension!

        Great hopefully this helps someone else

  14. luiz says:

    a question would even read this script pop3 and saved the attached files

  15. David says:

    Great Little Program Here! Works Great! Took me a bit to get it implemented, but now seems ok. I am running into two small issues, 1, SELinux has to be disable to get it to work. For some reason Postfix gets a permission denied when trying to run the MailReader.php file. Any ideas on that one? 2nd question, I can’t seem to be able to get the program to parse the images out of e-mails sent from Apple’s Desktop Mail Client. It works on the mobile app, but not the desktop version. Let me know if you need an email from me to test it.

    • stuporglue says:

      No idea on the SELinux issue. I always disable SELinux on my personal machines (since they’re just for home/play use) and run mailReader.php on my bluehost server.

      There must be something slightly different about the Mail.app client. Unfortunately I’m swamped with work for at least the next 2 weeks. You can send me a sample email if you want, but I won’t be able to look into it until early/mid August. If you do find the fix before that, please let me know and I’d be happy to update the script.

  16. Franky B says:

    Thank you so much for this script. Works great.
    I have been desperately looking for a solution to log all email (in / out) to mysql. Removing and saving attachments to disk is pure genius. Few questions, enhancement/guidance requests.

    Background, right now I have a postfix/dovecot mail server. I have also configured an email piping script that removes all attachments, saves them to disk, and then modifies the body with download urls for the messages. It works great, until you get into multiple replies (threads), then the messages show up at the bottom. I also have altermime (amavis-new) configured to append disclaimers (which we use for signatures) This also has the same issue where if there are multiple replies, then multiple signatures are appended on the bottom of the message.

    I dont code in PHP, so any guidance would be greatly appreciated.
    Here is what I am trying to do:

    1. Pipe all messages (incomming / outgoing to this script)

    2. Have the script remove all attachments and save them to disk (if it can use S3CMD to save to Amazon S3 that would be even nicer).

    3. I want to get rid of amavis inserting the disclaimers (sigantures) and instead have php lookup the signature for the sender (if in my domains) and then attach the proper signature (html or txt) depending on the message. The reason I like the idea of email server handling signatures is that you dont have to configure multiple mail clients (Ipad, Outlook, webmail) with signatures, and you can keep them consistent throughout. There should be some checking to make sure that signatures are only included on fresh messages. Dont want disclaimers (signatures) piling up on the bottom of the screen.

    4. I would like to get rid of my perl script and have this script continue to pipe the message back to sendmail for delivery. Essentially what it needs to do is list the attachments and say you have X number of days to download them. The reason this is of value is that on a mobile device you dont get attachments wasting valuable space and bandwidth.

    5. If the script is able to send messages (meaning its not a dead end but actually will send the emails out), then i would love to be able to also include a dynamic url for an image within the signature. a 1×1 tracking pixel with a querystring of the message id as inserted into the database. This would be valuable is tracking opens / reads of messages sent. There are services like readnotify that do this, but hiding it in the disclaimer/signature is more likely that the client will allow images to load and the message to be tracked.

    This is probably a lot to ask, and I have no idea if the features described above would be of value to your use of the script. I really do appreciate you sharing your code, and any pointers or guidance you can give me on the above features list would be greatly appreciated.

    PS. the script I am using now for removing attachments is here:
    http://linuxcoffee.com/detach

    PSS. the way I have alter-mime configured for disclaimers is here:
    http://serverfault.com/questions/401900/amavis-atermime-dynamic-email-signatures-disclaimers

    THANKS IN ADVANCE
    Franklin

  17. mike says:

    if you need to save a file in the script that you run.. you MUST make the file paths absolute.. not local.. aka GOOD = /var/www/blah/upload/ BAD= upload/
    because when the file is ran from the email its user is root, but its running path is garbage and it will create files with shit permissions so you have to change them using
    chmod($save_directory.$name, 0777);

    • stuporglue says:

      It does depend on the specific server setup since that will determine which user is running the script, what environment variables are set, what ~ means and what the working directory is.

      The full path should always work — unless the email server is run in a chrooted environment (or your web/ssh environment is) in which case absolute paths won’t match from one environment to the other.

      If relative or absolute paths don’t seem to be working you can try something like this:

      $path = dirname(__FILE__) . 'save_directory';

      That will set $path relative to the current directory no matter what the environment is.

  18. jeff says:

    The script posted above seems to work well, but I get a second email error saying the email was undeliverable, but only if there is an attachment. This is part of the email. Is this related to this script? (I don’t experience this error for any other script). Thanks!

    —— pipe to |/home/mrvchem/public_html/mailprocessor/mailReader2.php
    generated by emailer@XXXXXXX.XX ——

    Failed loading /usr/local/IonCube/ioncube_loader_lin_4.4.so: /usr/local/IonCube/ioncube_loader_lin_4.4.so: undefined symbol: empty_string
    PHP Warning: Zend Optimizer for PHP 5.2.x cannot be found (expected at ‘/usr/local/Zend/lib/Optimizer-3.0.2/php-5.2.x/ZendOptimizer.so’) – try reinstalling the Zend Optimizer in Unknown on line 0

    • stuporglue says:

      The mail server will send back an error email if any text is output at all, even just some spaces or newlines.

      Your two best options are probably to either:

      1) Fix the cause of the error.
      PHP Warning: Zend Optimizer for PHP 5.2.x cannot be found (expected at ‘/usr/local/Zend/lib/Optimizer-3.0.2/php-5.2.x/ZendOptimizer.so’)

      I don’t know why this would happen, maybe you need to update a php.ini file to point at an updated location for ZendOptimizer.so?

      2) Turn down the error reporting level so that this warning doesn’t cause a printed message.

  19. martin says:

    the script is realy very nice! Congratulations.

    it works perfect if i send text in subject, text in the body an also an attachment.
    but if i send only text in the subject and text in the body without an attachment, i get the body always emtpy.

    what could be the reason please?

    regards from madrid, spain
    martin

  20. martin says:

    just want to add some more info:
    if the content type is text/plain and i dont’t have an attachment, $body is alwayts empty.
    if the content type is Content-Type: multipart/alternative; (email in html format) i get the body in $body.
    with attachments, i always get the body.

    regads, martin

  21. martin says:

    and another one just to complete my info:
    i get the error
    Notice: Undefined property: stdClass::$parts in /var/www/vhosts/…/webroot/pipe.php on line 267

    regards, martin

  22. martin says:

    i got everything to work after reinstalling.
    i use it now in a cakephp shell.
    i also added utf-8 encoding.
    thanks for this nice code.

    regards, martin

  23. Hey Stuporglue,

    Awesome script! I can see this being used in so many ways!

    Just for others that may stumble upon this in the future, I added support for CSV files like such:

    switch($body_part->ctype_primary){
    case 'text':
    switch($body_part->ctype_secondary){
    case 'plain':
    $body = $body_part->body; // If there are multiple text/plain parts, we will only get the last one.
    break;
    // ADDED
    case 'csv':
    saveFile($filename,$body_part->body,$mimeType);
    break;
    }
    break;

    My files seem to get named with large numbers in front of them (i.e. 1348899227_blog-post.pdf). I only tested sending from a gmail account. The main reference I see to the filename is here:

    if(array_key_exists('name',$body_part->ctype_parameters)){ // everyone else I've tried
    $filename = $body_part->ctype_parameters['name'];
    }else if($body_part->ctype_parameters && array_key_exists('filename',$body_part->ctype_parameters)){ // hotmail
    $filename = $body_part->ctype_parameters['filename'];
    }else{
    $filename = "file";
    }

    How can I get rid of these numbers and save the file name exactly as is? I understand this would cause issues with overwriting items that have the same name, but for my application, this is not an issue. Thanks and keep up the great work!

    • stuporglue says:

      Thanks for stopping by!

      while(!$unlocked_and_unique){
      // Find unique
      $name = time() . "_" . $filename;
      while(file_exists($save_directory . $name)) {

      It’s the call to time() and the following underscore that’s adding to the filename. You’ll need to comment out or modify the while(file_exists loop so that it permits overwriting.

      • Thanks for your response stuporglue! I wish you had a “notify when comment has been made” button so that I could’ve seen this sooner! No worries though, I figured it out after some tinkering and got it working exactly the way I want. This is an awesome script that serves as the core of one of the scripts I built.

        For those of you who don’t want to deal with databases but still wish to store some of the data, I created a function that can save it to a CSV. It only says the details of one of the attached files (the last one) because that’s what I needed for my particular application. Hopefully this saves someone time in the future:

        // ADDED BY AJAY
        // Put the results in the csv file if needed (only saves last attached file!)
        if($save_msg_to_csv){

        // CSV doesn't aren't exist on the server? Let's create it
        if(!file_exists($csv_filesrc)){
        // Create a new CSV with the right heading
        $header = array('date','from','subject','body', 'filename', 'size', 'mime');
        $new_csv = fopen($csv_filesrc, 'w');
        fputcsv($new_csv, $header);
        fclose($new_csv);
        }

        // Tack on the data to the existing CSV file
        $date = date('m/d/Y h:i:s a', time());
        // Go through the attached files and keep the information for only the last one
        $filesar = Array();
        foreach($saved_files as $f => $data){
        $filesar['f'] = $f;
        $filesar['size'] = $data['size'];
        $filesar['mime'] = $data['mime'];
        }
        $new_row = array($date, $from_email, $subject, $body, $filesar['f'], $filesar['size'], $filesar['mime']);
        $csv = fopen($csv_filesrc, 'a');
        fputcsv($csv, $new_row);
        fclose($csv);

        }

        Keep up the good work stuporglue!

  24. Johannes says:

    Thanks for this great script.
    I do run into problems though. I got it working – a little. When I send my emails this it what happens:

    a) email with text only:
    The sender email and subject line are saved into the db. The body remains empty. The year, month, day, hour, second pretty much puts all emails at Christ’s birth: 0000-00-0….. Reply email comes through saying “Thanks and all”

    b) email with attachment:
    nothing happens – email is “lost”

    Hmm.
    Any ideas? Thanks for your time!
    Johannes

  25. Fabiano Coelho says:

    This is great. Just one question.

    What if I just need to receive the email form a pipe configuration and need to resend the msg as it comes with images ?
    I already did that with html and it works fine, but when I drag a photo to the body of the email I only get the code (I guess it is not decoded and sent back).
    I need to create a mailing list, so people will send emails to a email address and the script will send it to the subscribers.

  26. Jonathan says:

    Why not set the first txt file as the body and capture the rest as attachments:


    case 'text':
    switch($body_part->ctype_secondary){
    case 'plain':
    if($body)
    {
    saveFile($filename,$body_part->body,$mimeType);
    }
    else
    {
    $body = $body_part->body; // If there are multiple text/plain parts, we will only get the last one.
    }
    break;
    }

  27. Jonathan says:

    can’t seem to get attachments right.
    – Hotmail doesn’t seem to hold te file name anymore
    – Can’t seem to open te files send by GMail

  28. Jonathan says:

    replaced:


    else if($body_part->ctype_parameters && array_key_exists('filename',$body_part->ctype_parameters)){ // hotmail
    $filename = $body_part->ctype_parameters['filename'];
    }

    width:


    else if($body_part->d_parameters && array_key_exists('filename',$body_part->d_parameters)){ // hotmail
    $filename = $body_part->d_parameters['filename'];
    }

    solution found on:
    http://w3schools.invisionzone.com/index.php?showtopic=42926

  29. Havard says:

    First of all – great article and nice script!

    Everything works just fine, except from that the files that are uploaded get the permissions -rw——- 1 nouser nogroup. I’ve set the foulders permissions to 777 (for testing), but the files wont inherit it… Have you got any ideas?

    • stuporglue says:

      Permissions are going to be set according to the user who is running the script. It looks like it’s running as nouser/nogroup. I’m not sure what the cause of that would be, sorry.

  30. Pingback: Dowson DVR CCTV System not updating with DynDns « Things To Whinge About « Whinging Pom

  31. Richard says:

    Hi Micheal,

    What is case ‘multipart':? I need the case image only, can i remove this?

    Thanks,

    Rex

    • stuporglue says:

      Email is sent as either plain-text or multipart. A multipart email has a plain-text part and a one or more other parts. Usually one part is a html or rtf, and zero or more other parts will be images or file attachments.

      If you’re trying to extract images only, then you need to check for the multipart case, find each part, check the type, and save it if it’s the expected image type.

  32. Jeff says:

    Any way to make this script save the file as the date then the time?
    EX.
    02-12-13–131422.jpg
    02-12-13–131627.jpg

    Or date then a counter or random number.

  33. Ofer says:

    Hi

    Thanks for the script.

    I have problem with saved ZIP file – it seems that the decoding is changing the cotent like change 0A to 0D0A and also 000000 0000 and 00 to 0000 and maybe more – any idea?

    Ofer

  34. Neokio says:

    Great stuff! Worked like a charm within minutes. Pear’s mime decoder does a much better job than most at handling things like ƒüñky characters in the subject with ease. Might be useful to note that mimeDecode.php is often installed within a Mail directory … require('Mail/mimeDecode.php'); might work better for some. Cheers!

  35. Alexa says:

    Hi stuporglue,

    Awesome script, thanks for sharing. But i’ve one problem here. I want to find texts within the body.

    e.g. In the email body, the user send her name and gender:
    [name]Alexa[/name]
    [gender]female[/gender]

    I want to find her name “Alexa” between [name][/name] and her gender “female” between [gender][/gender].

    How to decode the body part as text?

    Good day!

    Alexa

  36. Alexa says:

    Hi stuporglue,

    I want to prevent the user upload the image from email body, only attachment. Can you teach me how to prevent this? Thanks a lot.

    Regards,

    Alexa

    • stuporglue says:

      Sorry for the delay in responding.

      I believe that you could just comment out this block

      case 'multipart':
      if(is_array($body_part->parts)){
      foreach($body_part->parts as $ix => $sub_part){
      decodePart($sub_part);
      }
      }
      break;

      but without testing I couldn’t guarantee that that’d work or that it wouldn’t break some other use case. I think that at that point in the code, the top-level multipart block have already been broken up, so any remaining multipart content types would be embedded data.

      If you figure it out, feel free to post a response here.

  37. Fiona Fenton says:

    Hi stuporglue,

    First, thanks for publishing this script – has help me out enormously.

    I’m having a problem though with larger attachments. In testing I’m submitting an email with 4 jpg attachments totalling 7mb and get the following error message:

    Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 1981738 bytes) in mimeDecode.php on line 665

    As the images only total 7mb, where is all the remaining memory disappearing to?

    • stuporglue says:

      Hi Fiona,

      I’m guessing that the file itself gets read in (7mb), then each time a body part gets parsed it is staying in memory too. Some judicious use of unset() may release those variables and allow the script to run.

      You can use memory_get_usage() (http://php.net/manual/en/function.memory-get-usage.php) throughout the script to find out the memory usage at any given point.

      • Fiona Fenton says:

        I’ve tracked it down to these lines:

        $decoded = $decoder->decode(
        Array(
        ‘decode_headers’ => TRUE,
        ‘include_bodies’ => TRUE,
        ‘decode_bodies’ => TRUE,
        )
        );

        The memory usage trebles when using decode().
        I’ve found plenty of examples of people having similar problems, but unfortunately no solution.
        I’m guessing the decode() function is reading the email into memory several times.

      • stuporglue says:

        That makes sense. You have the orignal $raw message which is the full size, of the email, then you split it into headers and bodies (so 2x) plus some overhead for other libraries that get loaded.

        You should be able to do an unset($raw) after “$decoder = new Mail_mimeDecode($raw);” since $raw isn’t every used again.

        If that doesn’t work on its own, and if you’re on PHP >5.3 you may be able to trigger a garbage colleciton by running gc_collect_cycles: http://fr.php.net/manual/en/function.gc-collect-cycles.php

    • Fiona Fenton says:

      unset($raw) doesn’t free up much memory. It’s the assignment of $decoded that’s eating up memory, so it would require some amendments to mime_parser.php to solve it.
      As a bit of a workaround, I’ve added ini_set(‘memory_limit’, ‘128M’); to the beginning of my file.

      • stuporglue says:

        unset doesn’t immediately free memory, but it will remove the reference to $raw, which will allow PHP to free the memory the next time the garbage collector runs.

        Setting the memory_limit high might be an easier solution if you’re able to do that.

  38. Bob says:

    Hi stuporglue,

    a WONDERFUL script, works like a charm thanks for this.
    I only have 2 issues:
    I can’t send attachments (images, others are disabled) from the standard mail app on my Mac (10.8.3) it doesn’t recognise the attachments, it works from an iphone, gmail and outlook. I tried to send as HTML and as Flat text message.
    My images are sometimes rotated …

    Can you help with it?

    • stuporglue says:

      I won’t be able to give any programming help for the next several weeks as I finish this semester of school.

      For the Mail.app issues if you can send me a sample email I will get to it when I can (about 3-4 weeks out right now). If you’re able to fix it, and are familiar with GitHub, please submit a pull request and I’d be happy to incorporate it.

      For the images — they’re probably actually rotated on your computer. Many modern photo viewing applications automatically rotate images. You could incorporate exifautotran to the mailReader.php script for jpg images if you wanted it to auto-rotate the images after saving)

  39. Suresh says:

    Hi,

    Thanks for sharing this script… I got the following error reply mail when upload files from hotmail.

    Can u help me?

    This message was created automatically by mail delivery software.

    A message that you sent could not be delivered to one or more of its
    recipients. This is a permanent error. The following address(es) failed:

    pipe to |/home/probe7/public_html/preview/receiptbasket/mailreader/mailPipe.php
    generated by suresh.ma672e2c6@probe7.com

    The following text was generated during the delivery attempt:

    —— pipe to |/home/probe7/public_html/preview/receiptbasket/mailreader/mailPipe.php
    generated by suresh.ma672e2c6@probe7.com ——

    Warning: array_key_exists() [function.array-key-exists]: The second argument should be either an array or an object in /home3/probe7/public_html/preview/receiptbasket/mailreader/mailReader.php on line 907

    —— This is a copy of the message, including all the headers. ——

    • stuporglue says:

      Hi Suresh,

      Sorry it took so long to reply. Since mailReader.php isn’t 907 lines long, it looks like you’ve modified mailReader, and I’m not sure what’s going on in your code.

      I would recommend looking on line 907 to see what the 2nd argument of array_key_exists is, and doing a var_dump($second_argument) on the line before(like 906).

      Then trace that backwards and figure out why it’s not an array.


      Michael

  40. Rob says:

    Body is not showing up for some emails.

    The piping script works like a charm for most of the cases, however the body is not reaching the db table for some emails such as live.com, tried to send email form my live.com account from outlook , tried sending email from robinders@technocratshorizons.com and so on, it appears to miss out on body for quite a few email, any help will be appreciated.
    works good for gmail, yahoomail,rediffmail,email.com
    Thanks a TON for sharing this awesome piece of code

  41. manohar says:

    Hey thanks…,

    i have one problem.
    when i sent mail with text it’s inserting into DB and i’ am getting success mail it’s good.
    but when i attach doc (pdf or png…file) that not storing in DB…and even i am not getting failure email(error reporting mail)…
    (i included mimeDecode.php, and i have 2 tables in my database )
    Please help me and sorry for bad English……!

    Regrades,
    Manohar.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *