Note: All of these tools are Linux command line tools. Less convenient than a GUI, but much more appropriate for bulk operations and scripting.

Preparing the Videos

Tovid can take any video format and make a DVD from it by calling ffmpeg and other libraries when it needs to. In practice, I find that tovid and ffmpeg fight like sibling when they get along it’s true love, but when they fight things get broken.

Encoding is the slowest part of making a DVD, so we’ll do it separately. This way we can tweak the DVD quickly without re-encoding each time. FFMpeg can do it with the command:

ffmpeg -i SourceFile.whatever -target ntsc-dvd outputfile.mpg

This will produce mpeg2 files which don’t need to be further encoded in order to be put on a DVD.

Splitting Videos

If the resulting outputfile.mpg is too big for a DVD, you will need to split it. FFMpeg can split your file using a starting second (-ss) and a time (-t).  With this syntax you can break your file into as many pieces as you need to fit them on a DVD.

ffmpeg -i SourceFile.whatever -ss 0 -t 7200 -target ntsc-dvd outputfile-part1.mpg
ffmpeg -i SourceFile.whatever -ss 7201 -target ntsc-dvd outputfile-part2.mpg

Creating the Metadata

Tovid will want a couple of pieces of metadata, depending on what menu options you choose. Main titles, submenu titles and a disc title.

Since I was digitizing family videos, I wanted the disc title to be the years spanned. So I created a CSV file with 4 columns. the year would become the disc title.

filename,year,long description,short description
SourceFile.whatever,1990,The family goes sledding at Ice Hill,Sledding

The Tovid Wrapper Script

I know Tovid is already a wrapper for dvdauthor, ffmpeg and whatever other tools it uses behind the scenes, but what’s another layer, right?

I put all of my videos, the CSV file and this script in a folder:

#!/usr/bin/env php
<?php

/*
 * @brief Make as few DVDs as needed to fit all of the videos listed in a CSV file
 *
 * Copyright Michael Moore <stuporglue@gmail.com>
 * This script assumes that
 *      * All of the videos have already been encoded into mpeg2 format for DVD
 *      * All of the videos paths are relatie to INPUTDIR
 *
 * As many videos are fit into one DVD as possible. Videos are added in the order
 * listed in the CSV file. If a video is too big for a DVD the script will tell
 * you, and then exit. DVDs are named either "year", "year1 - year2" or
 * "year disc n", using the year from the CSV.
 *
 * Requires php-cli installed so that we can fork processes.
 *
 * Our metadata.csv file format:
 * pathtofile,year,looooooooooooong title here,short title
 *
 * We assume the file has a header row.
 */

//
//         Settings
//
define('DVDSIZE',8500000000); // Our estimate of the number of bytes available on a DVD+R DL
define('MENUOVERHEAD',45*1024*1024); // Generous leeway for the menu system
define('INPUTDIR',"/home/myuser/videos/input/"); // starting point for all file paths. Use / for abs. paths
define('TMPDIR','/tmp/tovid/'); // Where do you want the tmp files?
define('OUTDIR',"/home/myuser/videos/DVDs/"); // Destination?
define('MAXCHILDREN',5); // How many tovids to run simultaneously? Probably 1-2 less than the # of cores available?
$fh = fopen(INPUTDIR.'/metadata.csv','r');

//
//         No Lifeguard on duty!
//
@mkdir(TMPDIR);
chdir(TMPDIR); // Tovid likes to dump in the cwd. Chdir so that tovid dumps its temp files somewhere usefulish

define('MAXVOB',1073709056); // The max size of a single VOB
global $pids;
$pids = Array();

$currentdvdsize = MENUOVERHEAD;
$currentdvdfiles = Array();
fgetcsv($fh); // remove header from csv file
while($file = fgetcsv($fh)){

    // Fast check!
    if((filesize(INPUTDIR .'/'. $file[0]) + $currentdvdsize) > DVDSIZE){
    makeDVD($currentdvdfiles); // Make a DVD!

    $currentdvdfiles = Array(); // Reset!
    $currentdvdsize = MENUOVERHEAD;
    }

    $currentdvdfiles[] = $file;

    // Calculate how much space this video will take on the disc
    // dvdauthor seems to:
    // 1) Never mix videos in the same VOB
    // 2) Make all VOBs except the last one come out to MAXVOB size (the last one can be whatever size smaller than MAXVOB)
    $currentdvdsize += (MAXVOB * ceil(filesize(INPUTDIR .'/'. $file[0])/MAXVOB));

    if($currentdvdsize > DVDSIZE){
    die("It looks like {$file[0]} is too big to fit on a DVD!\n");
    }
}

// Make any remnants
if(count($currentdvdfiles) > 0){
    makeDVD($currentdvdfiles);
}

function makeDVD($files){
    global $pids;

    // Make title
    $last = count($files) - 1;
    if($files[0][1] != $files[$last][1]){
    $title = "{$files[0][1]} - {$files[$last][1]}";
    }else{
    $title = "{$files[0][1]}";
    }

    $count = 2;
    $origtitle = $title;
    while(file_exists(OUTDIR . "/$title")){
    $title = "$origtitle disc $count";
    $count++;
    }

    // List of files
    $input = Array();
    $shorttitles = Array();
    $fulltitles = Array();
    foreach($files as $file){
    $input[] = INPUTDIR . "/{$file[0]}";
    $fulltitles[] = $file[2];
    $shorttitles[] = $file[3];
    }

    $cmd = "tovid disc
    -files  " . implode(" ",array_map('escapeshellarg',$input)) . "
    -titles " . implode(' ',array_map('escapeshellarg',$shorttitles)) . "
    -menu-title " . escapeshellarg($title) . "
    -menu-fontsize 18
    -title-color '#ff7700'
    -title-stroke black
    -titles-fontsize 18
    -titles-color '#ff7700'
    -showcase-titles-align east
    -rotate 5
    -wave default
    -submenus
    -submenu-titles " . implode(' ',array_map('escapeshellarg',$fulltitles)) . "
    -submenu-title-color '#ff7700'
    -submenu-stroke black
    -loop 0
    -submenu-length 20
    -noask
    -out " . escapeshellarg(OUTDIR . "/$title");

    // Now wait for our turn...
    while(count($pids) >= MAXCHILDREN){
    $status = NULL;
    $exited_pid = pcntl_wait($status);            
    if(pcntl_wexitstatus($status) != 0){
        print "FAILURE IN " . TMPDIR . "/tovid.$exited_pid!!!\n{$pids[$exited_pid]}\n";
    }
    unset($pids[$exited_pid]);
    }

    $cmd = str_replace("\n",' ',$cmd);

    print "LAUNCHING!!!\n$cmd\n";

    $pid = pcntl_fork();
    if($pid == -1){
    die("COULDNT FORK!");
    }else if($pid){
    $pids[$pid] = $cmd;
    sleep(3); // give tovid a chance to claim its temp directories
    }else{
    $cmd .= " > " . TMPDIR . "/tovid." . getmypid() . " 2>&1";
    exec($cmd);
    exit();
    }
}

Using the Script

Save the PHP script above to a file named tovidBatch.php and make it executable.

Edit the defined constants to fit your directories and output media (DVDSIZE). If desired, edit the tovid disc command to build your DVDs the way you want them.

Finally, run (on a command line)

php tovidBatch.php

Your videos are on their way!

It has been a big disapointment and I do not plan on buying an HP laptop again and can’t recommend that anyone else buy one either.

I chose the dv7 because I needed something with some processing power. My dv7 has anIntel i7 clocked at 1.87Ghz, with 6 gigs of RAM.

Update (January, 23, 2011)

The screw that was loose on the case eventually came out. Within a day the hinge near where the screw fell out of broke, and I could hear a piece of metal shaking around inside the case. The warranty expires at the end of March.

The HP support person I spoke with was professional in the way that a police officer who pulls you over for speeding is professional. He kept trying to get me to say that I had dropped, bumped or otherwise abused the laptop and warned me multiple times that if I sent it in and they found signs of abuse that I would have to pay for repairs.

Still, 50 minutes later and a box was on its way to my house. The box took 2 days to get here, 2 days to get to HP, and they must’ve fixed it that day because I got it back 3 days after that.

I had only complained about the hinge. The loose power plug and missing keyboard/worn off keyboard I had chalked up to wear-and-tear and didn’t expect them to fix under warranty, but they did.

So, do I love HP laptops now? No, but I’m happy with how their warranty service turned out, and hopeful that I won’t have these problems once the warranty expires!

The HP Pavilion dv7 Overheats

Unfortunately, the dv7 is inadequately cooled. When I start the computer temperature of the computer quicky reaches 75-78 degress (Celcius) and the fans are running loudly. As soon as I start a couple of desktop applications (Firefox, Pidgin, Thunderbird and a terminal) the temperature reaches about 85.

When I got this machine, I was excited because from time to time I rip VHS tapes (old homevideos) and convert them to DVDs for family members. The encoding process is long and slow, and I expected to benefit from all 4 cores.

Sadly, I can’t even run a video encoder at full speed on a single core without the computer overheating, and shutting down.  The computer shuts down if it reaches 100 degrees. I have to run ffmpeg (or mencoder) at 55% CPU usage (of one core) in oder for the fans to keep up with the CPU.

I’m effectivly stuck with a single core 1.03 GHz (55% of a single 1.87GHz core) computer since I can’t use more than that at once. Extremely disapointing.

TIP: If you're in a similar situation, and you're on Linux, you
can use the program cpulimit throttle a specific process or
program. When I try to encode video on Windows (in iTunes) the
computer overheats no matter what I do.

NOTE #1:  This happens in both Windows AND Linux, so it’s not some weird Linux bug. I nearly always use my laptop on a hard surface — and I have to have it on something hard with great airflow if I want to even think about encoding a video.

NOTE #2: My fans and vents are clean. I can see through them to the fans, and have used compressed air on them anyways just in case.

The HP Pavilion dv7 Has a Low Quality Keyboard

I admit that I use  my computer more than most people do. Between work and hobbies, it’s not unusual for me to be on my computer 10-12 hours a day, and often more.

This laptop however, is the first one that I have ever worn the letters off the keys. It is also the first one that I have ever had a key spring break during normal use.

Luckly the dv7 comes with a numeric keypad which I don’t use, so I took the keyspring from one of those keys and repaired my broken key.

Again, very disapointing

The HP Pavilion dv7 Case is Poorly Built

Within a few months of getting my dv7 I noticed that the screen wouldn’t stay in position. It would always settle back an extra inch from where I had pushed it. It turned out that the top plastic piece of the case was coming loose from the bottom piece, which in turn left the lid hinge a little loose.

I tightened the screw on the case, and have had to tighten it again every couple of months.

I’m Done Whining Now

I don’t want/need a replacement from HP badly enough to talk to someone in a call center. I’m just going to keep using this computer until it dies. I’m betting it doesn’t make it a year. My guess is that the threads for the case screw eventually wear out and the screen won’t stay up.

I have a rock-solid reliable P4 desktop computer (an HP, no less!) I can use if this thing goes, and next time I’ll buy something more reliable, like a Thinkpad or a Mac Book.

So, if you’re in the market for a laptop, use extreme caution when buying an HP Pavilion. You might not get as good of a machine as you expect.

It also gave me a single place to put a try/catch block with a debug_print_backtrace to figure out where the offending query was coming from. In my admitedly brief experience with MongoDB so far, Mongo doesn’t lend itself to such practices quite as easily.

With this project I’ve been working on at work we’ve got 34,000 lines of code. We have a global $mongo object that we run queries against. When a Mongo query chokes up our site I need to know what exact call has been made, with what arguments and coming from where.

So I wrote this shim, MongoLogger that would pretends to be Mongo. During development I replace the $mongo object with my MongoLogger object, like so:

$mongodb = new Mongo("mongodb://un:pw@host/db"); // Connect to Mongo
$mongo = $mongodb->db;                   // pick your database
$mongo = new MongoLogger($mongo); // Replace the $mongo

MongoLogger then uses the PHP magic methods __call and __get and the almost magic method call_user_func_array to do the needed logging, and to pass the arguments on to the real $mongo object.

Note: Magic Methods, an call_user_func_array in particular, are slow. Not recommended for production use!

This is simpler than what I wrote for work, but the concept is the same and the important bits are here. It’s what I wish I could’ve found when I first started worrying about how to debug MongoDB calls.

<?php

class MongoLogger{
    /**
     * @class MongoLogger
     *
     * @brief A debugging shim between your code and MongoDB. Should be disbaled for production
     *
     * Usage:
     * $mongodb = new Mongo("mongodb://un:pw@host/db"); // Connect to Mongo
     * $mongo = $mongodb->db;                           // pick your database
     * $mongo = new MongoLogger($mongo);                // and all calls made to the original $mongo just keep working...
     *
     * YAY MAGIC METHODS! http://php.net/manual/en/language.oop5.magic.php
     * Makes use of magic methods __construct, __call and __get and the almost magic method call_user_func_array.
     */

    var $collections = Array();

    function __construct($mongo){
    global $real_mongo;
    $real_mongo = $mongo;
    }

    public function __call($name,$args){
    global $real_mongo;

    error_log("Mongo->$name called with " . print_r($args,TRUE));

    try {
        $res = call_user_func_array(Array($real_mongo,$name),$args);
        return $res;
    } catch (Exception $e){
        debug_print_backtrace();
        error_log(print_r($e,TRUE));
        throw $e;
    }
    }

    /**
     * @brief Return a MongoLoggerCollection for the requested collection
     * This is the main function that gets called in this class. We just pretend that
     * we've got whatever we're asked for (just like Mongo does)
     *
     */
    public function __get($name){
    if(!array_key_exists($name,$this->collections)){ $this->collections[$name] = new MongoLoggerCollection($name); }
    return $this->collections[$name];
    }
}

class MongoLoggerCollection {
    /**
     * @class MongoLoggerCollection
     *
     * @brief Represents a collection in Mongo. Most of your calls will come through here.
     */
    var $collection;

    function __construct($name){
    $this->collection = $name;
    }

    public function __call($name,$args){
    global $real_mongo;

    // Logging!
    $debug = debug_backtrace();
    $caller = $debug[1];
    error_log($caller['file'] . ':' . $caller['line']);
    error_log("\$mongo->{$this->collection}->$name(");
    foreach($args as $arg){ error_log(print_r($arg,TRUE)); }
    error_log(")");

    try {
        $res = call_user_func_array(Array($real_mongo->{$this->collection},$name),$args);
        return $res;
    } catch (Exception $e){
        debug_print_backtrace();
        error_log(print_r($e,TRUE));
        throw $e;
    }
    }
}

One challenge that we faced was storing our data. Should we store our data and convert to Protocol Buffers every time we sent it, or should we just work in Protocol Buffers and store it to the database directly?

We decided to store it in the database in a format compatible with the Protocol Buffer classes so we could easily access it as a Protocol Buffer object again later.

The following classes and scripts were written to help make that bridge between the Protocol Buffer classes generated by protoc-gen-php and MySQL.

They perform two main functions:

1. Generate MySQL table create statements to build tables to hold the protocol buffer data
2. Make classes which extend the protoc-gen-php classes with extra functions for database storage and retrieval (and a few bonuses)

 Generating The MySQL

We’ll start by generating some tables for our database. You’ll need php-cli installed, and you’ll need protoParser.php and protoMySQL.php in your PHP include path (or current directory) and makeMysql.php in your $PATH (or current directory).

Edit protoMySQL.php’s preferences (lines 17-36) to suit your configuration and needs.

Now something as simple as:

php ./makeMysql.php *.proto

should generate the MySQL table create statements you will need.

Generate The DB Classes

With protoc-gen-php, each .proto object gets a corresponding class. eg. list.proto.php is created from list.proto

makeClasses.php creates listDB.php which extends the classes generated by protoc-gen-php. Each proto object gets its own protoDB class and file.

php ./makeClasses.php *.proto

Those classes should then be used instead of their original non-database supporting proto classes.

DB Class Functions

__construct($id_or_object = NULL, $limit = PHP_INT_MAX)

$id_or_object
If it’s an object, we assume it’s a non-database variant, and use its members to populate this object.
If it’s numeric, we fetch the object from the database
Otherwise, we pass it up to the parent object.

$limit
Used in the parent constructor

get($id = NULL, $args = NULL)
$id
The database ID of the object to fetch.

$args
Enough MySQL arguments to uniquely identify the proto to retrieve.
If an array, each field is added to the query.
If a string, the string is appended to the query as is, after the WHERE clause.

Returns object if found, NULL if not found (or if multiples found)

unique()
Calls array_unique on any repeating elements.
Note: Arrays of objects are compared using their __toString methods

Returns nothing

load($object)

If you have an equivalent object (eg. a non-database or database version of the object) you can load it into the current object with this method.

Returns nothing

nullOrVal($val)
Determine if we should append NULL or an escaped string. Used in MySQL queries to ensure safe values.

Returns “NULL” or a mysql_real_escape’d string

delete()
Shallow delete from database. Since sub-objects could be shared/referenced by other proto objects this only deletes this object’s entry in the database

Returns nothing

put()
INSERT or UPDATE this object in the database.

Returns the insert ID of the object

toJSON($asArray = FALSE)
Returns a JSON representation of the current object, or an array appropriate for use in json_encode.

fromJSON($json)
Load the variables in the object from a JSON string

purge()
Like delete, but does delete referenced proto objects from database.
Returns number of sub-objects deleted.

License, Warranty and Support

My employers have been kind enough to let me release these scripts under and Open Source license. They are released under the GPL v.2 without any warranty.

We are actually switching away from MySQL on this particular project, and so these scripts are unlikely to receive any further updates.

I will provide such support as I have time for through the comments on this blog post.

Happy programming!

What about pictures that don’t have exif info? Maybe you’ve got an old/cheap camera, scanned images, or another source that dosn’t include the orientation information. I’m in the middle of scanning about 5000 pictures, and I didn’t want to manually rotate all the images…so I started looking for solutions.

OpenCV: Open Source Computer Vision

I figured if I could detect a face in a photo, then it was right side up. I started searching for solutions and the Internet seemed to agree that the best free solution for facial detection was OpenCV, but I couldn’t find a script that did just what I wanted.

OpenCV “is a library of programming functions for real time computer vision.” It can be used in robotics, for doing cool stuff with web cams, motion detection, and more.  I was mainly interested in because of its facial recognition features.

Then I found a this script by Jo Vermeulen from way back in 2008. It’s made for use with webcams, and the particular script is just a toy. He does have another script which interacts with DBus to provide present/away status changes for your IM client. With just a little bit of work I was able to repurpose his script to do what I needed.

Whatsup: An OpenCV Python Script To Detect Correct Photo Orientation

Prerequisites:

•  Install libcv2.1
• Install python-opencv
• Download one or more Haar Cascades and save them to /usr/local/share/ (more on this below)

The script I made is called whatsup.

Usage: whatsup [--debug] path_to_file

whatsup  returns the number of degrees it thinks your photo needs to be rotated to be right side up.  You can then use that number of degrees in whatever program you want, be it imagemagick, jpegtran, gd, or whatever. whatsup assumes that your image is squared up (not square) and only returns multiples of 90 (0,90,180,270).

Using the –debug option will show the image with a box around the feature that gave the whatsup the final result.

#!/usr/bin/env python

# This script reads in a file and tries to determine which orientation is correct
# by looking for faces in the photos
# It starts with the existing orientation, then rotates it 90 degrees at a time until
# it has either tried all 4 directions or until it finds a face

# INSTALL: Put the xml files in /usr/local/share, or change the script. Put whatsup somewhere in your path

# Usage: whatsup [--debug] filename
# Returns the number of degrees it should be rotated clockwise to orient the faces correctly

# Some code came from here: http://blog.jozilla.net/2008/06/27/fun-with-python-opencv-and-face-detection/
# The rest was cobbled together by me from the documentation here [1] and from snippets and samples found via Google
# [1] http://opencv.willowgarage.com/documentation/python/core_operations_on_arrays.html#createmat

import sys
import os
import cv

def detectFaces(small_img,loadedCascade):
    tries = 0 # 4 shots at getting faces. 

    while tries < 4:
	faces = cv.HaarDetectObjects(small_img, loadedCascade, cv.CreateMemStorage(0), scale_factor =1.2, min_neighbors =2, flags =cv.CV_HAAR_DO_CANNY_PRUNING)
	if(len(faces) > 0):
	    if(sys.argv[1] == '--debug'):
		for i in faces:
		    cv.Rectangle(small_img, (i[0][0],i[0][1]),(i[0][0] + i[0][2],i[0][1] + i[0][3]), cv.RGB(255,255,255), 3, 8, 0)
		cv.NamedWindow("Faces")
		cv.ShowImage("Faces",small_img)
		cv.WaitKey(1000)
	    return tries * 90

	# The rotation routine:
	tmp_mat = cv.GetMat(small_img)
	tmp_dst_mat = cv.CreateMat(tmp_mat.cols,tmp_mat.rows,cv.CV_8UC1) # Create a Mat that is rotated 90 degrees in size (3x4 becomes 4x3)
	dst_mat = cv.CreateMat(tmp_mat.cols,tmp_mat.rows,cv.CV_8UC1) # Create a Mat that is rotated 90 degrees in size (3x4 becomes 4x3)

	# To rotate 90 clockwise, we transpose, then flip on Y axis
	cv.Transpose(small_img,tmp_dst_mat) # Transpose it
	cv.Flip(tmp_dst_mat,dst_mat,flipMode=1) # flip it

	# put it back in small_img so we can try to detect faces again
	small_img = cv.GetImage(dst_mat)
	tries = tries + 1
    return False 

# Detect which side of the photo is brightest. Hopefully it will be the sky.
def detectBrightest(image):
    image_scale = 4 # This scale factor doesn't matter much. It just gives us less pixels to iterate over later
    newsize = (cv.Round(image.width/image_scale), cv.Round(image.height/image_scale)) # find new size
    small_img = cv.CreateImage(newsize, 8, 1)
    cv.Resize( image, small_img, cv.CV_INTER_LINEAR )

    # Take the top 1/3, right 1/3, etc. to compare for brightness
    width = small_img.width
    height = small_img.height
    top = small_img[0:height/3,0:width]
    right = small_img[0:height,(width/3*2):width]
    left = small_img[0:height,0:width/3]
    bottom = small_img[(height/3*2):height,0:height]

    sides = {'top':top,'left':left,'bottom':bottom,'right':right}

    # Find the brightest side
    greatest = 0
    winning = 'top'
    for name in sides:
	sidelum = 0
	side = sides[name]
	for x in range(side.rows - 1):
	    for y in range(side.cols - 1):
		sidelum = sidelum + side[x,y]
	sidelum = sidelum/(side.rows*side.cols)
	if sidelum > greatest:
	    winning = name

    if(sys.argv[1] == '--debug'):
	if winning == 'top':
	    first = (0,0)
	    second = (width,height/3)
	elif winning == 'left':
	    first = (0,0)
	    second = (width/3,height)
	elif winning == 'bottom':
	    first = (0,(height/3*2))
	    second = (width,height)
	elif winning == 'right':
	    first = ((width/3*2),0)
	    second = (width,height)
	cv.Rectangle(small_img,first,second,cv.RGB(125,125,125),3,8,0)
	cv.NamedWindow("Faces")
	cv.ShowImage("Faces",small_img)
	cv.WaitKey(3000)

    returns = {'top':0,'left':90,'bottom':180,'right':270}

    # return the winner
    if sys.argv[1] == '--debug':
	print "The " + winning + " side was the brightest!"
    return returns[winning]

# Try a couple different detection methods
def trydetect():
    # Load some things that we'll use during each loop so we don't keep re-creating them
    grayscale = cv.LoadImageM(os.path.abspath(sys.argv[-1]),cv.CV_LOAD_IMAGE_GRAYSCALE) # the image itself

    # Get more at: https://code.ros.org/svn/opencv/tags/latest_tested_snapshot/opencv/data/haarcascades/
    cascades = ( # Listed in order most likely to appear in a photo
	    '/usr/local/share/haarcascade_frontalface_alt.xml',
	    '/usr/local/share/haarcascade_profileface.xml',
	    '/usr/local/share/haarcascade_fullbody.xml',
	    )

    for cascade in cascades:
	loadedCascade = cv.Load(cascade)
	image_scale = 4
	while image_scale > 0: # Try 4 different sizes of our photo
	    newsize = (cv.Round(grayscale.width/image_scale), cv.Round(grayscale.height/image_scale)) # find new size
	    small_img = cv.CreateImage(newsize, 8, 1 )
	    cv.Resize( grayscale, small_img, cv.CV_INTER_LINEAR )
	    returnme = detectFaces(small_img,loadedCascade)
	    if returnme is not False:
		return returnme

	    image_scale = image_scale - 1
    return detectBrightest(grayscale) # no faces found, use the brightest side for orientation instead

# Usage Check
if ((len(sys.argv) != 2 and len(sys.argv) != 3) or (len(sys.argv) == 3 and sys.argv[1] != '--debug')):
    print "USAGE: whatsup [--debug] filename"
    sys.exit(-1)

# Sanity check
if not os.path.isfile(sys.argv[-1]):
    print "File '" + sys.argv[-1] + "' does not exist"
    sys.exit(-1)

# Make it happen
print str(trydetect()),

About OpenCV Feature Detection

In order to detect features, like faces, OpenCV needs to be trained. You then use the training file, called a Haar Cascade, to define the detection. OpenCV provides lots of different ready training files here.

To use whatsup you’ll need to download one or more harrcascade*.xml files and put them in /usr/local/share (or edit whatsup to point at the place you decide to save them).

What you need to be aware of is that detection works best at the resolution the training file was created for. So  if you’re using haarcascade_frontalface_default.xml then you want to be giving the detection 24×24 pixel faces.

In order for this to happen, whatsup tries scaling the images to different sizes and tries detecting faces in those different sizes. I am using scans that are roughly 1200×800 and so I start with a scaling factor of 4 so that the first image tried is 1/4th the size of the original. If your images are larger then you probably need to start with a larger scaling factor.

What If No Faces/Features Are Detected

If no features are detected in any of the image sizes, then whatsup determines which side of the photo is brightest, and returns the number of degrees needed to rotate the brightest side upwards.

The assumption is that if no people are found, then maybe it’s a landscape photo and the sky should go at the top.

There are lots of times where this is incorrect (eg. a lit ski hill at night), but for my photos it will be true more times than not

How Accurate Is It / Improving Accuracy

I am getting better than 80% accuracy, but probably not 90%.

The better you know your photos the smarter you can make the script for your use.  If you choose Haar Cascade files that are more applicable to your photo set you are less likely to get false positive.

If you know what sizes your faces typically are you can choose appropriate scales or order the image scaling to happen in the most likely order. You could even make your own hasscascade.xml files if you have certain features you want to look for.

Whatsup in Daily Use

I have actually incorporated whatsup into a script that gets run every time I scan something on my scanner, but this is the bash script I used for testing and developing it.

Make sure whatsup is in your path, and that you have jpegtran and jpegexiforient installed. Jpegtran does lossless jpeg rotations, jpegexiforient sets that missing Exif flag that lets programs know which way to display a photo.

Save this script in the same directory as your jpegs you want to test this on and run it.

#!/bin/bash
for i in saved/*
do
    echo -n "Processing $i : "
    degrees=`whatsup $i`
    if [ $degrees -gt 0 ]
    then
	echo $degrees
	cp $i /tmp/tmp.jpg
        jpegtran -rotate $degrees /tmp/tmp.jpg > $i
	jpegexiforient -1 $i
	sleep 1
    else
	echo ""
    fi
done

Disclaimers

While whatsup doesn’t modify your photo, any program you would use it with does, including the bash script above. Please make responsible use of backups and testing as I disclaim any liability for any lost data.

I’m not a pro python coder, so the script could probably be optimized somehow.

When he died I inherited some of his tools. Most of them live in my toolboxes in the garage, but these ones seemed old enough that I decided to make a display with them.

The display is hung above my computers in my office. The tools are all hung on pegboard just like my daily-use tools in the garage.

Roughly top to bottom, left to right:

A brace, a tape measure (inside the brace), monkey wrench, an awl, a folding ruler, a putty knife, a level, a ball-peen hammer, a pocket knife, book,  plumb bob, eggbeater drill, wire cutters, pipe wrench, needle nose pliers, screw driver, planer.

The book is The Complete Home Handyman’s Guide. I like it partly because it starts out with the very basics. Figure 1 is a diagram titled “Claw Hammer”.  Trying to imagine grandpa ever needing such basic instruction makes me smile. The book does get into all sorts of home repairs though and is actually still very applicable to the 1950s rambler I live in now.

Inside the cover is a short inscription showing that the book was given to grandpa by his father-in-law Walt Johnson.

I started by considering what tools I have that spin. The main contenders were the router, the drill and the drill press. I decided that I use the drill too often to use it, and I thought that the router might spin too fast. For a test, I put a screw into a piece of wood and put the screw into the router (where the router bit goes) the screw folded over the the wood whipped around sideways. Not good.

So, I went with the drill press. This green machine is a reliable but dated 1982 1/3 HP 6 Amp motor drill press. The lid lifts up so you can manually switch the belt between gears to change the speeds.

Step 1: Do a Little Jig

The first step was to turn the drill press sideways. I had to find a way to turn it sideways while supporting it securely and being able to access the power button. I was able to unscrew the arms of the drillpress height adjuster and then I built this:

This jig holds the drill press drilling axis horizontal, and gives makes the drill press stable while operating in a horizontal position.

Step 2: Create a Dead Center

I used a grinder and a file to make the end of a bolt rounded and smooth, this would be my dead center. A live center would be better, but I didn’t have any appropriate parts available.

Failure Number 1 – Attaching Dead Center to a Wooden Platform

While a live center would be preferable, I decided to start with a dead center for simplicity. Since I don’t have any welding equipment, I used a piece of 3/4 inch plywood and some bolts to secure and provide the dead center.

It started out find, but after about 5 minutes of spinning a piece of wood, the bold had vibrated loose from the wood.

Success! Bolting the Dead Center to The Drill Press Platform

Although I lost about 2.5 inches in doing so, I next tried bolting the dead center directly to the adjustable drill press platform.

Failure Number 2:  The Chuck Falls Off

After using the lathe for a short while, the chuck fell off. I pounded it on nice and hard, and have been using less lateral force. I hope that if I let the tools do more cutting and I do less pushing, it won’t be an ongoing problem.

Step 3: Cutting Some Blanks

I cut two blanks. One from a piece of lilac branch from the yard and one from a 2×4 (pine). I used a table saw and cut each blank to be 1.75×1.75×5 inches.

Step 4: Making Some Tools

I needed some tools to cut the wood so I turned to the two pieces of metal I had on my workbench which I wasn’t going to be using anytime soon.

That’s right, a railroad spike. I have a handful of them, and I’m not sure that they’re good for anything else. As for the chisel, my grandpa had more chisels than he probably ever used, which means I have more than I know I will ever use. I sacrificed one of them to the grinder to make its cutting angle more appropriate for wood turning

Results

The pine (2×4) wood didn’t turn out well at all, but I didn’t really expect it to, I mean it’s a 2×4, not some nice wood. The lilac turned out pretty decent for a first time.  A little bit more drilling and fiddling later and I am the happy owner of a drill press lathe and two through-wire construction fishing plugs that need to be painted.

on

Warnings

If you’re the sort of person who heeds warnings, don’t turn your drill press into a lathe. The comments on this instructible which I found while researching my own lathe project indicate that the chuck coming off is a common problem and could be dangerous.

If you’re the sort of person who doesn’t heed warnings, I’m not sure what you’re doing reading this, but best of luck, and be safe. I assume no responsibility for your success, safety or lack thereof  should you undertake this or a similar project.

PHP Protocol Buffers

UPDATE!

I haven’t tested it yet, but DRSlump commented below to let me know about a new, alive and mostly complete implementation he is using and working on. I haven’t tried it yet since it uses PHP 5.3 and we haven’t quite moved over from 5.2 yet, but here it is for your enjoyment:

DRSlump’s Protobuf-PHP

I could find no complete implementation for PHP at this time. There are several 1/2 completed protocol buffer implementations out there for PHP, most of which date back to near the initial protocol buffer announcement from Google.

• protobuf-php
• pb4php
• protoc-gen-php

If you are going to use Protocol Buffers in a PHP project, you will probably want to evaluate the features and approach of each of these projects. For the project I am working on I decided to go with protoc-gen-php since it seemed to have the most complete (3/4ths complete?!) implementation and the most recent update.

Status of protoc-gen-php

There were two branches of protoc-gen-php when I started using it. Bramp was the original author and responded to emails, Iamamused has a few more features, but didn’t respond to any of my emails. I took the email response to mean, in part, that despite the lack of updates that Bramp was still interested in at least maintaining the project.After forking his project to add some features I needed, he did in fact indicate that he will indeed merge appropriate changes from my fork and from Iamamused back into the main branch.

Until that happens though you’ve got three choices:

• Bramp’s branch — the original and probably best long-term choice
• Iamamused’s branch — more features than Bramp’s branch (sint, for example) but no indication if he will be maintaining it long term
• Stuporglue’s branch (mine) — See below for features and caveats

My fork of Protoc-gen-php

I don’t really have interest in being a project maintainer long term. If you use my branch, I’m happy to help in whatever capacity I’m able until Bramp merges the features I’ve added back into his branch.

Here’s what I’ve added:

• Signed Int support (sint32 / sint64)
• Float and Double support
• Support for the [packed=false] option so that repeating packables can be used (ie. not packed support, just the ability to specify non-packed)

The big caveat with float support is that PHP uses doubles to represent both floats and doubles, so when a float-double gets packed into actual an actual float in the protocol format it does lose precision. The best solution will be to either use a string or a double.

Need Other Features Added?

I have implemented the features I did because I needed them for the project I am working on. I’d be happy to add other features on a consulting basis, or to merge in your changes if you submit patches to me. Like I said though, hopefully it will be merged back into Bramp’s branch pretty soon, and he can handle the patches. It would be one less thing for me to worry about!

UPDATE:

We went into a store and they called T-mobile again for us. It turns out that the magic words are “Close my account”. That gets you to customer loyalty. We are now happy prepaid customers. We did also sign up with a VoIP service and tie that into our Google Voice numbers to get free incoming calls when we’re at home, but that’s food for another post.

What I’m trying to do isn’t that complicated. I currently have a family plan with two phone lines on it. The 750 Talk plan for $59.99 per month.

Our contract expired last month, which meant I was in the market for a new plan. After downloading our recent usage into a spreadsheet and checking out all our options, the best place for us to be right now is on separate monthly pre-paid plans, the 1500 Talk & Text with 30 MB data plan, for $30.00 per phone.

Same price for 4 t imes the minutes, not too bad, right? What the family plan has that the prepaid doesn’t is that the family plan includes free nights and weekends, and free T-Mobile to T-Mobile calling. We’re both using about 700-900 total minutes per month with no texting; essentially what this plan would do is give us texting at the same price we’re paying now.

Problem 1: Service Reps Don’t Want to Talk to You If You Have a Plan That’s Not In Their Department

The first issue I had was that each service rep wanted to send me back to the post-paid department since I have a post-paid plan. I literally had to interrupt them and say “Wait!” to not get sent back there again and again.In the end, no one really knew who I was supposed to talk to.

Post Paid
    ||
    \/
Pre Paid
    ||
    \/
Customer Care ? (Gustavo)
    ||
    \/
Customer Care
    ||
    \/
Flex Pay
    ||
    \/
Pre Paid (but I said no, and hung up)

Above is how I was transfered. I was automatically started in post paid, which makes sense since that’s the plan type I have. They sent me to pre-paid (see problem 2 below, for more on this) who said that Customer Care should be able to help me switch plans. Customer care apparently wasn’t really custoemer care because they transfered me to the real Customer Care. The real customer care actually seemed like they understood my problem, but couldn’t change my plans for me, and sent me to Flex Pay. Flex pay got confused and tried sending me back to pre-paid. At this point I figured enough was enough and said goodbye.

Problem 2: Internal Plan Categories Don’t Match T-Mobile’s Website’s Plan Categories

Check out the Family Plan page. Go ahead, open that link and search for the term “post paid” or “postpaid”. The only place it appears is in the light grey fine print at the bottom of the page, and yet customer service kept referring to the post-paid department when they talked about transferring me. The term postpaid was pretty easy to figure out, after all, I am paying for my account at the end of each month.

Where it starts to get fun is if you visit their Prepaid Plans page.  Looks like some nice pre-paid plans, right? You’ve got your pay by the minute section and your prepaid monthly plan. Pretty straightforward right? Except that once you have gotten transfered to the pre-paid department and start talking about the monthly plans, they get confused. The lady I spoke with kept saying “We don’t have any plans with 1500 minutes!”. Finally she figured out which plan I was talking about and said “OH, that’s not prepaid, that’s a flex plan”.  What do you mean it’s not pre-paid? It says on the website “PREPAID PLANS”. Go back to the Prepaid plans page and search for the word “Flex” — it’s nowhere to be found.

Now, I’m not sure if the pre-paid customer service rep was just confused or what, but I don’t even see the word Flex anywhere in the Plans menu list.I have added it for your convenience.

There is in fact a FlexPay Plans webpage, but I suggest you use Google to find it because it’s sure not in the menu or on the Plans Overview webpage. Actually, I suggest not bothering with finding it because the plans aren’t very good.

In any case, it doesn’t look like the T-mobile website likes Flex Pay very much, and it seems like the pre-paid people need some more training on their pre-paid plans. Or the website needs to be more clear on the definition of pre-paid.

Problem 3: Getting Help From T-Mobile is Like Getting Help From a Child

I’ll give T-Mobile this much, I was speaking to a live person in about 3 minutes, and every one of the 5 people I spoke with was friendly.

Working with them was like getting help from a child though. A happy friendly kid who you have to explain things to over and over again. Every time you speak to someone you are required to enter your phone number, say your name and the last 4 of your Social Security number, and then you have to re-explain your whole issue to the new person. Isn’t this the reason that computers were made? To help us with problems like this? Once I have proven who I am, shouldn’t I be able to get transfered through the whole support system without doing so again?

Conclusion

After 30 minutes I’d had enough. I guess I’ll have to go into a physical store. It just seems that a phone company should be able to help a customer change phone plans over the phone. I mean, isn’t that kind of the business they’re in?

Sorry for the rant, I usually try to keep this website more technical and interesting with less whining, but this is what you get tonight.

Sidenote

Runners up in the search for a new plan were

• H2O Wireless – $40/month for unlimited voice and text — GSM prepaid
• Boost Mobile – $50/month starting decreasing every 6 months to $35/month after 18 months. iDEN prepaid

The price to match was $60/month for two lines for about 900 minutes per line. Boost would’ve been worth the extra money for unlimited data, but we got my wife a nice GSM phone with built-in GPS (no data plan needed!)and I haven’t seen a smart phone that fits my needs yet, so I’d rather stick with my smallish dump-phone. H2O would’ve been an acceptable option, but since we won’t be maxing out the 1500 minutes on the T-Mobile plan, the price was the only difference and T-Mobile won out. Well…it will if they can get our plans switched over.

1. It stops me from checking something in, noticing it doesn’t work and committing a fixed version only moments later.
2. It prevents other users from accidentally checking in broken code

A quick search online will reveal that it’s easy to check the syntax of your PHP files before they are committed. All you need to use is a pre-commit hook. Unfortunately scripts I found would all stop on the first error which was not the behavior I wanted. I wanted to check all of the files I was committing and get a list of all errors immediately. So, I wrote my own pre-commit hook.

Checking All PHP Files With A pre-commit Hook

The script I came up with is in PHP. Save this in the hooks directory of your Subversion repository. Name it pre-commit and make it executable.

#!/usr/bin/php
<?php 

// Set these manually since Subversion doesn't set ENV
$PHP = '/usr/bin/php';
$SVNLOOK = '/usr/bin/svnlook';
$AWK = '/usr/bin/awk';
$GREP = '/bin/egrep';
$SED = '/bin/sed'; 

$REPOS = $argv[1];
$TXN = $argv[2]; 

// Find the changes...
$CHANGED=`$SVNLOOK changed -t "$TXN" "$REPOS" | $GREP "^[U|A]" | $AWK '{print $2}'`;
// ...as an array
$CHANGED = split("\n",trim(rtrim($CHANGED))); 

$errors = Array();

// Perform specific actions based on the file extension
foreach($CHANGED as $FILE){
 switch(pathinfo($FILE,PATHINFO_EXTENSION)){
 case 'php':
 case 'class':
    // Get just the error/no error message from php -l
    $cmd="$SVNLOOK cat -t '$TXN' '$REPOS' '$FILE' | $PHP -l | head -2 | tail -1";
    $msg=trim(rtrim(`$cmd`));
    if(preg_match('/No syntax errors detected/',$msg) != 1){
       $msg = preg_replace('/in - /','',$msg);
       $errors[] = "In $FILE: $msg";
    }
   break;
  case 'js':
    // You could do something else for JavaScript -- like JSLint, if you're brave
    break;
 }
} 

// Print all the errors in a nice list
if(count($errors) > 0){
 $warning ="
*************************************************************************
* Please correct the following errors before commiting these changes! *
*************************************************************************
";
 error_log($warning);
 for($i = 1;$i <= count($errors);$i++){
 error_log("$i. " . $errors[($i - 1)]);
 } 

 exit(-1);
} 

exit(0);

This script will check all of the files that changed based on their file extension. You could check .js files one way and .php another way. You can extend this script by simply adding more cases to the switch statement.

The script collects all errors and then prints a nice list when it’s done. The output looks like this:

svn commit -m "pre-commit hook test"
Sending        test_scripts/info.php
Sending        test_scripts/user_read.php
Transmitting file data ..svn: Commit failed (details follow):
svn: Commit blocked by pre-commit hook (exit code 255) with output:

*************************************************************************
*  Please correct the following errors before commiting these changes!  *
*************************************************************************

1. In /test_scripts/info.php: Parse error: syntax error, unexpected $end on line 4
2. In /test_scripts/user_read.php: Parse error: syntax error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING on line 8

I’m not going to say that it’s the most elegant output in the world but it does the job and I haven’t checked in a bad PHP file since.