I've created a mailing list tool using PHP, Javascript for UI stuff -- love that DOM!, and Perl. In any case, this article explains the techniques I've used, problems I've had, and solutions I've employed.

Okay, so I have been tasked with making, basically, an MLM (Mailing List Manager). Well, the MLM is sorta broken into two parts -- the LIST management part, and the EMAIL creation and sending part. This article only discusses the email sending aspect.

I encountered a number of problems and solutions on this project, and I consider it a good, well-rounded example of small web application development. Small? Well, ultimately, it really is a pretty small project, but I did use a number of skills and techniquest to complete it. (Did I say "complete"? Hah! A Web application is never complete!)

Problem 1: Sending email using PHP

Description of solution needed...

Problem 2: Improving the sending of email to allow it to scale up to dozens of emails at once.

Well, the simple answer is "forking". But that turned out to be not really possible using PHP alone. PHP has some process forking tools, but they are only usable when using PHP in CLI mode. See the manual for (sketchy) details (http://us3.php.net/manual/en/ref.pcntl.php).

Okay, so PHP won't fork when running as a web server module. Or, at least, it won't work as advertised (it crashed the server when I tried it!). So I needed to create a system tool that I could then use with "exec()" or "system()" -- PHP system calls. Using those, I could put an ampersand at the end of the command and the Unix shell will do the forking. So I wrote a simple email sender in Perl, and used that. It really is quite basic! Here it is:

#!/usr/bin/perl
$TO_LIST_FILE = shift @ARGV;
$EMAIL_CONTENTS = shift @ARGV;

@to_list = get_to_list( $TO_LIST_FILE );
$message = get_message( $EMAIL_CONTENTS );

foreach $to ( @to_list ){
        $message_out = $message;
        $message_out =~ s/\@\@TO\@\@/$to/g;
        sendEmail( $message_out );
}

unlink( $TO_LIST_FILE );
unlink( $EMAIL_CONTENTS );

sub get_message{
        $out = '';
        $filename = shift @_;
        open(INFILE,  $filename)   or die "Can¿t open $filename: $!";
        while (<INFILE>){
                $out .= $_;
        }
        return $out;
}
sub get_to_list{
        $filename = shift @_;
        open(INFILE,  $filename)   or die "Can¿t open $filename: $!";
        while (<INFILE>){
                next if /^\s*$/;
                chomp;
                push @out, $_;
        }
        return @out;
}

sub sendEmail {
        # Optional: put a sleep statement here to help reduce server load.
        # sleep 1;
        my $message = shift @_;
        my $sendmail = '/usr/sbin/sendmail';
        open(MAIL, "|$sendmail -oi -t");
        print MAIL "$message\n";
        close(MAIL);
}

A couple of minor points:

• The optional "sleep" line might help reduce server load. Make it sleep for a 60s between emails, if you like.
• To provide a bit of security, the email message contents files are deleted immediately after being used.
• Also, for security reasons, I have chmodded the script's permissions to be RWX only by the Web Server user ("nobody").

The Perl script takes 2 arguments: the full system path of a file containing a list of email addresses, and the full path to a file containing the preformatted email message. The message body needs to use a "mail merge" field in place of the "To:" in order for the Perl script to substitute each email address in the list. I also redirect STDER to /dev/null

Important Considerations

It is important to realize that the Perl script is so closely dependent on the PHP script that they are, in many senses, components of the same script. I just wrote one small part in Perl, is all.

Also, this combination of scripts simply will not work on a Windows platform.

Also, also, one needs to have sendmail installed (and properly configured) for the actual mailing part.

Problem 3: Adding arbitrary lists of email addresses

I have an existing database of names and email addresses (and other things). I used that. I think the details are a little outside the scope of this article, though, except a mention that the email addresses are all used by putting them into an array.

Problem 4: Adding attachments and creating HTML-formatted email

Wow, this was a doozy. But fun. There are a lot of new things about this part that I had to learn. Also, I describe how to create HTML-formatted emails in this section, as the techniques to do that are the same as those used to create email attachments. Here are some of the details.

Things I had to Learn About

• Using the proper email headers to define different parts.
• Using MIME headers to define related, alternative, and mixed content parts.
• Using PHP functions to base64 encode binary content.
• How to embed related images in-line with the HTML email (I didn't use this technique, but I learned how to do it!)
• How to use JavaScript to create (and delete) content. For the user interface, of course.
• Other stuff...

Of course, that's a lot of stuff. Maybe I should break this out into a separate article altogether!

Simple summary: get uploaded files, base-64 encode those files, create a properly-formatted email message including MIME attachment boundaries. Send this to mailer.

...details coming soon...

Conclusion

I still have a number of features to add. I also need to describe the list management part. Oh, if only I had time! Woe!

Scribus is excellent, in that it really does produce high quality, print ready documents that can be sent to a print shop for reproduction. However, there is a lot of bad hiden under that good exterior.

Updated April 19, 2006. I have a bunch of new complaints.

I just gotta say, I am somewhat dissappointed in Scribus (which I will refer to as "SC" from here on out), mostly because of all the hype I have read. On the other hand, I am pretty pleased that (1) Scibus exists, and (2) Scribus actually does work!

I also gotta say, I'm going to make a number of comparisons here to the GIMP and Inkscape, being the other FOSS apps that complete the necessary tool set for FOSS desktop publishing. Also, there is some ovelap in the Inkscape and Scribus developer community.

So what are the good things?

First of all, like I said, Scribus really can do what it says it can do: produce quality, publishable documents.

Scribus has a lot of tools available, and a lot of features. SC can import SVG, EPS vector formats, and a wide range of bitmap image formats. It can flow text in highly controllable ways. It can maintain margins and formats and page layouts in a manner that one would expect from a destop publishing tool.

However...

Well, on the whole, Scribus feels and acts like unfinished Beta software. Now, I know that this is FOSS stuff and the unfinished feeling is probably because SC is, well, unfinished. So here is a list, in no particular order, of specific bugs, instabilities, and goofy user interface problems. I really should post these to SC bugzilla, if they have one.

• "Undo" does not work well at all. In fact, "undo" hardly works at all -- there is effectively no undo functionality in Scribus!! In this day and age, I expect "undo" to work exactly as expected, which is to undo the last change I made, whatever that was. A change in paragraph styles, a change in the position of an item, a change in an objects margins or shape or line weight, a change in font on one word in one section, an object deleted, an object newly created, anything. Scribus seems to have an undo feature that only works for one kind of change, undoing a repositioning of an object. And it only "undo"s one level deep! I think this is my number one complaint about Scribus and the thing that makes SC most difficult to work with.
• It is frustratingly difficult to edit the bounding box, contour line, and general shape of a "shape" or image.
• Screen redraws are very quirky. Often, a selected object (highlighted in a red dotted line with control handles at the corners) will appear to become un-selected even though it is not. A creen redraw, either by minimizing/maxmizing or switching desktops or similar will remove this artifact, and the item appears "selected" again. But it is frustrating, nonetheless.
• Several bugs appeared in the handling of images. Resizing, and checking the different scaling otions ("Free Scaling" or "Scale to Frame Size") back and forth can make an image "goofy" -- have the wrong aspect ratio or clip out of the bounding box in a way that is not intended.
• Menus are poorly organized and hard to navigate. Things are not where I expect them to be. This problem is a pretty common one among software in general; IMO, menus are one area where following the leader should be a requirement. ...need a specific example here...
• Layers are poorly managed. One can never tell when an item one wants to manipulate is one this layer or that layer. A tool option like Gimp or Photoshop's "automatically select layer" would be great. There are preferences for almost all the tools but the select arrow. Why not add that as a preference?
• Frequent crashes and general instability!
• I have more...

Keep in mind that on the whole, I am very happy the Scribus exists, and that its end results are really quite nice. I was able to produce a prefessional grade PDF brochure for my wife's business. So, these notes are more like well-intentioned bug reports rather than outright complaints.

April 19 Update

So, I have been using Scribus a bit more. I need, it appears, to produce multi-page PDF documents more often than I would have imagined. Well, the more I use it, the more I hate it. I like only one thing: the fact that the final PDF's have been deemed "acceptable" by various print bureaus, and thus, DTP on Linux really is a reality. But using the fucking thing is a horrid pain in the ass!

Why?

Importing EPS or SVG sounds like a good feature, yes? But Scribus is very often unable to render imported vector formats properly. (Hmm, I really should include screen shots!) A fairly simple EPS that I was trying to place last night, for example, came out with the border and enclosed shapes in completely different places. Same thing with imported SVG!

(Another issue with imported EPS is that gradients get flattened in EPS graphics. But this appears to happen with all EPS in all programs -- does EPS not support gradients?)

I tried three different times to import an EPS graphic and get it to look right in the final PDF, and failed each time. I tried snipping the EPS into components and importing the components. I tried using only a bit here or a bit there and using Scribus's native vector drawing tools to do the rest. No luck.

So, how to insert a graphic, and maintain transparency? PNG? Well, that's what I had to resort to, but the problem with PNG is that is does not support CMYK. And CMYK Tiff's do not support transparency!

This leads me to the problem of CMYK and color management. How can I produce documents that look nice on screen, but more importantly, PRINT NICELY?? GIMP does not support Color Management. Scribus does, but documentation on using it is shaky at best. Maybe it is just a general lack of understanding of how to use color mangement. My question is quite simple -- how can I make what I see on screen print nicely? And maybe this has nothing to do with Scribus.

Okay, so enough on that, and back to my beef about "undo". Undo does not do anything in Scribus. It is simply not implemented. This alone is almost enough for me to hate using Scribus. WTF? The Scribus developers have implemented a "skinning" feature for the UI (useless, if you ask me), but "undo", a CRITICAL feature of moderne GUI software, is left unimplemented. Can't explain it.

Ultimately, I simply cannot figure out how Scribus gets shuch good marks by the "press" and people in the Linux community. Scribus sucks. Plain and simple. Assuming continued development, Scribus will not always suck, but for right now, it is still in the "suck" phase of its life cycle. Like the GIMP ca. 1998.

So the trio of DTP tools for Linux -- vector/Inkscape, bitmap/GIMP, and layout/Scribus -- are still not yet up to professional standards. Inkscape is Really Close(TM) and GIMP is Really Super Close(TM), needing only some print tools like CMYK separations and proper color management. Assuming continued development, I imagine that 2008 will be the year that Linux finally Is Ready(TM). By that time, OpenOffice will probably not suck, either. Not the most cohesive summary, yes, but it is a web-published blog, after all. Whaddya expect?

Well, at least Scribus did work, finally. Just a horrid, frustrating mess to have to use.

HTML was not intended to be a way to describe how a page looks. The modern use of HTML+CSS is a far more powerful way to design web pages than some older "hacks". Here are my suggestions on how to start using CSS.

I am cutting and pasting this from a Usenet post from earlier today. I wrote the post! It is not plagarism!

For layouts, use *either* CSS *OR* tables[1], but please not both. Unless a table is being used as intended: for tabular data (and a lot of things fall into that category).

In general, for a CSS layout, my "best practices" are as follows:

1. Start with a plain vanilla HTML 4.01 strict page which has absolutely zero author-imposed layou or design characteristics. USE SEMANTICALLY CORRECT HTML!! That is to say, a pargraph uses a <P> tag and a list of items uses <UL> or <OL> and <LI> tags, etc. DO NOT USE <DIV>s FOR EVERYTHING!!!!!!!!!!!!!!!!!!!!
2. Give classes and/or Id's to every element. Remeber, an element can have more than one class. Also, each ID must be unique!
3. Take an image of what you want the layout to look like and start applying CSS layout rules. This last bit is the tricky part, of course. Also, you can start to add a smattering of <DIV>s here and there to force the layout how you want it. But use pure <DIV>s sparingly!!

Why the warnings against DIVs? A <DIV> has no real semantic meaning and using only <DIV>s will create a (valid, yes) HTML page that is not search optimizable, not "508" accessible, and ultimately not as flexible as one that uses "proper" HTML markup as HTML was intended to be used.

[1] Some will say "never use tables anymore for layout" but I don't completely agree, for a few reasons. For myself, though, I have stopped using tables for layout.