Build Your Own Blog Archives

If you've read the website blog on smack talk you will know that I've just “enabled” emoji's on the website. Stupidly, I thought this would be an easy task as I figured it's just a matter of getting the database (MariaDB) to accept the code for each emoji. It kind of is that simple, but a cascading sequence of events happened that dramatically increased the amount of work needed to get there. I've already done a blog title “I knew an old woman that swallowed a fly” otherwise I'd have used that here.

OK, so what you need to do is have the database and each table allow unicode which I'm 100% sure I'd already done. It's quite possible that I did, but emoji's have become a thing since then and they require a specific unicode. In this case UTF8MB4. Not a big deal, just write a SQL script to change the database encoding then iterate through all the tables changing them.

It failed on about the third or fourth table. I've significantly cut down the number of tables post the LEMP conversion, but there are still 51 left. The reason it failed is that the primary key was too big. Apparently the max size of the key is 767 which is 768 bytes. This is not something Neville told me #blameneville. Obviously converting from the latin character set to UTF-8 means that each letter takes up more storage space. I'm guessing 4x the amount given utf8mb4 is a 4 byte UTF-8 encoding. I have some tables that have a 255 character varchar field as the key - apparently this is a bad thing to do anyway as it creates a huge index which I totally get, but didn't .... #blameneville. These 255 character fields are too big to be keys with the new encoding.

Right, so I could have cut the corner here and just converted tables that were going to store emojis rather than all the tables. But if a job's worth doing ... you really should get someone else to do it for you. Next step then, figure out which tables had inappropriate keys. Some of them were pretty easy - I could just assign an auto number as they key and move on. Unfortunately the biggest issue was the email field that I used as a key for the folks that are registered and to link to the race game and playlists (the latter now hidden, but still exists as I moved RagBox to personal use only).

Sidetrack for a minute. I made the email field 255 characters long because an email can be that long. Actually, I think it can be longer. Can you imagine though if your email address was 255 characters long and having to type it in every time you wanted to register or logon with something.

Let's get back on track - data conversion. The fix is to assign each person a unique ID, then to add that field to each of the tables that was linked via the email address. Iterate through the table and populate that new field with the ID associated with each email. Then make that field required and part of the key (as it would have other bits that make up the key such as the competition number). Then delete the email field as it's no longer needed in the referencing tables.

The structure's now in place and we can accept emojis. Only we can't because every page that puts data into the tables is trying to put an email address in an email field that no longer exists and is not populating a required ID field. This is for things like registering with the race game, changing registration, making selections, editing selections, becoming a member, changing your membership, password resets blah, blah, blah. It's more than you think.

Right, so just change those pages and we're up and running. Err, no. The whole logon process that I created recently uses email addresses as the ID and I changed it to the new ID (which has logged everybody out). This bit is actually quite funny. I knew I'd be logged out, but when I tried to log back in, I was locked out because it couldn't validate me for some of the back end stuff that I need to do to run the scripts to make the changes. So I had to un-code the logon, to push the changes, to then re-enable the logon so I could log back in.

We're done now though and the changes are in. I 💗 data conversion. There's something about just figuring out data structures and what needs to be done that I find really calming. I think it's the sense of putting order to something. Parts of the back end are still broken. I need to fix some stuff with the music, but it shouldn't be too difficult. I just need to finish the mobile friendly responsive pages for the front end first. The joy of hobbies.

This UI/UX upgrade I'm doing that leverages Bootstrap, utilizes a base 12 flex box model. This is one of the things my dad has been evangelizing forever. Not the flex box bit, but the base 12 bit rather than base 10. We all know base 10 because it's the number system we use and it's easy as when you multiply something by 10, you stick a zero on it. The challenge comes with division as 10 is not an easy number to divide - it's only divisible by 10,5,2 and 1, whereas 12 is divisible by 12,6,4,3,2 and 1 along with relatively easy manipulation at 8, 9 and 10. 12 is therefore a much easier number if you want to divide it into smaller components. Flex box is quite easy to visualize - imagine that every row has 12 boxes in it. You can then format your layout within that, so if you want 2 columns you make each column 6 boxes wide. If you want 3 columns, you make each column 4 boxes wide etc. You can also do odd sizes like make the first column 2 boxes wide and the other column 10 boxes wide. If you really need it, you can even sub divide each box into 12 more boxes to give you as much flexibility as you need. Hence flex box.

I was surprised when I caught the side of the Alpha Tuari during the pre season practice as I noticed that it had Flex Box on the side of the front wing. Turns out this is a container storage company that offers flexibility in the size of shipping containers.

All of this is great, but it's also the nickname I gave to a gymnast I used to date.

Yeah, so I logged onto the old Google console which is where you go to look at stats on your site and how well it's doing for the all important search engine optimization (SEO), that I couldn't give a shit about. And yeah, it was telling me that I wasn't responsive ... as if I've never heard that before ... as if I didn't know that.

Well, even though I couldn't give a shit about Google rankings because I'm not looking to sell anything on this site - you either find it and like it, or you don't. Matters little to me. But the whole responsiveness thing relates to the user experience of your site and is heavily weighted to retards. I'm sorry, we're not allowed to use that term anymore, the correct term is “iPhone users”. The Google algorithm uses mobile, particularly the iPhone as the benchmark for a site's experience these days.

Well, my site pre-dates the iPhone. Only by a few months, but it was built in an era of desktop browsing. I've updated it several times, but it's been many years since I've looked at the user interface (UI). I will be honest, the last time I updated it, I did look at Apple's site because, well, even though there tech is limiting, they know how to make something look good. Credit where credit's due. At the time, pretty much everyone was doing fixed width pages. Mine have been set to 1,000 pixels (pretty much keeping it just under the 1024 pixel margin that many monitors had at the time). What this enabled you to do was create a fixed base that a browser wouldn't mess (too much) with. You could then position things as you wanted within that fixed base. Accounting for any nuances the plethora of browsers that were around at the time had.

This works and it does work on a mobile device. The problem is that it is wider than most phone screens including that amazing bit of kit, the iPhone, which means that you have to move around a lot to see the content. Additionally, because it's fixed width and the fonts are fixed heights, they're consider small for a mobile device. I have to be honest, I don't disagree with any of this.

What many companies started to do was build parallel sites, so you would have your main website and then a mobile site. I did look into this a few years back and it just didn't make sense to maintain two sites. There are also multiple other problems with this that places like supermarkets ran into where you get different discounts on the mobile site than on the desktop site because the content isn't synced.

The brilliant thing about doing absolutely nothing about this when I last looked at it is that the world has moved on again (as it tends to do). Now, you can build a responsive site that essentially looks at the screen size and then adjusts the content to fit. That means one site, all devices. There are many ways to do this and it would be relatively nasty to get right for complex designs, however, the other brilliant thing about waiting is that folks have made the tools available to plug in and do all the sizing stuff for you. A huge shout out to Bootstrap for making their code available.

So, I'm slowly going through every page on the site again to make it responsive using Bootstrap. The good thing about this change, rather than the move from WAMP to LEMP, is that I can do it page by page as opposed to one big move. You'll slowly see pages change over the coming weeks as we take on a new responsive look.

This is a great question as the website has undergone the biggest change since I started it and nobody can probably tell the difference. The site has always been hosted on a WAMP stack (Windows, Apache, MySQL and PHP) and I've moved it to a LEMP stack (Linux, Nginx, MariaDB, PHP) on Docker. Additionally, I changed the file structure and some of the base coding across the site. It's amazing what you do when you're bored I guess. Off the top of my head, here's the details and impact:

• Docker - previously all the websites and services were mushed together with one Apache server with four virtual hosts connecting to one PHP instance that connects to one MySQL instance. Now, we have an Nginx reverse proxy in a container that connects to four other Nginx servers on their own containers that connect to their own PHP container that in turn connects to its own MariaDB container. This means that each site can be split off and moved very easily if needed
• Windows to Linux is a big change because Linux is case sensitive and Windows is not. My coding and file naming hasn't been consistent, so I've had to correct a lot of links, but this does mean I can port the websites to any hosting environment at anytime with no further modifications if needed.
• Changing the file structure again means re-coding all links. A lot of this was done to be consistent on how media is stored so I can access it from a media server. Previously videos from photos were under photos, videos from games under games (with random sub folders), videos from videos under videos .... you get the point. Now it's all neat and tidy.
• Changing to MariaDB from MySql actually doesn't have an impact as they both use the MySql language and libraries. MariaDB is an open source fort from MySql after the latter was bought by Oracle.
• Upgrading PHP causes deprecated functions to need to be changed. There were a few of these, but the biggest impact was deprecating implicit translation of values or null, specifically, passing non integer or null values to functions that require an integer. Previously it rounded the value or treated null as zero, but it don't do that no more.
• Moving the database calls from MySqli to PDO meant that I had to re-code every part of the site that calls the database. I also added try catch to the call functions to make it a bit neater. This change is more secure and potentially more portable.

On top of that, there were some fixes to some broken pages that had crept into the site, the creation of the 2023 F1 Fantasy Race Game adding a pretty meaningless Chart Generator because I needed it and decided to just make it available.

I also standardized the logon process across the site and created a my account page. There's nothing really there yet, but it's a framework that I can build on. You now need to logon to post to the forums along with playing the Race Game. In doing this, I decided to add encryption to the password file, but you should still not use a password you use elsewhere. I also got rid of all the home page naming. Originally everything was a home page, e.g. photoshomepage or videoshomepage and I've just removed the homepage bit. No reason, other than it made me feel better and if I'm redoing all the links on the site, now's as good a time as any to make those changes.

It's all been a lot of fun as I've been learning a lot of things along the way.

Yes, the good old chain of events where something sounding simple starts and things take on a life of their own. I received an email a while back from Google letting me know that they were no longer going to allow authentication to their mail servers using an ID and password. This impacts the weekly email that is sent from this site and some other functions. Simple, just change to Oauth 2.0 and you're good, right?

In order to use Oauth 2.0 I need to update my mail program (PHPMailer) ...

... in order to use PHPMailer with Google, I need to implement SSL and turn on https with Apache ...

... in order to turn on https with Apache I needed to upgrade my entire stack (WAMP) because somehow the ssl_module on the version I had wasn't working. Assume it wasn't compiled right as there were several posts from folks having a similar problem ...

... upgrading is always fun as there's a bunch of functions that have depricated since I wrote my code, so that means re-writing a lot of the backend code for the site ...

... before installing and rewriting code, need to take a backup ... my NAS fried, so I had to get a new one ...

... ok, so I got a new super duper NAS and it came with loads of bells and whistles, so I did get quite sidetracked with that. It comes with a streaming functionality, so you can stream to it and it will record the stream and then restream out to any number of RTMP services. Right now I have it setup to stream out to Twitch, YouTube and Facebook. There's literally no reason for me to do this other than the fact that I can. And sometimes, that's the best reason of them all :) ...

... sidetracking done and back to the task at hand, backup website, installed new software, install composer to get new PHPMailer, install certbot so I can get SSL certificate from Let's Encrypt which is great as it provides free certificates and free is good, particularly to a project website like this that is not monetized ...

.. infrastructure in place, do some tidy up and recode parts of the website not working. Bit of a note here, there's still some, but it's not related to this. There's a flash plugin used on one page that I need to update when I get time, but where does one get time once you're a megastar streaming to absolutely nobody on three platforms?

All in all, a huge amount of fun as it's been a while since I've done anything with the site and even though visitors won't notice the difference, I do.

I've finally been conned into giving giving up my back door!

Whilst this may at first sound like another post about my illness, it is not. I'm referring to the fact that I've enjoyed the ability to run my own webserver for the 7 years I've been hosting this website, even though my Internet Service Provider (ISP) Comcast has posted for many years that they block port 25 inbound. They do indeed block port 25 outbound, but the inbound has always remained open. At least I've always managed to get inbound port 25 maybe through some backdoor access. Anyway, that has always seemed pretty reasonable to me as the risk of leaving the inbound open would only be to my server and not to them.

A few months back I started getting letters and emails that I needed to upgrade my modem to the latest and greatest in order to get better upload and download speeds. I ignored these to start with as I was in the process of divorce and moving house and it really didn't seem like the most important thing at the time if I'm being honest. Well, after I moved, I thought I'd treat myself to the super duper new faster modem, so I purchased one, hooked it up and a phone call to the ISP later I was up and running. What they didn't tell me was that they flashed the modem with a config file that blocks inbound port 25 as well as outbound. I might be wrong, but I'm pretty sure they're controlling this access with the security policies on the cable modems. I'm sure there's a way to hack this, but to be honest, it seems like that's a path of trying to get yourself banned and more trouble than it's worth.

Google used to offer free email domain hosting with their Google Apps service, but this has been deprecated since December 2012. Microsoft appear to be offering this service with Outlook.com and, if I had more time, I might look into this as it does look like a pretty neat service. The reason I didn't jump on it is because it's a web service they provide and I need to know if I can connect my own interface into the email server to send emails - for sending out things like the weekly email summary. I'm pretty sure you can and it's probably simply running on port 587 or maybe 465, but I got to thinking about how much I actually use email for this website and it ranges somewhere between very little and extremely little. I've set up emailing through gmail for some friends, so that seemed the fastest route to go through as it didn't involve any research. So gone is xx@eastbayrag.com to be replaced with a single email of eastbayrag@gmail.com that will serve as this website's point of contact going forward. Well, I've left my email server up and running behind its Comcast protected firewall so if it ever gets opened or there's a way to have your inbound email route to a port other than 25 I may start it up again, but until then ....

Now, for the nobody who's reading this as I'm sure this is purely for my own benefit, you may be wondering how the weekly emails have been working whilst I've had this issue. Well, the block only stops me receiving email. I can send email by relaying my server off the Comcast one, so outbound works fine. Crazy really.

I doubt anyone’s been wondering about this, but on the off chance that it was something you were interested in, I thought I’d let you know what goes on behind the scenes. The website that you view is less than half of what has been created. For every part of this site, there are some administration tools. Really just a web interface that lets me (or Mr. O) make changes to the data on the site on the fly. As all of the actual data is stored in a database, it’s pretty easy to create a front end that lets you make updates.

If you’ve looked at the photos part of the site, it’s no coincidence that each of the pages look the same. I’ve created a front end tool that allows me to link to a directory, enter the page title and it will upload all the pictures in the specified directory, create thumbnails and the double click pictures, then create the pages that are you get to see. Pretty clever, even if I say so myself.

The big challenge really is that it takes more effort to create the maintenance tools than it does to create the part of the site itself. Take, for example, the forum section that I’ve just created. The site bit itself is quite clever in that it only uses four pages that display different data depending on variables that are passed to each page. On top of that there are another 11 files that allow you to create the entries (categories, sub-categories, thread titles and thread details) and edit the thread details if necessary making a total of 15 files.

There are 22 files in use to support the forums. These allow me to edit or delete any of the entries made. The ability to delete is actually the hardest thing to program here as you need to make it a cascading delete otherwise you would be left with orphaned records, for example, being left with sub-categories that don’t have a category to belong to. In addition to that, I created the ability to move either sub-categories or thread titles – so, if a thread seems to be in the wrong place I can just move it (or if many similar threads get posted under “Other” I can create a new category and move them over). Before you ask, you don’t need to move a category – just editing it has the same effect. There’s no real reason to move a thread detail – it’s unlikely to be posted in the wrong place, but in the unlikely event that it was, it’s probably easier to just cut and paste it into the new position (and by cut and paste I mean create a new entry in the right place with the same text and delete the old record).

So, if you were wondering, now you know.

Well, the time has come for expansion. William's second birthday was coming up so I decided it might be nice to take a little bit of video rather than just taking photos. And with video comes the need for more disk space. Recognizing the need in advance, I rushed out to buy the biggest hard disk I could find for my computer/server so I had somewhere to put the video to download. The picture below shows me with my purchase prior to installation. Proper American style "big"!


So, joking aside, I've seen lots of adverts stating how easy it is to install an internal hard drive. If you can use a screwdriver you're done! Well, that wasn't my experience and I've changed many hard drives before. Admittedly my prior hard disk installations were about 15 years ago in an MS-DOS v5.0 environment so things were probably a little simpler then.

Part of the problem may have been that I ordered my gigantic hard disk on-line and it arrived by itself in the anti-static wrapping, but no box or instructions or anything of any help really. Not a problem, I can figure this out. First, take the computer to bits - easy enough, the Dell I have has a removable side panel that opens with the turn of a knob. OK, so you need to know which drive is going to be the master and which is going to be the slave. Again, easy, the one in there already has the operating system on it so that will be the master and the new one will be the slave. OK so the existing drive is in the wrong bay. Yep, no worries, just take that one out and swap them over, screw them in and ...... oh! As I said, the new drive came on its own in the anti-static wrapping. When I say there was nothing else with it, I really mean nothing else - not even the screws needed to hold the drive in with. Luckily I have an old floppy (the years have not been kind to me) from which I raided the necessary screws. Couple of turns and it's all back together and ready to go.

Now, before I even turned the power on I realized I'd forgotten to set the jumpers on the drives to designate which is the master and which is the primary. I'd plugged them into the right place, just hadn't told them what their role was. Not sure how this popped into my mind - it was a flash back from having previously installed drives as there's nowhere that it tells you that this is a required step (given that it didn't come with any instructions). Being optimistic I turned it on to see if it would work but, as expected, it didn't. So, take it all to bits including removing both drives, set the jumpers correctly (after searching the manufacturers website for the settings) and put it all back together again.

Switch the computer on again and hey presto .... Nothing! OK - restart go into the bios, make a few changes, restart and the bios at least picked up the drive. Now, back in the good old days, once you configured the bios to recognize the drive you had the option for format it from the bios. Well, not anymore, you can still use the DOS fdisk or Windows disk management. Might as well use Windows, that's bound to be easy. In fairness, it is very easy to use. I did, however, notice that it was saying that the disk was only 128Gb when in fact it was 500Gb. I thought I'd format it anyway and then see if it figured it out during the process. Forever the optimist and again wrong. It only recognized the drive as being 128Gb. I restarted, looked in the bios again and that had it as a 500Gb drive. The hardware driver also had it as a 500Gb drive so what was wrong?

After more searching of the world wide web and coming across an article on systems only recognizing drives up to 137Gb unless large drives are enabled in the Windows registry. So, it seems that your system needs to be able to cope with large drives and then you need to tell Windows that you've stuck one in. If you have this problem and end up on this site, you need to enable large block addressing by:

• Open regedit (Start > run > regedit)
• Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Atapi\Parameters
• Right click the word "parameters" in the left window and select "new" then "DWORD value"
• Name it "EnableBigLba"
• Double click it and enter a value of "1"

In fairness, it's not that difficult and would be very easy if there were a step by step guide. I'm pretty sure the newer operating systems don't require you to make the Windows registry edit, but I'm using Windows 2000 so that may have contributed to making this more difficult than it needed to be. That said, a little more involved than just being able to use a screwdriver.

A deviation from the theme of building your own blog as this deals with the technical stuff on the experrymental page and therefore more of an article on how do you make stuff fly round the screen.

The experrymental page came about as somewhere that I was practicing Javascript for the sake of practicing Javascript. Unlike the blog pages or most of the site where the processing is being done server side with PHP, this page is executing the code on the user's PC (which is why you may get messages from your popup blocker, given that these blockers are looking for scripts that are trying to run on the user's machine).

So, the premise for the page was pretty simple, just make available the rubbish that I was practicing with. Seemed pretty straight forward, but as soon as I made it available it seemed to take on a life of its own and ultimately became a project to enable a user to enter text onto the screen and have it rotate round in different ways whilst having the background change color and the EBR logo flying around.

The page is pretty simple - each "bit" has a timer that counts down to zero and when it reaches zero, it does something different, for example the logo will change direction, anything changing color will get a new target color to change to or moving text will have a new destination to move to. Each timer is independent so you don't end up with all changes happen at the same time. Also, whilst researching some of this stuff I found several examples of text being rotated around the mouse so I thought it would be good to include that as well.

Great! Now we have the definition of what's going to be created, just need to code it. Now comes the big challenge. Everything that is coded to be processed server side is easy as I know what is going to do the processing. Unfortunately with client side processing, I have no idea what you (the user) are going to look at this website with - Microsoft Internet Explorer, Netscape Navigator, Mozilla Firefox to name a few examples. And they all seem to process stuff differently which is pretty annoying. That means that a lot of the scripts need to start with finding out what browser the user is using and then insert logic of "if IE, do this, otherwise do that".

For the most part I'm pretty sure the page works OK on all latest version browsers. There are also some "fudges" in there to make it work on some older browsers. Specifically IE6 as that's what I have at work and I noticed it wasn't working properly. The problem I have with IE6 is that it doesn't seem to let me specify the size of a box (div tag). So, if you look simply at the background color effects, there are 36 boxes that change color according the pattern specified. In IE6, nothing was happening as these boxes (div tags) are empty - it seemed to be resizing the boxes according to content. The fudge is that I've just put tables in there and filled them up with non-breaking spaces. Seems to work.

What's all this got to do with PI I hear you ask. Well, bugger all really. PI comes in when calculating the shapes for the text to rotate or be displayed in. The triangle, square and rectangle just use a simple means of keeping track of where each letter is in an array to enable rotation. If we look at the square, it calculates the size of the square (in particular each corner), then each letter is assigned a letter of l, r, u or d for left, right, up or down. When a letter with array d gets equal to or less than the position of the bottom row, it changes direction and becomes an r. Really simple.

Then came the circle (and oval, but it's the same as a circle except the y coordinate is halved). Back to high school math here and something to do with the fact that the circumference of a circle is 2πr and the fact that Sine (sin) and CoSine (cos) allow you to convert the 2π bit into a value of -1 to 1. Without going into all the math, if you divide 2π by the number of letters (lets call this n) you have you can plot a circle by using x = r*Sin((2π/n)*letternumber) and y = r*Cos((2π/n)*letternumber). This is just a fancy way of drawing a circle by using a pen and a piece of string. The string is pinned to the page and then you move the pen round the center to get the circle. All the above equations do is calculate that for you from that center point.

As Homer would say mmmmmmm PI. Rather than go back and correct the triangle, square and rectangle, I left them as they were and modified the circle to create the Pentagon through Dodecagon. Given that we can calculate the circumference of a circle, if I divide the circumference by (say) 10 and join each dot, I would get a decagon. Tada! Nothing more difficult than that to get these shapes. OK, maybe a little. I still use the array associated by each letter to assign a number to each letter to determine what side it is on. By using a number it is flexible to cope with any shape and can also be used in the calculation, so a letter on side 2 should keep moving the specified increment (determined by the speed setting) until it reaches the start of side 3. It would know it's reached its destination by x = r*Sin((2π/n)*(sidenumber +1)) and y = r*Cos((2π/n)*(sidenumber +1))

In theory the formula will take any number of sides. I stopped at 12 as any more and it's difficult to tell the difference between the shape and a circle so it becomes meaningless. One may also conclude that I thought it would be too hairy, but that would be another joke for another time.

Well, after writing that intensive article above that has the potential to actually be helpful to someone, I looked around and thought wow! I'm really creating something here. But it didn't take long for reality to set in.

Those of you that have been following the OBlogs will know that the weather in the UK has been pretty miserable. On the other hand, we are in the middle of the hot California sun which is great unless you are a computer that needs to be kept cool.

Not to worry. Those of you that are worried about the EBR server farm will be happy to know that we've spared no expense to install the latest in air conditioning technology to keep the system cool as you can see from the picture below (which can be double clicked to get an even better view):



The funny thing is, I was thinking about moving the system downstairs where it's cooler, but that would mean connecting a couple of wireless routers. Given the entire system ran at $142.50 it didn't really seem cost effective to shell out for the wireless routers. Whilst downstairs, however, I found this old fan which seems to be doing the job pretty well.

A couple of week's back, a friend of mine asked if there was any way he could send me a file. Not for the site, just a video. So I created a quick form that would allow him to submit a file. Not a problem, in fact a pretty easy task until you find out he wants to send a 1.6Gb file. Still not a problem, you just have to adjust the php.ini file to allow for an upload of this size. Contrary to a lot that I've read, the only changes that need to be made are to the post_max_size and upload_max_filesize variables to allow for a file of this size. (I set them both to 10000M and that seemed to work fine). Quick test and done - works OK so I sent him a link. Next I get a note saying that there's no way to tell if it's working. In other words, you click submit and nothing happens until the file has completed the upload which, with a 1.6Gb file over a home network takes about 12 hours depending on the upload speed of the sender. After assuring him that it was in fact working, even though nothing appeared to be happening, he let it run and I got the file. Still, seemed like a good idea to figure out how to create some kind of progress bar.

This, I find is incredibly difficult. First, this appears to be something that has only recently been made available in PHP (v5.2). As best I understand, the latest version has some hook that you can get information from once an upload starts. Great - I'm on the latest version so this shouldn't be too difficult. PHP has these extensions (called PECL's) that you can install to give additional functionality. A quick look and there's a php_uploadprogress extension so surely you just install it and add the right syntax in your code and we're done. Well, best I can make out from the logs I've visited is that the person who developed the php_uploadprogress extension doesn't have a Windows machine, so it works great on *NIX systems, but the dll file doesn't actually work. It appears to have been fixed by someone else and is ready to be uploaded as a revised version to the PECL library, but doesn't appear to have been done yet. What's more, I found the revised dll file on another site <<<link removed as site no longer exists>>> at lunch, however, when I went to download the file at night, the site was down for 3 days. Thankfully, I finally got access and downloaded the dll and it worked.

Next thing is to code a php file to do all the calculating of the upload. I used 4 files to get this to work as follows:
- upload file form (where the file is submitted from)
- upload file progress (to process the upload once complete)
- upload progress (to display the progress of the upload)
- upload progress bar (used to create the dynamic picture of an increasing bar)

Starting at the beginning, obviously you need a form from which to submit the file. This is pretty easy. The form type must be of type "multipart/form-data" to send a file. The field to attach a file is an input field with a type of "file". You also need to create a hidden field before the file field called "UPLOAD_IDENTIFIER" in order to get the uploadprogress dll to work. You need to create a unique ID for the upload to work. I did this simply by using date+time+random. I'm not expecting a plethora of uploads such that this will not work.

Second step is to pop open a window from which to report the upload progress. Originally I tried to do this using the onclick event with the button which worked fine in Firefox, but not in Internet Explorer. The solution is to use the onsubmit event in the form as this seems to work in both browsers that I have. This window needs to refresh itself every x seconds to get updated information on the progress of the upload. I refresh every second, but you can figure out what works best for you. I used the onload event of the body tag to call the setTimeout function. This has a refresh time of 1 second (1000 milliseconds) and has the simple action to refresh (reload) the window or close it if the upload is complete. You can then use the uploadprogress_get_info() function to retrieve information about the progress of the file upload. Basically, installing the PECL and creating the UPLOAD_IDENTIFIER allows the system to tag upload information that can be fetched and put into an array with the uploadprogress_get_info() function. It provides the following:
- time_start - The time that the upload began.
- time_last - The time that the progress info was last updated.
- speed_average - Average speed.
- speed_last - Last measured speed.
- bytes_uploaded - Number of bytes uploaded so far.
- bytes_total - The value of the Content-Length header sent by the browser.
- files_uploaded - Number of files uploaded so far.
- est_sec - Estimated number of seconds remaining.


The graphic of the bar is created by calling a third file and passing it the variable of where the progress is. So, just calculate the % complete using the bytes_uploaded compared to bytes_total and pass it over. Nothing more complicated than that. (Actually, I did spend a lot of time trying to output a GD file in the upload_progress file, but couldn't figure out how to do it without saving the file and loading it which seemed excessive. If I've missed something here, it would be greatly appreciated). The graphic uses the GD capabilities in PHP so you will need to load the GD PECL. If you don't want to do that, you can just display text based progress.

Finally, the uploaded file is processed. In some places of this site, I do things with the file - normally to re-sample a graphic for display, so I reject files I can't handle.

The only thing missing from this description is that I use cookies (mmmmmm cookies) to store information as to where the upload is. Given that the upload progress disappears once the upload is complete, it is difficult to tell the difference of before the upload has started and once it is complete. As the file upload form is loaded, it creates a cookie with the unique ID that will be used for the file upload (in order to allow multiple uploads). It gives the cookie a value of 0. When the upload progress window is opened, it looks to see if the upload has started (if an info can be received from uploadprogress_get_info()). Once started, it gives the cookie a value of 1. The upload file process sets the cookie value to 2 when it loads as this is when the upload is complete. This is used to close the pop-up window instead of refreshing.

All the files I used including the working dll are in the attached <<<link removed as files are old>>> file. I haven't done anything with the dll file and am not trying to take credit for the work of others, that belongs to the original creator as noted on the PHP website or the person who updated it on the link provided earlier in this article. It just took me a long time to collect the bits, so I'm making them all available here. In addition to the notes above, these require you to have a directory on your root called "uploadedfiles". You also need a setting in your php.ini file to set the temporary directory for the uploadprogress extension to save files uploadprogress.file.filename_template = [root]:/tmp/%s.txt (note you need to include the %s as this is replaced with the file name of the upload - the unique identifier you created).

The files in the attached zip are the basic working files (they don't have any fancy headers or footers). If you want to have a go and see the upload progress in action, send me a file (just try and make it interesting). I've obviously added a lot of stuff to the form at the end of the link to match the house style of this site.

A couple of last notes. At times during a big upload, the upload file disappears, so you will jump to 100% complete and then back to whatever the correct % is (one more reason why I had to use cookies). Second, in MSIE, the bar flashes on and off each time it loads.

And finally, if you still have problems, let me know and I'll see if I can help.

As well as being an absolute classic Motörhead song, B-side of Iron Fist if memory serves (which I have on red vinyl tucked away in box somewhere in the UK), this is a little article on caches.

I'm not going to attempt to explain all the stuff you have to put into the top of a web page to stop the browers trying to store things to memory (as there appears to be different code for each browser) you can just search for this and you will find it. Where I had problems was stopping the browsers from putting images in memory. For the graffiti wall, as described above, an image is created for the text that is entered. This is fine, you get taken to a preview page so you can see if you want to regenerate if the color or font don't work well. Now is when you hit the problem. It's easy to get it to re-create the image (say img0001.jpg) and to get the browser to reload the page. But, the browser think's it already knows what img0001.jpg already looks like (it copies it down to your temporary files so it doesn't have to keep getting them from the server - again the browser seems to think it knows better than the person writing the page). The result being that even though the image has changed on the server, you won't see the change on your browser as you will be looking at a stored image somewhere on your hard disk.

Originally I thought there was some kind of image cache that needed to be disabled, but it is just as simple as the browser is trying to save time with image reloads.

How to fix? Trick the browser into thinking it's not seen the image before. You can transfer variables to an image or file when it's loaded and we can use that to our advantage here. instead of loading an image with src='img0001.jpg', use src='img0001.jpg?'.rand() [note PHP syntax, you may need to use +Math.random() instead of .rand() if you want to do this in javascript]. The question mark just states where the file ends and where the variables start. unless the variable has been created before, the browser will ask the server for the new image. Using the random function is a simple way of generating a number that the browser [shouldn't] have seen yet.

But what's more important is that I got to write an umlaut.

I guess it's time for another picture. There's been far too many articles on this page without anything to spice it up. So, what could be more appropriate than yet another thrilling installment of "pictures of my computer"? I know you want more ...

But I thought I'd spice it up! After looking at the other picture for hours on end (as it's totally fascinating), it dawned on me that I could get more desk space if I but the box under the table. So, backups complete and all that good stuff, it's time to move the machine. And voila! This is what it looks like now:


Much nicer don't you think. And if you haven't already, double click on the image to see how much improvement there is.

Weren't expecting that were you! Well, you probably were if you paid any attention to the title, but it amused me. Don't forget to tune in next week for Rag's bored, lets see what he can come up with.

Ah! The Graffiti site. A somewhat novel idea in that I thought of this by myself rather than stealing the idea from somewere else, although I'm sure there's another one somewhere else. I doubt I'm that innovative.

So, lets start with how it works. The page you are directed to is a frameset page. What that means is that you are viewing several web pages at the same time. The frameset itself is like a container that simply says what it's going to hold and where. The graffiti page is fairly simple in that it only holds two pages - the graffiti wall and the graffiti form (the entry page). The buttons at the bottom of the page use a little javascript to alter the size of each frame in order to "pop up" the graffiti form and allow the viewer the ability to submit entries or navigate to another part of the site.

Once you enter the details and click submit there is a fair amount of technical stuff that goes on. The text is converted into an image and angled using a back end PHP script with embedded GD (a graphics package that integrates with PHP - it comes bundled with it on the latest versions of PHP). I have to say, the GD package is fantastic as it allows you to do so many things. I've created dynamic pie charts and bar graphs with it that are linked to the vote page. (I'll probably write something on these later). The text is created using random colors and random fonts (from a list of about 10 fonts). Then, after you create the image, you are taken back to the graffiti wall to view your creation.

Things I learned whilst doing this are that old browsers won't display the alpha channel of png files. That sounded like I knew what I was saying didn't it :) Actually it just means that the old browsers will not show transparent png images, they convert it to black. Given that the whole idea is to have the wall in the background, a black image is next to useless, so we now use gif files. (GD is great - you can create whatever you want). Next thing is that gif files are not as good at displaying text as png's. (At least those I created aren't). So you need to choose your fonts carefully. This was actually agonizing as when I started, I went mad downloading free fonts that were very intricate. These look great in a document, but less so in a gif file. Anyway, the result works even if the text is displayed much bigger than I intended and is separated out significantly. Originally, I wanted to show a lot of graffiti together so it looked more like actual graffiti.

Finally, and probably the biggest shocker, is what crap are people trying to sell on the Internet and does anyone actually buy this? Obviously, for the wall, I wanted a brick wall background. Rather than go out and take my own picture of a brick wall, I figured that I'd just search and download one. Well, there are thousands and most people are trying to charge for them. It's a picture of a wall! Why would you buy that? Most of the sites were looking for around $8.95, but there was one that was asking $49.10. $49.10 for a picture of a brick f wall! And I think we all know what Axel Foley would have said to that. And you can take it with a little lemon twist!

I had to include some techy stuff somewhere, so here it is. I believe this code is universal and will work in any language.

While ($abilitytostand != False) {
         drink($beer);
         if($beer==Null){SELECT $beer FROM fridge WHERE label = ‘Carlsberg’}
}

I didn’t name this blog BYOB for nothing. Thanks for reading.

With that list of software, I guess the question is “How does it work?” Well, lets start with what happens at your end. When you launch your browser, you point it towards a webpage, lets say www.eastbayrag.com. Your browser shoots a request off to your Internet Service Provider (ISP) that asks “where do I find www.eastbayrag.com” Your ISP answers with “I don’t know, but this server over at ZoneEdit does” so it trundles off to ZoneEdit to get the IP address. When it gets the IP address (which is kept updated with Inadyn software) it then points you to my server. The request comes into my server and is picked up by the Apache software. This figures out which page you want and gives it to you.

Now, from my end, I get to control what you get to see. Apache is configured to pick up index files if you request a directory and to perform different functions on different document types. Basically, .html files are just given to you, .php files are read and parsed for includes (e.g. the header and footer of most pages are the same and this is achieved through an include rather than repeating the same code over and over again on each page) and .php files are pre-processed with PHP.

PHP allows me to script items as we go along making it easier to maintain the website without creating thousands of pages. Some examples of this are as follows:

1) The photos pages display thumbnails of each image and allow you to double click to view a larger image. Each directory is created the same with a subdirectories of “small” and “big.” The PHP code looks in the small directory and displays all the images in the table format you see. It also embeds some javascript that allows the double click function.

2) The blog pages are sections stored in a MySQL database. PHP is used to retrieve the information and display it according to which blog you select. When you submit a comment, you make an entry that is picked up by PHP and put into the database so it can be retrieved later.

In its simplest form, the PHP script does something and then outputs it in another code (HTML/XHTML) that can be read by your browser. Hence the term pre-processor. So you use one language to write another language. Actually a little more complicated, HTML/XHTML can’t do anything it just displays stuff. Javascript is a language that exists on your browser that allows things to happen, so PHP is used to output to multiple languages that are read by your browser. The easiest example of this is the buttons at the top of this page. There is a little bit of javascript that basically says when you put your mouse over the top of the button to swap the image.

Simple really!!

A big thank you to all of the software providers I’ve used. The list is as follows:

PHP used as a preprocessor for most pages. PHP allows you to script pages and images dynamically on the server before they are sent to the browser. For example, the page you are browsing now doesn’t really exist – it is created using PHP and served to you in XHTML format with some javascript thrown in.

MySQL is the database engine used to store data as needed. These blogs are stored in a MySQL database and fetched by PHP to be displayed in XHTML format.

Apache as mentioned previously is the webserver software. When you connect to this site, Apache sends you the page you are looking for.

Crimson Editor is the software used to write the code for the webpages whether they be in HTML, PHP or whatever.

hMailServer is the email server that I use for mail sent to eastbayrag.com.

ZoneEdit is the domain name registrar I use to direct all the www.eastbayrag.com traffic to my server.

Inadyn is the software used to update ZoneEdit with my IP address (sadly no longer active). I have a dynamic IP address so I need to keep my Domain Name Server (DNS) updated with my IP address if it changes so requests are correctly sent to my computer.

LogValidator from W3C. As mentioned above, this is used to test the appropriateness of coding against the standards.

Perl is another scripting language. This is required to use the LogValidator program from W3C. This looks really powerful and I look forward to using it more. At the minute I’m just using it to run the LogValidator.

TightVNC allows remote access to the server so I can perform maintenance or transfer files remotely.

Probably the only useful thing I’m going to write in this entire blog. There is some bug between Windows 2000 and Apache that results in remote addresses [‘REMOTE_ADDR’] being captured as 0.0.0.0

Took me a while to search for the solution, however, in your Apache configuration (httpd.conf) file include the line - Win32DisableAcceptEx – makes all the problems go away. Doesn’t matter where you include this apparently, although I’ve not moved it around to see if there’s anywhere I can put it where it won’t work.

This blog probably belongs later on, however, it’s something I wished I knew earlier in the project so I’ve included it here. Did you know that all web documents are supposed to conform to certain standards? Well, I didn’t. The World Wide Web Consortium (W3C) have created various standards for documents to conform to releases of HTML or XHTML. Kind of weird, but finding this out after creating a number of pages confused me as it seemed to be telling me that I need to state what document I had created. The pages I created worked in that they load into the browsers and display pretty much what I want, so why bother? If it ain’t broke, don’t fix it. Maybe, but if a job’s worth doing, it’s worth doing well.

Excuse me while I ramble off into a war of sayings. Truth is, the standards are there for a reason – to ensure that the instructions are interpreted correctly. Also, they’re used by the search engines to a certain extent, so you benefit by following the standard.

W3C offers tools that can be used to test the validity of the webpages you’ve created. Something that is well worth taking advantage of.

The one thing that confuses me though is whether or not W3C is the correct acronym. I know that when you multiply something out it doesn’t matter which order the components are in, but following the logic of Y2K it would seem to me that W3C = WCCC rather than WWWC. Maybe Worcestershire Country Cricket Club have started defining standards for the Internet.

On another ramble and back to the title, did you see the episode where Xena and Gabrielle kissed? One of TV’s greatest moments I feel.

But then again …. Well, that’s a maybe, but I have no regrets over using Apache or any of the software available under General Public License. I will include a list later of all the software that I’ve used. For the time being, suffice it to say that the total cost is $0.

The only thing I would do different if starting again would be to strip the Windows operating system and stick Linux on. Maybe this will be a project for the future. (And this from someone who has been force-fed Microsoft. I even have an xBox and an xBox 360).

I guess the first decision is which webserver software to use. There’s plenty to choose from, but by far the most common are Microsoft’s IIS and Apache. Well, my Ebay box came with W2K pro on it, so it makes sense to use IIS right? Well, not so fast. There are plenty of limitations with IIS unless you pay for the server license (only 10 concurrent logons) which may cause problems. Again, use your favorite search engine to look for IIS versus Apache if you want to read more articles about the difference.

It wasn’t these differences that made me choose Apache. I spent hours trying to find IIS to install it. IIS is free with your operating system – all you have to do is stick in your system disk and load it as an optional extra. Ah! That would be the problem. This is an OEM machine that comes with the software bundled, so there is no disk to stick in and install from. Surely then you just need to download it from Microsoft’s website. Hmmm, not quite. The website just tells you to stick the disk in and install from there.

Being a product of the Microsoft generation, it made me nervous to try something different. Going out the comfort zone, but if Microsoft aren’t going to make their product available then we have to look elsewhere. Nothing ventured, nothing gained – so I installed Apache.

(True story – many years ago [pre 9/11 I hasten to add] a group of us were traveling through an airport with shotguns that need to be declared at customs. The airport in question was small and a couple of the guys missed the “red channel” and walked through. When confronted by the customs official with "didn’t you see the arrows?" They responded with “Arrows? I didn’t even see the Indians!” Needless to say the customs officials were not amused and we all got to stay longer than expected in the airport whilst the entire group of 20 had their luggage searched with a fine tooth comb).

Yes, we’re onto the setup. Rather than bore you with all the details, here’s where we take advantage of other resources. DSLWebserver (edit http://www.dslwebserver.com no longer seems to be working) has pretty much everything you need to know about setting up your own webserver. Instead, we’ll look at some of the choices made.

There’s an advert on TV in the States that has a pre-teen girl talking to her mother in "text message" speech. Complete with subtitles for those of us that don’t understand abbreviations like OMG or TISNF. I guess I’m the older generation now as I obviously need subtitles. Far be it from me to understand these abbreviations, I mean … WTF.

Any of you remember those old jokes when computers first came out about homosexual command line prompts – C:Enter I prefer the American version (utilizing the fact that this "#" is the pound symbol rather than the hash symbol) – C:Enter:###

What’s all this got to do with this blog then? Bugger all really – thought I’d sidetrack a little. I ended up on some site somewhere during this project that talked about how to write the best blogs. Really, all it said was to include photos to make it more interesting. My guess is that article was written by a pre-teen that had far more interesting things to write about than putting together a webserver and blog site. But, not wanting to debunk the idea before I’ve tried it, please find a photo of my Ebay purchase. And, through the wizardry of some javascript, if you double click on the image in this blog, the picture will open up in a larger window so you can see the details more clearly. Go on, give it a try ……


…….there, where else can you have so much fun?

So, how did it all begin? Rather than how, more of a "why did it all begin?" Originally from the UK, I got a job offer in a land far, far away (the good old US of A) which I decided to accept. Nothing strange until you get to the bit about the girlfriend deciding to relocate as well. (Not strange that a girlfriend would want to move to the US, strange that someone into computers and Star Trek has a girlfriend).

Anyway, job in the US --> Girlfriend comes to US --> Get married --> Have baby --> Need to share photos with the grandparents back in the UK.

So why not use one of the many websites that allow you to share photos? First, they make it difficult to transfer files in bulk – you can download one at a time, but you can’t leave the computer downloading a large zip file. Second, it would be too easy.

Third, and probably the most important reason, I was zoning out one Sunday afternoon watching the cars go round and round whilst surfing the web. Ended up on Ebay and started bidding on stuff. Found a computer that was mis-listed and going cheap, bid on it and won. Therefore have a spare PC that needed a project in order to justify the purchase.

All makes sense really when you think about it.

Welcome to the Build Your Own Blog (BYOB) blog. These entries describe the process I went through to create this website and how some of the features have been created. This website is created and hosted on a server that sits in my house, rather than utilizing one of the web hosting services, so articles will cover setting up your own webserver as well as the content components.

These blogs will not necessarily walk through all the technical details as there are much better reference sites elsewhere on the web. Links will be included to these reference sites where applicable. The focus here will be to add commentary or describe why certain choices have been made. And most importantly, to add a little humor where possible.