Build Your Own Blog

If you've read the website blog on smack talk you will know that I've just “enabled” emoji's on the website. Stupidly, I thought this would be an easy task as I figured it's just a matter of getting the database (MariaDB) to accept the code for each emoji. It kind of is that simple, but a cascading sequence of events happened that dramatically increased the amount of work needed to get there. I've already done a blog title “I knew an old woman that swallowed a fly” otherwise I'd have used that here.

OK, so what you need to do is have the database and each table allow unicode which I'm 100% sure I'd already done. It's quite possible that I did, but emoji's have become a thing since then and they require a specific unicode. In this case UTF8MB4. Not a big deal, just write a SQL script to change the database encoding then iterate through all the tables changing them.

It failed on about the third or fourth table. I've significantly cut down the number of tables post the LEMP conversion, but there are still 51 left. The reason it failed is that the primary key was too big. Apparently the max size of the key is 767 which is 768 bytes. This is not something Neville told me #blameneville. Obviously converting from the latin character set to UTF-8 means that each letter takes up more storage space. I'm guessing 4x the amount given utf8mb4 is a 4 byte UTF-8 encoding. I have some tables that have a 255 character varchar field as the key - apparently this is a bad thing to do anyway as it creates a huge index which I totally get, but didn't .... #blameneville. These 255 character fields are too big to be keys with the new encoding.

Right, so I could have cut the corner here and just converted tables that were going to store emojis rather than all the tables. But if a job's worth doing ... you really should get someone else to do it for you. Next step then, figure out which tables had inappropriate keys. Some of them were pretty easy - I could just assign an auto number as they key and move on. Unfortunately the biggest issue was the email field that I used as a key for the folks that are registered and to link to the race game and playlists (the latter now hidden, but still exists as I moved RagBox to personal use only).

Sidetrack for a minute. I made the email field 255 characters long because an email can be that long. Actually, I think it can be longer. Can you imagine though if your email address was 255 characters long and having to type it in every time you wanted to register or logon with something.

Let's get back on track - data conversion. The fix is to assign each person a unique ID, then to add that field to each of the tables that was linked via the email address. Iterate through the table and populate that new field with the ID associated with each email. Then make that field required and part of the key (as it would have other bits that make up the key such as the competition number). Then delete the email field as it's no longer needed in the referencing tables.

The structure's now in place and we can accept emojis. Only we can't because every page that puts data into the tables is trying to put an email address in an email field that no longer exists and is not populating a required ID field. This is for things like registering with the race game, changing registration, making selections, editing selections, becoming a member, changing your membership, password resets blah, blah, blah. It's more than you think.

Right, so just change those pages and we're up and running. Err, no. The whole logon process that I created recently uses email addresses as the ID and I changed it to the new ID (which has logged everybody out). This bit is actually quite funny. I knew I'd be logged out, but when I tried to log back in, I was locked out because it couldn't validate me for some of the back end stuff that I need to do to run the scripts to make the changes. So I had to un-code the logon, to push the changes, to then re-enable the logon so I could log back in.

We're done now though and the changes are in. I 💗 data conversion. There's something about just figuring out data structures and what needs to be done that I find really calming. I think it's the sense of putting order to something. Parts of the back end are still broken. I need to fix some stuff with the music, but it shouldn't be too difficult. I just need to finish the mobile friendly responsive pages for the front end first. The joy of hobbies.

This UI/UX upgrade I'm doing that leverages Bootstrap, utilizes a base 12 flex box model. This is one of the things my dad has been evangelizing forever. Not the flex box bit, but the base 12 bit rather than base 10. We all know base 10 because it's the number system we use and it's easy as when you multiply something by 10, you stick a zero on it. The challenge comes with division as 10 is not an easy number to divide - it's only divisible by 10,5,2 and 1, whereas 12 is divisible by 12,6,4,3,2 and 1 along with relatively easy manipulation at 8, 9 and 10. 12 is therefore a much easier number if you want to divide it into smaller components. Flex box is quite easy to visualize - imagine that every row has 12 boxes in it. You can then format your layout within that, so if you want 2 columns you make each column 6 boxes wide. If you want 3 columns, you make each column 4 boxes wide etc. You can also do odd sizes like make the first column 2 boxes wide and the other column 10 boxes wide. If you really need it, you can even sub divide each box into 12 more boxes to give you as much flexibility as you need. Hence flex box.

I was surprised when I caught the side of the Alpha Tuari during the pre season practice as I noticed that it had Flex Box on the side of the front wing. Turns out this is a container storage company that offers flexibility in the size of shipping containers.

All of this is great, but it's also the nickname I gave to a gymnast I used to date.

Yeah, so I logged onto the old Google console which is where you go to look at stats on your site and how well it's doing for the all important search engine optimization (SEO), that I couldn't give a shit about. And yeah, it was telling me that I wasn't responsive ... as if I've never heard that before ... as if I didn't know that.

Well, even though I couldn't give a shit about Google rankings because I'm not looking to sell anything on this site - you either find it and like it, or you don't. Matters little to me. But the whole responsiveness thing relates to the user experience of your site and is heavily weighted to retards. I'm sorry, we're not allowed to use that term anymore, the correct term is “iPhone users”. The Google algorithm uses mobile, particularly the iPhone as the benchmark for a site's experience these days.

Well, my site pre-dates the iPhone. Only by a few months, but it was built in an era of desktop browsing. I've updated it several times, but it's been many years since I've looked at the user interface (UI). I will be honest, the last time I updated it, I did look at Apple's site because, well, even though there tech is limiting, they know how to make something look good. Credit where credit's due. At the time, pretty much everyone was doing fixed width pages. Mine have been set to 1,000 pixels (pretty much keeping it just under the 1024 pixel margin that many monitors had at the time). What this enabled you to do was create a fixed base that a browser wouldn't mess (too much) with. You could then position things as you wanted within that fixed base. Accounting for any nuances the plethora of browsers that were around at the time had.

This works and it does work on a mobile device. The problem is that it is wider than most phone screens including that amazing bit of kit, the iPhone, which means that you have to move around a lot to see the content. Additionally, because it's fixed width and the fonts are fixed heights, they're consider small for a mobile device. I have to be honest, I don't disagree with any of this.

What many companies started to do was build parallel sites, so you would have your main website and then a mobile site. I did look into this a few years back and it just didn't make sense to maintain two sites. There are also multiple other problems with this that places like supermarkets ran into where you get different discounts on the mobile site than on the desktop site because the content isn't synced.

The brilliant thing about doing absolutely nothing about this when I last looked at it is that the world has moved on again (as it tends to do). Now, you can build a responsive site that essentially looks at the screen size and then adjusts the content to fit. That means one site, all devices. There are many ways to do this and it would be relatively nasty to get right for complex designs, however, the other brilliant thing about waiting is that folks have made the tools available to plug in and do all the sizing stuff for you. A huge shout out to Bootstrap for making their code available.

So, I'm slowly going through every page on the site again to make it responsive using Bootstrap. The good thing about this change, rather than the move from WAMP to LEMP, is that I can do it page by page as opposed to one big move. You'll slowly see pages change over the coming weeks as we take on a new responsive look.

This is a great question as the website has undergone the biggest change since I started it and nobody can probably tell the difference. The site has always been hosted on a WAMP stack (Windows, Apache, MySQL and PHP) and I've moved it to a LEMP stack (Linux, Nginx, MariaDB, PHP) on Docker. Additionally, I changed the file structure and some of the base coding across the site. It's amazing what you do when you're bored I guess. Off the top of my head, here's the details and impact:

• Docker - previously all the websites and services were mushed together with one Apache server with four virtual hosts connecting to one PHP instance that connects to one MySQL instance. Now, we have an Nginx reverse proxy in a container that connects to four other Nginx servers on their own containers that connect to their own PHP container that in turn connects to its own MariaDB container. This means that each site can be split off and moved very easily if needed
• Windows to Linux is a big change because Linux is case sensitive and Windows is not. My coding and file naming hasn't been consistent, so I've had to correct a lot of links, but this does mean I can port the websites to any hosting environment at anytime with no further modifications if needed.
• Changing the file structure again means re-coding all links. A lot of this was done to be consistent on how media is stored so I can access it from a media server. Previously videos from photos were under photos, videos from games under games (with random sub folders), videos from videos under videos .... you get the point. Now it's all neat and tidy.
• Changing to MariaDB from MySql actually doesn't have an impact as they both use the MySql language and libraries. MariaDB is an open source fort from MySql after the latter was bought by Oracle.
• Upgrading PHP causes deprecated functions to need to be changed. There were a few of these, but the biggest impact was deprecating implicit translation of values or null, specifically, passing non integer or null values to functions that require an integer. Previously it rounded the value or treated null as zero, but it don't do that no more.
• Moving the database calls from MySqli to PDO meant that I had to re-code every part of the site that calls the database. I also added try catch to the call functions to make it a bit neater. This change is more secure and potentially more portable.

On top of that, there were some fixes to some broken pages that had crept into the site, the creation of the 2023 F1 Fantasy Race Game adding a pretty meaningless Chart Generator because I needed it and decided to just make it available.

I also standardized the logon process across the site and created a my account page. There's nothing really there yet, but it's a framework that I can build on. You now need to logon to post to the forums along with playing the Race Game. In doing this, I decided to add encryption to the password file, but you should still not use a password you use elsewhere. I also got rid of all the home page naming. Originally everything was a home page, e.g. photoshomepage or videoshomepage and I've just removed the homepage bit. No reason, other than it made me feel better and if I'm redoing all the links on the site, now's as good a time as any to make those changes.

It's all been a lot of fun as I've been learning a lot of things along the way.

Yes, the good old chain of events where something sounding simple starts and things take on a life of their own. I received an email a while back from Google letting me know that they were no longer going to allow authentication to their mail servers using an ID and password. This impacts the weekly email that is sent from this site and some other functions. Simple, just change to Oauth 2.0 and you're good, right?

In order to use Oauth 2.0 I need to update my mail program (PHPMailer) ...

... in order to use PHPMailer with Google, I need to implement SSL and turn on https with Apache ...

... in order to turn on https with Apache I needed to upgrade my entire stack (WAMP) because somehow the ssl_module on the version I had wasn't working. Assume it wasn't compiled right as there were several posts from folks having a similar problem ...

... upgrading is always fun as there's a bunch of functions that have depricated since I wrote my code, so that means re-writing a lot of the backend code for the site ...

... before installing and rewriting code, need to take a backup ... my NAS fried, so I had to get a new one ...

... ok, so I got a new super duper NAS and it came with loads of bells and whistles, so I did get quite sidetracked with that. It comes with a streaming functionality, so you can stream to it and it will record the stream and then restream out to any number of RTMP services. Right now I have it setup to stream out to Twitch, YouTube and Facebook. There's literally no reason for me to do this other than the fact that I can. And sometimes, that's the best reason of them all :) ...

... sidetracking done and back to the task at hand, backup website, installed new software, install composer to get new PHPMailer, install certbot so I can get SSL certificate from Let's Encrypt which is great as it provides free certificates and free is good, particularly to a project website like this that is not monetized ...

.. infrastructure in place, do some tidy up and recode parts of the website not working. Bit of a note here, there's still some, but it's not related to this. There's a flash plugin used on one page that I need to update when I get time, but where does one get time once you're a megastar streaming to absolutely nobody on three platforms?

All in all, a huge amount of fun as it's been a while since I've done anything with the site and even though visitors won't notice the difference, I do.

I've finally been conned into giving giving up my back door!

Whilst this may at first sound like another post about my illness, it is not. I'm referring to the fact that I've enjoyed the ability to run my own webserver for the 7 years I've been hosting this website, even though my Internet Service Provider (ISP) Comcast has posted for many years that they block port 25 inbound. They do indeed block port 25 outbound, but the inbound has always remained open. At least I've always managed to get inbound port 25 maybe through some backdoor access. Anyway, that has always seemed pretty reasonable to me as the risk of leaving the inbound open would only be to my server and not to them.

A few months back I started getting letters and emails that I needed to upgrade my modem to the latest and greatest in order to get better upload and download speeds. I ignored these to start with as I was in the process of divorce and moving house and it really didn't seem like the most important thing at the time if I'm being honest. Well, after I moved, I thought I'd treat myself to the super duper new faster modem, so I purchased one, hooked it up and a phone call to the ISP later I was up and running. What they didn't tell me was that they flashed the modem with a config file that blocks inbound port 25 as well as outbound. I might be wrong, but I'm pretty sure they're controlling this access with the security policies on the cable modems. I'm sure there's a way to hack this, but to be honest, it seems like that's a path of trying to get yourself banned and more trouble than it's worth.

Google used to offer free email domain hosting with their Google Apps service, but this has been deprecated since December 2012. Microsoft appear to be offering this service with Outlook.com and, if I had more time, I might look into this as it does look like a pretty neat service. The reason I didn't jump on it is because it's a web service they provide and I need to know if I can connect my own interface into the email server to send emails - for sending out things like the weekly email summary. I'm pretty sure you can and it's probably simply running on port 587 or maybe 465, but I got to thinking about how much I actually use email for this website and it ranges somewhere between very little and extremely little. I've set up emailing through gmail for some friends, so that seemed the fastest route to go through as it didn't involve any research. So gone is xx@eastbayrag.com to be replaced with a single email of eastbayrag@gmail.com that will serve as this website's point of contact going forward. Well, I've left my email server up and running behind its Comcast protected firewall so if it ever gets opened or there's a way to have your inbound email route to a port other than 25 I may start it up again, but until then ....

Now, for the nobody who's reading this as I'm sure this is purely for my own benefit, you may be wondering how the weekly emails have been working whilst I've had this issue. Well, the block only stops me receiving email. I can send email by relaying my server off the Comcast one, so outbound works fine. Crazy really.

I doubt anyone’s been wondering about this, but on the off chance that it was something you were interested in, I thought I’d let you know what goes on behind the scenes. The website that you view is less than half of what has been created. For every part of this site, there are some administration tools. Really just a web interface that lets me (or Mr. O) make changes to the data on the site on the fly. As all of the actual data is stored in a database, it’s pretty easy to create a front end that lets you make updates.

If you’ve looked at the photos part of the site, it’s no coincidence that each of the pages look the same. I’ve created a front end tool that allows me to link to a directory, enter the page title and it will upload all the pictures in the specified directory, create thumbnails and the double click pictures, then create the pages that are you get to see. Pretty clever, even if I say so myself.

The big challenge really is that it takes more effort to create the maintenance tools than it does to create the part of the site itself. Take, for example, the forum section that I’ve just created. The site bit itself is quite clever in that it only uses four pages that display different data depending on variables that are passed to each page. On top of that there are another 11 files that allow you to create the entries (categories, sub-categories, thread titles and thread details) and edit the thread details if necessary making a total of 15 files.

There are 22 files in use to support the forums. These allow me to edit or delete any of the entries made. The ability to delete is actually the hardest thing to program here as you need to make it a cascading delete otherwise you would be left with orphaned records, for example, being left with sub-categories that don’t have a category to belong to. In addition to that, I created the ability to move either sub-categories or thread titles – so, if a thread seems to be in the wrong place I can just move it (or if many similar threads get posted under “Other” I can create a new category and move them over). Before you ask, you don’t need to move a category – just editing it has the same effect. There’s no real reason to move a thread detail – it’s unlikely to be posted in the wrong place, but in the unlikely event that it was, it’s probably easier to just cut and paste it into the new position (and by cut and paste I mean create a new entry in the right place with the same text and delete the old record).

So, if you were wondering, now you know.

Well, the time has come for expansion. William's second birthday was coming up so I decided it might be nice to take a little bit of video rather than just taking photos. And with video comes the need for more disk space. Recognizing the need in advance, I rushed out to buy the biggest hard disk I could find for my computer/server so I had somewhere to put the video to download. The picture below shows me with my purchase prior to installation. Proper American style "big"!


So, joking aside, I've seen lots of adverts stating how easy it is to install an internal hard drive. If you can use a screwdriver you're done! Well, that wasn't my experience and I've changed many hard drives before. Admittedly my prior hard disk installations were about 15 years ago in an MS-DOS v5.0 environment so things were probably a little simpler then.

Part of the problem may have been that I ordered my gigantic hard disk on-line and it arrived by itself in the anti-static wrapping, but no box or instructions or anything of any help really. Not a problem, I can figure this out. First, take the computer to bits - easy enough, the Dell I have has a removable side panel that opens with the turn of a knob. OK, so you need to know which drive is going to be the master and which is going to be the slave. Again, easy, the one in there already has the operating system on it so that will be the master and the new one will be the slave. OK so the existing drive is in the wrong bay. Yep, no worries, just take that one out and swap them over, screw them in and ...... oh! As I said, the new drive came on its own in the anti-static wrapping. When I say there was nothing else with it, I really mean nothing else - not even the screws needed to hold the drive in with. Luckily I have an old floppy (the years have not been kind to me) from which I raided the necessary screws. Couple of turns and it's all back together and ready to go.

Now, before I even turned the power on I realized I'd forgotten to set the jumpers on the drives to designate which is the master and which is the primary. I'd plugged them into the right place, just hadn't told them what their role was. Not sure how this popped into my mind - it was a flash back from having previously installed drives as there's nowhere that it tells you that this is a required step (given that it didn't come with any instructions). Being optimistic I turned it on to see if it would work but, as expected, it didn't. So, take it all to bits including removing both drives, set the jumpers correctly (after searching the manufacturers website for the settings) and put it all back together again.

Switch the computer on again and hey presto .... Nothing! OK - restart go into the bios, make a few changes, restart and the bios at least picked up the drive. Now, back in the good old days, once you configured the bios to recognize the drive you had the option for format it from the bios. Well, not anymore, you can still use the DOS fdisk or Windows disk management. Might as well use Windows, that's bound to be easy. In fairness, it is very easy to use. I did, however, notice that it was saying that the disk was only 128Gb when in fact it was 500Gb. I thought I'd format it anyway and then see if it figured it out during the process. Forever the optimist and again wrong. It only recognized the drive as being 128Gb. I restarted, looked in the bios again and that had it as a 500Gb drive. The hardware driver also had it as a 500Gb drive so what was wrong?

After more searching of the world wide web and coming across an article on systems only recognizing drives up to 137Gb unless large drives are enabled in the Windows registry. So, it seems that your system needs to be able to cope with large drives and then you need to tell Windows that you've stuck one in. If you have this problem and end up on this site, you need to enable large block addressing by:

• Open regedit (Start > run > regedit)
• Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Atapi\Parameters
• Right click the word "parameters" in the left window and select "new" then "DWORD value"
• Name it "EnableBigLba"
• Double click it and enter a value of "1"

In fairness, it's not that difficult and would be very easy if there were a step by step guide. I'm pretty sure the newer operating systems don't require you to make the Windows registry edit, but I'm using Windows 2000 so that may have contributed to making this more difficult than it needed to be. That said, a little more involved than just being able to use a screwdriver.

A deviation from the theme of building your own blog as this deals with the technical stuff on the experrymental page and therefore more of an article on how do you make stuff fly round the screen.

The experrymental page came about as somewhere that I was practicing Javascript for the sake of practicing Javascript. Unlike the blog pages or most of the site where the processing is being done server side with PHP, this page is executing the code on the user's PC (which is why you may get messages from your popup blocker, given that these blockers are looking for scripts that are trying to run on the user's machine).

So, the premise for the page was pretty simple, just make available the rubbish that I was practicing with. Seemed pretty straight forward, but as soon as I made it available it seemed to take on a life of its own and ultimately became a project to enable a user to enter text onto the screen and have it rotate round in different ways whilst having the background change color and the EBR logo flying around.

The page is pretty simple - each "bit" has a timer that counts down to zero and when it reaches zero, it does something different, for example the logo will change direction, anything changing color will get a new target color to change to or moving text will have a new destination to move to. Each timer is independent so you don't end up with all changes happen at the same time. Also, whilst researching some of this stuff I found several examples of text being rotated around the mouse so I thought it would be good to include that as well.

Great! Now we have the definition of what's going to be created, just need to code it. Now comes the big challenge. Everything that is coded to be processed server side is easy as I know what is going to do the processing. Unfortunately with client side processing, I have no idea what you (the user) are going to look at this website with - Microsoft Internet Explorer, Netscape Navigator, Mozilla Firefox to name a few examples. And they all seem to process stuff differently which is pretty annoying. That means that a lot of the scripts need to start with finding out what browser the user is using and then insert logic of "if IE, do this, otherwise do that".

For the most part I'm pretty sure the page works OK on all latest version browsers. There are also some "fudges" in there to make it work on some older browsers. Specifically IE6 as that's what I have at work and I noticed it wasn't working properly. The problem I have with IE6 is that it doesn't seem to let me specify the size of a box (div tag). So, if you look simply at the background color effects, there are 36 boxes that change color according the pattern specified. In IE6, nothing was happening as these boxes (div tags) are empty - it seemed to be resizing the boxes according to content. The fudge is that I've just put tables in there and filled them up with non-breaking spaces. Seems to work.

What's all this got to do with PI I hear you ask. Well, bugger all really. PI comes in when calculating the shapes for the text to rotate or be displayed in. The triangle, square and rectangle just use a simple means of keeping track of where each letter is in an array to enable rotation. If we look at the square, it calculates the size of the square (in particular each corner), then each letter is assigned a letter of l, r, u or d for left, right, up or down. When a letter with array d gets equal to or less than the position of the bottom row, it changes direction and becomes an r. Really simple.

Then came the circle (and oval, but it's the same as a circle except the y coordinate is halved). Back to high school math here and something to do with the fact that the circumference of a circle is 2πr and the fact that Sine (sin) and CoSine (cos) allow you to convert the 2π bit into a value of -1 to 1. Without going into all the math, if you divide 2π by the number of letters (lets call this n) you have you can plot a circle by using x = r*Sin((2π/n)*letternumber) and y = r*Cos((2π/n)*letternumber). This is just a fancy way of drawing a circle by using a pen and a piece of string. The string is pinned to the page and then you move the pen round the center to get the circle. All the above equations do is calculate that for you from that center point.

As Homer would say mmmmmmm PI. Rather than go back and correct the triangle, square and rectangle, I left them as they were and modified the circle to create the Pentagon through Dodecagon. Given that we can calculate the circumference of a circle, if I divide the circumference by (say) 10 and join each dot, I would get a decagon. Tada! Nothing more difficult than that to get these shapes. OK, maybe a little. I still use the array associated by each letter to assign a number to each letter to determine what side it is on. By using a number it is flexible to cope with any shape and can also be used in the calculation, so a letter on side 2 should keep moving the specified increment (determined by the speed setting) until it reaches the start of side 3. It would know it's reached its destination by x = r*Sin((2π/n)*(sidenumber +1)) and y = r*Cos((2π/n)*(sidenumber +1))

In theory the formula will take any number of sides. I stopped at 12 as any more and it's difficult to tell the difference between the shape and a circle so it becomes meaningless. One may also conclude that I thought it would be too hairy, but that would be another joke for another time.

Well, after writing that intensive article above that has the potential to actually be helpful to someone, I looked around and thought wow! I'm really creating something here. But it didn't take long for reality to set in.

Those of you that have been following the OBlogs will know that the weather in the UK has been pretty miserable. On the other hand, we are in the middle of the hot California sun which is great unless you are a computer that needs to be kept cool.

Not to worry. Those of you that are worried about the EBR server farm will be happy to know that we've spared no expense to install the latest in air conditioning technology to keep the system cool as you can see from the picture below (which can be double clicked to get an even better view):



The funny thing is, I was thinking about moving the system downstairs where it's cooler, but that would mean connecting a couple of wireless routers. Given the entire system ran at $142.50 it didn't really seem cost effective to shell out for the wireless routers. Whilst downstairs, however, I found this old fan which seems to be doing the job pretty well.