PHP and the DigitalOcean MySQL cluster

One of the weak points in our Rancher 1 cluster has always been the database. Because databases and docker (due to the persistence issues) has always been an issue, we solved that by installing a MySQL server on the same droplet that our Rancher server was running on. While it worked fine, it was always a bit of a weak spot: If that single MySQL server would crash, all our applications would go down. And since we work mostly on customer projects and don't have a dedicated ops person, we wouldn't always been able to immediately respond to such a crisis.

So when DigitalOcean announced their managed MySQL cluster, I was quickly excited about this new project. It took a while for them to launch it publicly and roll it out to "our" region (AMS3), but they did so just over a month ago, so I decided to go for a testdrive.

Creating my first database

Creating a database wasn't all that hard. DigitalOcean has an interface for this. It is literally filling in your database name and clicking a button to set up your database. The same goes for creating a user. This interface is quite limited though: You can not input any configurations (such as the authentication method for users), which can cause issues (as you will later see). But in all its simplicity, it does what it is supposed to do. Create a database. Create a user.

Migrating the database

Migrating the database was actually quite easy. Dump the database from the old MySQL server, import it on the new server. I used MySQL Workbench to do this, and that worked fine.

Mind you: The first application I did this one had a very simple schema. DigitalOcean managed MySQL uses MySQL 8, so in more complex databases I can imagine there might be issues due to backwards compatibility breaks. YMMV.

Configuring my application

I had a very simple issue to solve on my application: My configuration did not allow me to set the port for the DSN, and since DigitalOcean sets a custom MySQL port, this meant I had to make a very simple code change so I could configure the port that was used. A simple one line change solved that issue. No problem so far.

When I deployed a new version of my application with this fix however, I ran into a bigger issue:

Fatal error: Uncaught PDOException: PDO::__construct(): The server requested authentication method unknown to the client [caching_sha2_password] in /var/www/web/index.php

Oops! Yes, MySQL now defaults to a new authentication method, caching_sha2_password. This is not supported by PHP yet.

This is where I come back to my earlier mention of how the simplistic user management of DigitalOcean is causing problems: We can't switch to another authentication method! Fortunately, we can execute a query to actually switch back to the old authentication method. So let's do that:

ALTER USER 'myuser'@'%' IDENTIFIED WITH mysql_native_password BY 's3cr3t_p@ssw0rd';

After doing this, my application worked again. Yay, my application is now running on the new managed MySQL cluster!

About caching_sha2_password

Jaap and I did some digging to figure out why this new MySQL 8 authentication method is not yet supported by PHP. Because really, MySQL is the main database engine used by PHP developers, so why would it not be supported? It turns out, it was! If you use PHP 7.2.9 (at least the docker image for that), you can actually use the new caching_sha2_password method. Your application works fine. However, after that change, a bug was reported after which the support for the new authentication method was reverted. As it looks right now, the changes for the authentication method are back in the master branch, which would mean PHP 8 will support the new authentication method. Until then, we'll have to switch back to mysql_native_password for our PHP applications (or use the PHP 7.2.9 docker image, which I would advice against).

The verdict

Migrating to the new database cluster was easier than I expected and, with one minor work-around, it is easy to get your application up and running with the new DigitalOcean MySQL cluster. We will surely be migrating all application to this new platform. It's up to DigitalOcean to extend their application a bit more, for instance to be able to configure the authentication method for a user or default character set of the database. So far, I'm a happy user of this new DigitalOcean product.

Disclaimer: This is not a sponsored post. I am not affiliated with DigitalOcean other than that I am a happy customer.


Excluding generated files in your project

So here's one of those things that I need to document, or I'll forget. One of those blogposts that I'll find through a search engine in a year ;)

A project I'm working on is using Symfony with Encore, and while running Encore I kept getting some weird errors that javascript couldn't find a file that was actually there. After some debugging I found out that this was related to the fact that the generated file was located in the volume that ensured my local files were available in the container.

The solution was actually pretty simple. Create an unmapped volume inside the mapped volume:

            - ../project:/var/www/html/project:cached
            - /var/www/html/project/web/js

Now when I ran Encore, I had no more errors and everything worked as it should.


The Next Step

Nearly 9 years ago Marjolein and I started Ingewikkeld together. It was mostly a joint freelance business, we weren't planning on having people work for us. My main focus was PHP development, and I wanted to help customers with their PHP-related problems, whether that was architecture, development, training. Anything related to PHP, really.

When we started, I really only focussed on my freelance PHP work, but at some point there was so much work, and I had to say no to so many potential clients, that we hired people. It started out as an interesting idea, but as soon as Jelrik was on the job market, things went quickly. Only 2 months later, Mike joined as well.

Jelrik has since moved on, but Mike (who I've been friends with even before Ingewikkeld was started) stuck around. He's still with Ingewikkeld. And over the years, Mike turned out to really complement my chaotic nature. And this has triggered a change at Ingewikkeld.

Some time ago already we started preparations for several changes to the Ingewikkeld company structure. I'm happy to announce today that we've set the first step, by adding Mike to the Ingewikkeld leadership. The new Ingewikkeld leadership will be:

  • Stefan Koopmanschap: Business director
  • Marjolein van Elteren: Creative director
  • Mike van Riel: Technical Director

I am really happy with this first step. And it's not the last. More things will change in the coming time, to make Ingewikkeld an even more solid business delivering even more quality services.


Sorting select fields in EasyAdminBundle

I'm currently working on an application using Symfony and their EasyAdminBundle. The experience has been great overall, although there are lots of details and specific usecases that are hard to figure out.

For instance when using relations in your entities and creating the related forms. Select fields for related entities are by default sorted by the key (usually the ID of the related entity), however you'd usually want to sort it alphabetically by the name of the entity. My initial thought was to use the @OrderBy annotation, however that only works for the actual OneToMany relations on the other side of the relation, not on the selectbox for the ManyToOne side of the relation. So that was quickly discarded.

Next up I found that you can do it in Symfony by specifying a query_builder parameter to your form configuration. The downside here is that by default, EasyAdminBundle works with a yaml configuration for your form so that makes it a lot harder to do this. I could do this in an extended AdminController, but that would mess with my form field order.

Eventually, however, I found this comment on Github that gave me the solution. Instead of specifying an anonymous function, you can also specify a static method to be called to fetch the values. And so, my solution was now easily implemented.

In my YAML file, I could now specify the query_builder parameter:

- { property: supplier, label: 'Leverancier', type_options: { 'query_builder': 'App\Repository\SupplierRepository::getSuppliersForSelect' } }

In said repository, I added the specified static method:

    static public function getSuppliersForSelect(EntityRepository $entityRepository)
    {
        return $entityRepository
            ->createQueryBuilder('s')
            ->orderBy('s.name', 'ASC');
    }

and now my select field has a nicely alphabetically sorted list of suppliers.


Take care of non-technical skills

Full disclosure: I am one of the founders and current organizers of WeCamp, an event that has a focus on not just technical skills but also personal skills.

In my 20+ years of professional experience in the PHP/software development world, I've worked at many companies and been into many companies as a consultant or freelance developer. Many of the companies I've come in touch with had programs set up for training of their developers. Most of those programs focused on improving technical skills. This makes a lot of sense, because in the current tech world, things change so fast that you need continuous learning to improve. And there is nothing wrong with that.

In recent years, I've seen the focus of training shift a bit from mostly PHP-related subjects to the whole ecosphere of software and tooling around PHP. This is a great shift, because PHP developers don't just write PHP. They use tools like ElasticSearch and memcache, Git and continuous integration, AWS and Azure, and numerous other products that you don't instantly know how to use. Performance, security, quality, it's topics that get more and more attention and rightfully so.

With a few exceptions, however, I've found that many companies still seem to ignore another important part: personal development. I'm talking about things like communication skills, planning skills, a focus on personal happiness. About knowing where you want to go in your life and what to focus on. The human side of the developer. Because, despite what many recruiters would like you to think, a developer is more than just a resource. Developers are just like humans.

I've heard managers complain about developers not having good communication skills, but I've hardly ever seen those same managers look for ways to improve those skills for their developers. I've heard managers complain about the lack of planning skills, or the fact that their developers have a hard time structuring their work day, but I've often seen those same managers only consider technical training for those same developers. And yet, the first non-sponsored link when I search the web for planning skills training is an effective planning skills training. Same for searching for communication skills training. The first result is a learning tree training. And that's just the first results. Go down the results and you'll find a lot more.

One way to focus on more than just technology

As mentioned in my full disclosure at the start, I am one of the founders and current organizers of WeCamp, a 5-day event focussing on improving both technical and non-technical skills that are essential to software development. We've received a lot of positive feedback on the key take-aways of the event being more than just technical skills. I am very proud of that. When we get feedback such as:

To developers, I'd say that the experience is unrivalled by anything in the market today. The coach's focus on your personal development is guaranteed to push you on exactly the points that need improving.

this means we've done our job. We push people to reflect their current position and where they're heading. We push them to evaluate if their current heading is what they really want. But we also help them set goals and achieve those goals. Whether this is about new tech they want to learn or non-tech skills they want to improve. Actually, when we asked what was the best thing about WeCamp 2017 in the evaluation questionnaire, one of the attendees responded with:

The blend of technical and personal development.

In that same questionnaire, when asked about why people would recommend WeCamp, we got things like:

Great learning and life experience and pushes you to get out of your comfort zone in a positive way.

I know I am biased because I'm very much involved in this event, but I really believe that by creating the safe space that we create for people to reflect on their life and career and by getting developers our of their comfort zone, we add a value that not many other events could.

Interested?

If you or your developers are interested in WeCamp, please check out our website. If you have any questions, please do feel free to contact me.


WordPress and HTTPS-terminating proxies

A blog I am writing for was looking for a new place to host their website. Since we have a nice cluster with Rancher up and running, I offered to host the site. It's WordPress, so PHP, so how hard could it be, right?

I spent quite a few hours migrating everything. The initial migration to Docker was not that hard. There is a great official WordPress image for Docker, which makes it extremely easy to set up a new WordPress site in Docker.

The next thing is handling file uploads. Using the do-spaces-sync plugin this was easily set up to use DigitalOcean Spaces. It took a while to upload all images from the old wp-content/uploads to Spaces, but once that was done, I had it working immediately after setting it up. So far, this whole migration was a breeze.

Until I flipped the switch on the DNS and pointed it to our new hosting. I immediately got caught in an infinite redirect loop, and I had no idea why. I've spent hours turning off plugins, turning them on again. Debugging everything, watching logs. I could not figure it out. In the headers I did find a header saying that the redirect came from WordPress:

X-Redirect-By: WordPress

Eventually, I tried explaining the problem in the #wordpress channel in the PHPNL slack and as I'm typing my explanation something dawns on me...

Our Rancher setup has a load balancer that terminates the HTTPS then forwards an internal request to the container using http. But in WordPress, I have configured the siteurl to be https://. So WordPress gets a request using http, figures out it should be using https, and redirects. This causes the infinite redirect loop!

Of course, I wasn't the first to encounter this problem. Once I know what the problem was, searching the Internet quickly gave me the solution. In Wordpress Codex of course. The only thing I needed to do was add a single line to my .htaccess file:

SetEnvIf X-Forwarded-Proto https HTTPS

Once I did that, rebuilt my containers and deployed them to Rancher, the problem was solved. All of a sudden, everything worked.


New domain

I've had the domain leftontheweb.com for ages. It's been with me since 2004. However, since I recently got a brand new .dev domain, I decided it was time for a change. Since I can't even remember how I came up with the old name, it's time for a change. A new name that is easy to recognize, easy to remember and easy to link to me.

The new domain name for this blog is:

skoop.dev

It only makes sense to switch to this domain. Skoop has been my nickname for as long as I have access to the Internet. And since my main occupation is still development, this switch makes sense.

Now, to find interesting topics to blog about again...


Some changes for me

For the past years a lot of my focus has been on the (PHP) community. I've spoken at numerous conferences and usergroups. And although I've been cutting down on the amount of conferences, I've done more usergroups in the past year than in the years before that.

In December 2018, I've made a decision to cut down on this a bit more. This has nothing to do with not wanting to speak anymore, but more with an opportunity that has arisen that I want to take. I want to put 110% of my effort into this, which means I have to cut down on other activities that I'm doing. Speaking at usergroups and conferences is one of those things.

PHP has been my biggest hobby for the past 20+ years. It is great that I have been able to make it my job as well. Since quite a few years, I've picked up on something I've been interested in for years. I've started doing live radio. My first radio show was on the now discontinued Internet radiostation On Air Radio, after which I've moved on to another Internet radiostation IndieXL. Both times I did everything in my own little radio studio that I had built at home. It was a lot of fun.

My interest in radio already began when I was a teen. A Dutch morning show was also broadcasting on TV, so I was "watching radio" every morning. In the 90's, the Dutch radiostation KinkFM introduced me to an incredible amount of alternative music. KinkFM was the best radiostation I could imagine in terms of music, but also in terms of DJ's. People with an incredible passion for and knowledge of music. When the station was stopped by its owner in 2011, I was incredibly sad.

2 years ago one of the original founders of KinkFM saved the brand name from the company that at that time owned the name. While he wasn't planning to restart the station, the response he got was overwhelming, so he started researching his options. I got in touch and over a year ago I started doing a Spotify playlist for them called KLUB KINK.

Late last year, the announcement came: A new radiostation focussing on alternative music will be launched. Since FM is something nearly of the past, the name will now be KINK.

I have been asked to evolve my Spotify playlist into a podcast, and next to that, present a radioshow. After giving it some thought and looking at my schedule, I have decided to take this opportunity. I love doing radio, and to be able to do it for my all-time favorite radiostation is amazing. Starting on Thursday February 7, I will be doing a radioshow every Thursday from 7PM to 9PM.

Will I be completely gone from conferences and usergroups? Of course not! But as I mentioed earlier, I really want this to succeed, I want to give it 110% of my effort, and that means making tough choices.


Git hooks on Windows

I recently was asked to add a git hook to our main repository to add the Jira issue number to the commit message in an automated way. We were so far handling this really inconsistently, with many commits not having the ticket number at all, while others would have them either at the start or the end of the commit message. Since our branches all contain the ticket number (naming is like `feature/APP-123-this-new-feature) this should be easy, right?

Searching around I found that Andy Carter has already published a hook written in Python to do just this. I copied the script and put it in the prepare-commit-msg file. Because I work on Windows these days I expected #!/usr/bin/env python not to work so I updated it to be #!C:\Program Files\Python\Python.exe.

I started testing my new hook, but it wouldn't work. I would constantly run into an error when committing: error: cannot spawn .git/hooks/prepare-commit-msg: No such file or directory.

After trying many different things including adding quotes to the shebang, just using python.exe (since Python was added to the PATH) and a few other things, I found out that I had simply been too excited to change the script. It turns out Windows does support #!/usr/bin/env python. So after simply changing the shebang back to it's original value the commit hook worked like a charm!


Introducing By The Campfire

It's been over a year that I first decided to start a podcast. Inspired by the Dutch podcast Wilde Haren De Podcast in which the host Vincent Patty has interesting conversations with his guests which offer a lot of information about the person as well as the topics they're interested in, I decided I wanted to do something similar with guests from my general area of interest. I first started working on a website, made a list of people I'd like to have on as a guest, and then... procrastinated for way too long.

A few weeks ago I finally got off my butt and made a serious attempt at scheduling some dates. And with pride I can say that the first episode is now published! In it, I talk with the awesome Rick Kuipers on a variety of subjects ranging from Fortnite to dancing, from speaking at conferences to freelancing and from chess to self-steering teams.

Everyone, I'd like to introduce By The Campfire. Because the best conversations are had in situations where there is very little distraction, like when you're sitting by the campfire. And yes, this idea is totally inspired by the campfire talks I've had at WeCamp over the years.

I'll be publishing the podcast on an irregular basis (depending on when I can schedule to meet with someone interested in being a guest on the podcast). You can subscribe to the podcast in several services already, although I am still awaiting approval from Apple and Spotify. I've got a list of all places where we've been approved on the About page. And of course, you can simply listen on the website.

Some thank yous are required. For instance to Stephan Hochdoerfer and the Ingewikkeld crew, who helped with finding the right name for the podcast. Also to someone I can't remember who exactly who came up with the idea of asking my guest to come up with a title for the episode, which results in a title such as How Fortnite Dances Can Help Your Speaking Career. To my wife Marjolein who graciously let me borrow her Zoom H4n recorder. And of course to Rick for being my first guest.