Better Raspberry Pi audio: the JustBoom DAC HAT

I decided that the sound output from the Pi’s built-in headphone jack wasn’t sufficient after all and so went searching for better options (a DAC—digital-to-analog converter).

The Raspberry Pi foundation created a specification called “HAT” (Hardware Attached on Top) a few years ago which specifies a standard way for devices to automatically identify and configure a device and drivers that’s attached to the Pi via its GPIO (General Purpose Input/Output) pins. There’s a number of DACs now that conform to this standard, and the one I settled on is the JustBoom DAC HAT. It’s a UK company but you can buy them locally from Logicware (with $5 overnight shipping no less).

The setup is incredibly simple: connect the plastic mounting plugs and attach the DAC to the Pi, then start it up and edit /boot/config.txt to comment out the default audio setting:

#dtparam=audio=on

Then add three new lines in:

dtparam=audio=off
dtoverlay=i2s-mmap
dtoverlay=justboom-dac

Then reboot. (If you’re running Raspbian Stretch or newer, the i2s-mmap line should not be added).

To say that I’m impressed would be an understatement! I didn’t realise just how crappy the audio from the Pi’s built-in headphone jack was until I’d hooked up the new DAC and blasted some music out. I’m not an audiophile and it’s hard to articulate, but I’d compare it most closely to listening to really low-quality MP3s on cheap earbuds versus high-quality MP3s on a proper set of headphones.

If you’re going to be hooking your Pi into a good stereo system, I can’t recommend JustBoom’s DAC HAT enough!

Raspberry Pi project: AirPlay receiver

I bought a Raspberry Pi almost exactly a year ago, intending on eventually replacing my Ninja Block and its sometimes-unreliable wireless sensors with hardwired ones (apart from the batteries needing occasional changing, there’s something that interferes with the signal on occasion and I just stop receiving updates from the sensor outside for several hours at a time, and then suddenly it starts working again). To do that, I need to physically run a cable from outside under the pergola to inside where the Raspberry Pi will live and I don’t really want to go drilling holes through the house willy-nilly. I want to eventually get the electrician in to do some recabling so I’m going to get him to do that as well, but until then the Pi was just sitting there collecting dust. I figured I should find something useful to do it with, but having a Linode meant that any sort of generic “Have a Linux box handy to run some sort of server on” itch was already well-scratched.

I did a bit of Googling, and discovered Shairport Sync! It lets you use the Raspberry Pi as an AirPlay receiver to stream music to from iTunes or iOS devices, a la an Apple TV or AirPort Express. We already have an Apple TV but it’s plugged into the HDMI port on the Xbox One which means that to simply stream audio to the stereo we have to have the Xbox One, TV, and Apple TV all turned on (the Apple TV is plugged into the Xbox’s HDMI input so we can say “Xbox, on” and the Xbox turns itself on as well as the TV and amplifier, then “Xbox, watch TV” and it goes to the Apple TV; it works very nicely but is a bit of overkill when all you want to do is listen to music in the lounge room).

Installing Shairport Sync was quite straightforward, I pretty much just followed the instructions in the readme there then connected a 3.5mm to RCA cable from the headphone jack on the Raspberry Pi to the RCA input on the stereo. It’s mentioned in the readme, but this issue contains details on how to use a newer audio driver for the Pi that significantly improves the audio output quality.

The only stumbling block I ran into was the audio output being extremely quiet. Configuring audio in Linux is still an awful mess, but after a whole lot of googling I discovered the “aslamixer” tool (thanks to this blog post), which gives a “graphical” interface for setting the sound volume, and it turned out the output volume was only at 40%! I cranked it up to 100% and while it’s still a bit quieter than what the Apple TV outputs, it doesn’t need a large bump on the volume dial to fix—there’s apparently no amplifier or anything on the Raspberry Pi, it’s straight line-level output. The quality isn’t quite as good as going via the Apple TV, but it gets the job done! I might eventually get a USB DAC or amplifier but this works fine for the time being.

On macOS it’s possible to set the system audio output to an AirPlay device, so you can be watching a video but outputting the audio to AirPlay, and the system keeps the video and audio properly in sync. It works extremely well, but the problem we found with having the Apple TV hooked up to the Xbox One’s HDMI input is that there’s a small amount of lag from the connection. When the audio and video are both coming from the Apple TV there’s no problem, but watching video on a laptop while outputting the sound to the Apple TV meant that the audio was just slightly out of sync from the video. Having the Raspberry Pi as the AirPlay receiver solves that problem too!

UPDATE: Two further additions to this post. Firstly, and most importantly, make sure you have a 5-volt, 2.5-amp power supply for the Raspberry Pi. I’ve been running it off a spare iPhone charger which is 5V but only 1A, and the Pi will randomly reboot under load because it can’t draw enough power from the power supply.

Secondly, the volume changes done with the “alsamixer” tool are not saved between reboots. Once you’ve set the volume to your preferred level, you need to run “sudo alsactl store” to persist it (this was actually mentioned in the blog post I linked to above, but I managed to miss it).

New job!

No, I haven’t left Atlassian, but come Monday I’m starting in a new role!

We have the concept of a role called a “Shield” that’s essentially support but for helping our own internal developers and users (as opposed to external customers), and the idea is that it’s the first point of contact for developers who need help or have questions about the particular service/platform/whatever that the Shield is supporting, as well as being able to step back and look at the bigger picture in terms of pain points that those developers run into and what sort of things could be done to minimise that. The name “shield” comes from the fact that you’re essentially shielding the rest of the developers on the team from the distractions that come from other people constantly contacting them throughout the day, and letting them get on with what they do best (actual coding and improvements to the product). I’ll also have the opportunity to actually do some coding and make improvements too, though. \o/

The team I’m joining runs our internal microservices platform that an increasing number of our applications are being run on, and though the microservices themselves can be pretty much any language you’d like (it’s all Docker-based), the code that the platform itself runs on is Node.js which ties in rather nicely with all my learning over it over the past almost eighteen months.

I’ve been doing external customer-facing support for over thirteen and a half years so this is going to be a lovely change! It’s going to be really weird starting anew where I know next to nothing about the inner workings of the thing I’m supporting though. 😛 I expect my brain is going to be dribbling out my ears come the end of next week.

Back to Tasmania

We went back to Tasmania again last week, and it was pretty great!

Where last time we stayed in Hobart for the whole trip, this time we drove up to Bicheno first, which is about a two and a half hour drive north of Hobart. The accomodation itself (the “Diamond Island Retreat”) was not great, the house was built in probably the 1970s and had clearly had next to nothing done with it since. The kitchen was terrible and the two frying pans were both quite burnt and scratched up, and there was zero internet access (at least in terms of wifi, thankfully there was plenty of 4G reception). It was completely clean and tidy, at least.

That being said, the location was amazing. This was the view from the back deck –

Diamond Island

You could walk down the paddock and down to the beach, which had some of the whitest sand I’ve seen. The first sunset was pretty epic as well.

Sunset

Reflections

Walking

More patterns

They do “penguin tours” right near where we were staying, there’s a whole section of land that’s restricted to the public and they get lots of penguins living and breeding there. We went during the decidedly off-season and so only saw a couple of penguins, but one of them waddled its way up the beach and right past us to its burrow! The other penguins we could only see in the distance down on the rocks near the beach. The tours are done after the sun has set and the guides have special torches that emit really yellow light so as not to hurt the penguins’ eyes. During the breeding season you can apparently see upwards of a hundred penguins all coming ashore to feed their chicks.

There’s a few other things to do around Bicheno as well, one is Freycinet National Park which has some epic hiking trails through it (neither Kristina nor I are hikers so we opted to just go by what we could reach by car).

Untitled

Untitled

Cape Tourville Lighthouse

Then right up the road from where we were staying is Douglas-Apsley National Park, which is the same deal as Freycinet with the hiking, and requires a good couple of kilometres of dirt road to get to the carpark.

Untitled

Untitled

Untitled

Untitled

Untitled

(There’s a few more photos from each in the photoset).

The last bit of Bicheno we saw was East Coast Natureworld, a big wildlife sanctuary and conservation area.

Esther the wombat

Emu

Lazing

Ostrich

Don’t ask me why there was an ostrich there, I don’t know. 😛 The baby wombat at the top is named Esther, and she was just sitting there in the keeper’s arms dozing while the keeper was talking. They also do conservation and breeding for Tasmanian devils there, and we got to see one of them being fed which was pretty neat!

Feeding Dennis the Tasmanian Devil

Nom nom nom

After that, we drove back down to Hobart and spent the rest of the trip just wandering around some more.

The side path

190

Up the hill

Mirror selfie

Lit from below

Aurora Australis

Grafitti

Happy doggo grafitti

Under construction

Docked

The huge orange ship is an icebreaker

Aurora Australis is an Australian icebreaker. Built by Carrington Slipways and launched in 1989, the vessel is owned by P&O Maritime Services, but is regularly chartered by the Australian Antarctic Division (AAD) for research cruises in Antarctic waters and to support Australian bases in Antarctica.

And it’s quite an impressive sight in person!

We also went up to Mount Nelson, which is the next-highest mountain in Hobart, but unfortunately they were doing hazard-reduction burns (basically controlled bushfires) so it was really smokey and you mostly couldn’t see anything. 🙁

Despite that, all in all it was an excellent trip.

iPhoneography

Both Kristina and I upgraded to the iPhone 7 last month, I’d heard the camera was good but I took it out for a spin last week when I went on a lunchtime photo walk with some co-workers, and man. I can see why the point-and-shoot market is dying! I processed all these photos in Lightroom on my iMac so it wasn’t solely done on the iPhone, but even so, I’m incredibly impressed.

It’s not going to replace a full DSLR setup in low-light or shallow depth-of-field situations, but where I’d be wandering around during the day taking photos at f/8 anyway…

Untitled

Untitled

Untitled

Untitled

Untitled

Untitled

Untitled

Untitled

Untitled

A year of Node.js

Today marks one year exactly since switching my website from Perl to Javascript/Node.js! I posted back in March about having made the switch, but at that point my “production” website was still running on Perl. I switched over full-time to Node.js shortly after that post.

From the very first commit to the latest one:

$ git diff --stat 030430d 6b7c737
[...]
177 files changed, 11313 insertions(+), 2110 deletions(-)

Looking back on it, I’ve learnt a hell of a lot in that one single year! I have—

  • Written a HipChat add-on that hooks into my Ninja Block data (note the temperature in the right-hand column as well as the slash-commands; the button in the right-hand column can be clicked on to view the indoor and outdoor temperatures and the extremes for the day)
  • Refactored almost all of the code into a significantly more functional style, which has the bonus of making it a hell of a lot easier to read
  • Moved from callbacks to Promises, which also massively simplified things (see the progression of part of my Flickr– and HipChat-related code)
  • Completely overhauled my database schema to accomodate the day I eventually replace my Ninja Block with my Raspberry Pi (the Ninja Block is still running though, so I needed to have a “translation layer” to take the data in the format that the Ninja Block sends and converts it to what can be inserted in the new database structure)
  • Added secure, signed, HTTP-only cookies when changing site settings
  • Included functionality to replace my old Twitter image hosting script, and also added a nice front-end to it to browse through old images

Along with all that, I’ve been reading a lot of software engineering books, which have helped a great deal with the refactoring I mentioned above (there was a lot of “Oh god, this code is actually quite awful” after going through with a fresh eye having read some of these books)—Clean Code by Robert C. Martin, Code Complete by Steve McConnell, The Art of Readable Code by Dustin Boswell and Trevor Foucher.

I have a nice backlog in JIRA of new things I want to do in future, so I’m very interested to revisit this in another year and see what’s changed!

Farewell Dreamhost

After 12 years of service, I’m shutting my Dreamhost account down (for those unaware, Dreamhost is a website and email hosting service).

My very first—extremely shitty—websites were hosted on whichever ISP we happened to be using at the time—Spin.net.au, Ozemail, Optus—with an extremely professional-looking URL along the lines of domain.com.au/~username. I registered virtualwolf.org at some point around 2001-2002 and had it hosted for free on a friend’s server for a few years, but in 2005 he shut it down so I had to go find some proper hosting, and that hosting was Dreamhost.

The biggest thing I found useful as I was dabbling in programming was that Dreamhost offered PHP and MySQL, so I was able to create dynamic sites rather than just static HTML. Of course, looking back at the code now is horrifying, especially the amount of SQL injection vulnerabilities I had peppered my sites with.

Around the start of 2011, I started using source control—Subversion initially—and finally had a proper historical record of my code. I used PHP for the first year or so of it, then ended up outgrowing that and switched to a Perl web framework called Mojolicious. The only option to run a long-lived process on Dreamhost is to use Fast-CGI, which I never managed to get working with Mojolicious, but fortunately Mojolicious could also run as a regular CGI script so I was still able to use it with Dreamhost, albeit not at great speed.

At the same time I started using Subversion, I also signed up with Linode who offer an entire Linux virtual machine with which you can do almost anything you’d like as you have full root access. I originally used it mostly to run JIRA so I could keep track of what I wanted to do with my website and have the nifty Subversion/JIRA integration working to see my commits against each JIRA issue. I slowly started using the Linode for more and more things (and switched to Git instead of Subversion as well), until in 2014 I moved my entire website hosting over to the Linode.

At that point the only thing I was using Dreamhost for was hosting Kristina’s website and WordPress blog, and the email for our respective domains. Dreamhost’s email hosting wasn’t always the most reliable and towards the end of 2015 they had more than their usual share of problems, so we started looking for alternatives. Kristina ended up moving to Gmail and I went with FastMail (who I am extremely happy with and would very highly recommend!), I moved her blog and my previously-LiveJournal-but-now-Wordpress-blog over to the Linode, and that was that!

Moving my website hosting to the Linode also allowed me to move over to Node.js and I’ve been going full steam ahead ever since. Since that posted I’ve moved over from callbacks to Promises (so much nicer), I wrote myself a HipChat add-on to keep an eye on the temperature that my Ninja Block is reporting, and I moved my dodgy Twitter image upload Perl script functionality into my site and added a nice front-end to it. Even looking back at my code from 6 months ago to now shows a marked increase in quality and readability.

So in summary, thanks for everything Dreamhost, but I outgrew you. 🙂

Final enkitchening, and concert

We had the tiler out this morning to put the backsplash in, and the kitchen is basically finished now. The bits above the cupboards still need painting, but in terms of every day use, it’s done, and looks brilliant! The plumbing and electrics were all hooked up yesterday, and you have no idea how delightful it is to actually have a dishwasher now after three-and-a-half years of not having one.

We’ve not used the new induction cooktop properly in anger yet, but we made pasta last night, I put the “Boost” mode on, and it took the water from cold to a roiling boil in five minutes flat. I also seasoned the stainless steel frying pans that we bought from IKEA and had written off as being useless because they stick (apparently seasoning is essential) though they’ve not been put to the test yet. The only casualty in the change to induction is the wok, which despite having a fridge magnet stick to the bottom of it, isn’t sensed by the cooktop. Kristina did a bunch of reading and the best type to get is a carbon steel one, so we’ve got one on the way.

Backsplash 1

Backsplash 2


I also saw Lacuna Coil live last night, for the first time in seven years (due to them not having toured here in that long), and they absolutely ruled. They played all the songs I was hoping for. ? I saw them back in 2007 as well, and was a fair bit closer then but with a significantly crappier phone camera (and also a far far higher tolerance for shitty photos, evidently). 😛 Irritatingly, I had to leave half-way through the last song or I was going to miss the last train home due to there being trackwork this weekend. I’m currently running on about five hours of sleep because I didn’t get home until just before 1:30am, and woke up at 6:40am because the tiler was coming. Note to self, don’t schedule things early on a Saturday morning in future.

Lacuna Coil 1

Lacuna Coil 2

Lacuna Coil 3

Lacuna Coil 4

Lacuna Coil 5

The Enkitchening, Part 3

The benchtops were installed today, and the sink and cooktop put into place, though neither were hooked up.

You guys, it’s starting to look like we have a real kitchen now!

Benchtops 1

Benchtops 2

The cooktop was about the same size as our previous one, perhaps slightly narrower due to not needing the control knobs on the side, but it looks hilariously smaller since the benches are so much larger.

Partial enkitchening

The kitchen people were finished with the cabinetry today so we’re back at home!

And OH MY GOD, they have done an unbelievably good job. The kitchen looks amazing and they haven’t even put the benchtops on yet, nor done any of the tiling for the splashback (the blue and purple stripes in the picture are tape that they haven’t taken off yet).

Partial kitchen 2

Partial kitchen 1

It really shows up how shitty the old kitchen was (compare how this unfinished kitchen looks to the old one in my last post), and how utterly wasteful with space it was. There’s a bit less floor space than the old one, but so much more storage!

Kitchen and kitchenless

We’re getting a new kitchen, and today the tradies came around to rip the old one out. Naturally we had to take some before and after photos (well, I suppose more “during” since the “after” will be the finished new kitchen).

The original kitchen seems to have been done as cheaply as possible. Nothing quite fit properly and there were several places where there was just a piece of particle board to fill a gap between a cupboard and the wall. The oven is new because the glass door on the old one (a Bunnings “Homebrand”-equivalent that was here when we bought the place) exploded all over the floor and Kristina last year. The fridge is also staying because there’s absolutely nothing wrong with it. I bought it over eight years ago and it’s never given us the slightest bit of trouble.

Before 1

Before 2

And then there’s now the total lack of kitchen! The space actually feels smaller without the kitchen in it.

No kitchen 1

No kitchen 2

Everything except the benchtops are being put in next week, as the they have to wait on the carpentry to be done so they can be properly measured and ordered.

Photos

On a lark I did a Google search for “virtualwolf”, and once I got past the first handful of pages (links to main things like my Twitter, Flickr, etc.) I was fascinated to discover just how many of my photos were being used in news articles and the like. I’ve always obsessively tagged my photos so they’re pretty easy to search, but a couple of years ago I went and changed the licensing on most of them to the Creative Commons “ShareAlike” license. The thought process was mostly, I’m never going to earn any money from my photography and have no desire to anyway, so why not! Flickr has the ability to search for images with specific licenses, so I suspect that contributed a lot as well.

There’s quite the range of subjects—

It’s pretty gratifying, really.

Beanie’s dog training

We went to a doggy training class today, on the suggestion of the dog trainer we had over a couple of times. She thought that he would do fine whereas we thought he’d just melt down as soon as he saw the other dogs and we’d immediately have to leave.

We’ve never been happier to be wrong! He was initially extremely barky when the first dog arrived but as soon as he got a good butt-sniff in he settled right down. It was quite incredible! I took a few happy-snaps on my iPhone, of course. At the end the next class arrived, which was a puppy class, and Beanie was happily running around and snuffling them as well.

The actual “training” part didn’t go quite so well, mostly because he was more interested in running around with the other dogs than paying attention to us, but still!

beanie1

beanie2

beanie3

beanie4

beanie5

beanie6

Pictionary Against Humanity

We had some friends over for a barbecue and some games last night and invented something awesome*.

We’d recently bought our own set of Pictionary so were playing that with slightly modified rules whereby one person would be drawing and everyone else was guessing what it was (instead of playing in teams, mostly because we initially didn’t have a useful number of people for teams). Someone mentioned Cards Against Humanity, and Kristina and Millsy both simultaneously had the idea of PICTIONARY AGAINST HUMANITY. Instead of playing a black question card as in Cards Against Humanity and everyone putting down the white cards for answers, we’d have our own individual pile of white cards and would choose one and have to draw it and everyone else guessed the card we were drawing.

It’s seriously amazing. Our rules are as follows:

  • Each player gets a pile of 20-30 white cards and checks through them all to weed out the cards they couldn’t draw (the really weird abstract ones, etc.), then shuffles the pile face-down in front of them.
  • Going around the table in turns, each player takes a single white card from the top of their personal deck—without showing anyone—and attempts to draw it, and everyone else tries to guess the card (with the exact wording of that card). If the drawing player decides it’s too difficult to draw, they can put it in the discards pile and take another card from the top of their deck.
  • There’s a two-minute time limit for guesses, and if the time runs out without anyone guessing it, the same player picks up another card from their deck and attempts to draw it.
  • The person who correctly guesses the white card that was being drawn takes that card and puts it face-up in front of them, and at the end of the game whoever has the most face-up cards wins.

Same as in CaH, the “end of the game” is whenever you want it to be. The best tip is to make sure everyone has played enough CaH that they’re quite familiar with all the white cards. We have the base game plus the first through sixth expansions but the fifth and sixth ones were relatively recent additions and so there were a number of cards we’d either not seen before or hadn’t seen enough to remember. Those ones were just put straight into the discard pile.

It’s all the hilarity of Cards Against Humanity but with the added bonus of making really shitty stick figure drawings. Give it a try!

* Ok fine, I’m sure someone has thought of this before, but whatevs.

Seven years

The subject has come up on a few occasions recently of how long people have been married for (three years, two years, two months!), and my mind is blown anew each time when I point out that as of today we’ve been married for seven years!

I picture seven years in my head and I think, “Yeah, that’s a pretty good length of time”, but then I realise that’s how long we’ve been married and it just does not even remotely feel like it’s been that long. It’s witchcraft! Before we got married we always used to joke that we had time-warpy powers because we’d be doing something and a bunch of time would pass and it wouldn’t feel like it’d been several hours. The same powers have clearly continued into marriage!

<3

Hunter Valley farm stay

Kristina and I went up to the Hunter Valley for a long weekend last weekend. We stayed at a place called Hunter Hideaway Farm, and it was mostly nice but… a bit odd.

We arrived at about 2pm on Saturday, and there’s a long dirt road leading to the house that winds through fairly dense trees and bush. The lady that runs the place with her husband does ceramic sculptures and had put some of them on the trees and along the side of the dirt road as you start to get closer to the house. I understand the effect she was going for but it ended up just coming off as really creepy… a distinct “Someone who is insane and is going to kill you lives here” sort of vibe. It was even worse coming home after dinner when it was pitch black.

After that weirdness, we had just gotten inside when we heard a dog barking. We looked out the front door and there was a fairly sizeable dog barking loudly and in a very unfriendly manner at us. The husband rushed up and grabbed a hold of the dog’s collar, saying that the dog isn’t friendly and they weren’t expecting us yet (despite check-in being at 2, and why on earth would you have an unfriendly dog on a place that constantly has new people in it?!). After that he hauled the dog off into their house, but all of this was a slightly off-putting start to the vacation. The place had a kitchen in it, but it also had a sign saying that you needed to wash and clean and put away everything or there’d be a charge. I can understand not wanting people to leave the place as a complete mess but the sign was worded very passive-aggressively and really rubbed both of us the wrong way.

The view from the second story was pretty nice though.

View from the second floor

The farm itself is an actual working farm, they breed Angus cows and have some horses and a couple of ponies as well. One of the horses was very derpy, which was great, but the ponies were completely uninterested in people and apparently were also prone to biting.

AWW YEAH!

Ponies!

Neither of us drink wine, so we were mostly interested in the food of the area, but the first night’s dinner was underwhelming. It was at the Royal Oak Hotel and was recommended by the farm stay people but mine just wasn’t hugely flavourful and none of the flavours that were in Kristina’s dinner went together. The sheer distance that everything was from everything else was a bit annoying as well, not that that was entirely unexpected.

Saturday night’s weather was totally clear and a new moon as well, so the view of the stars was incredible; standing outside looking up at the Milky Way is always humbling.

The second day was spent doing some more wandering of the property in the morning (during which time that angry dog was out again and came running at us, but we just stood still and the wife came out and apologised and called him back, saying that he’s normally tied up—so maybe keep him tied up then), going out and buying lots of really nice cheese and chocolates, then relaxing and reading books in the afternoon. There was another, much friendlier, dog there that seems to have been a neighbour’s dog, and she was wandering around with us but mostly just running everywhere at full speed. There’s also an extremely picturesque lake that the cows enjoying sitting in and drinking from.

Running

Drinking

Sunset brought some really nice light and some great photos. The switch out of daylight savings was perfectly timed because we could admire the sunset and then go out to dinner, as opposed to all the gloriousness occurring while we were out.

Grasses

Horses running

Sunset

Sunday night’s dinner was amazing. We went to an Italian restaurant called Lillino’s, and it was one of those meals where just everything was perfect and it needed absolutely nothing—no extra salt, no extra parmesan, nothing. If you’re in the Hunter Valley you really need to go eat there.

It was definitely nice to get away from everything and spend some time relaxing and taking photos, but overall we’re not going to bother going back to the Hunter Valley, it’s not really our thing (and the farm stay was weird).

Home improvement: Curtain Edition

Lounge room with new curtains

We put up new curtains—and a new curtain rod—on the door leading out to the patio today, and it looks so much more cozy in here now (previously the window had on it the same horrible blue/grey straight-out-of-an-office vertical blinds that are still on the front window).

Kristina commented that it’s the first place she’s lived in that feels like an adult house, and it’s true… when renting you’re really just temporarily occupying someone else’s place for a period of time, whereas now we’re actually able to put our own mark on things and do what we want and it’s pretty great.

Stubbing services in other services with Sails.js

With all my Javascript learnings going on, I’ve also been learning about testing it. Most of my website consists of pulling in data from other places—Flickr, Tumblr, Last.fm, and my Ninja Block—and doing something with it, and when testing I don’t want to be making actual HTTP calls to each service (for one thing, Last.fm has a rate limit and it’s very easy to run into that when running a bunch of tests in quick succession which then causes your tests to all fail).

When someone looks at a page containing (say) my photos, the flow looks like this:

Request for page → PhotosController → PhotosService → jsonService → pull data from Flickr’s API

PhotosController is just a very thin wrapper that then talks to the PhotoService which is what calls jsonService to actually fetch the data from Flickr and then subsequently formats it all and sends it back to the controller, to go back to the browser. PhotosService is what needs the most tests due to it doing the most, but as mentioned above I don’t want it to actually make HTTP requests via jsonService. I read a bunch of stuff about mocks and stubs and a Javascript module called Sinon, such but didn’t find one single place that clearly explained how to get all this going when using Sails.js. I figured I’d write up what I did here, both for my future reference and for anyone else who runs into the same problem! This uses Mocha for running the tests and Chai for assertions, plus Sinon for stubbing.

Continue reading “Stubbing services in other services with Sails.js”

And now for something completely different: roast potatoes!

Growing up, we’d frequently go over to my grandma’s house on a Sunday evening for a full roast dinner. Roast beef, roast veggies, gravy, the works. Her roast potatoes were always amazing, they were wonderfully crispy on the outside and I never knew how she did it. Whatever my parents’ method is pales in comparison as they never get them at all crispy.

I made roast potatoes tonight following a recipe from Taste.com.au entitled The Best Roast Potatoes, and my god, they’re not joking. They’re almost identical to how my grandma’s used to come out! If you’re a fan of roast potatoes, you should definitely give that recipe a go.

  1. Preheat oven to 200°C or 180C fan-forced. Peel potatoes and cut into 5cm chunks. Place into a large saucepan. Cover with cold water.
  2. Bring to the boil over medium-high heat. Reduce heat to medium. Simmer for 5-10 minutes or until partially cooked (potatoes should be only just tender when pierced with a skewer).
  3. Drain potatoes well. Return to saucepan over low heat until any remaining water evaporates. Shake saucepan vigorously to rough up surface of potatoes (this will make them crunchy when roasted). Alternatively, scrape surfaces of potatoes with a fork to create a rough texture.
  4. Pour oil into a large roasting pan. Place roasting pan into oven for 5 minutes or until oil is hot. Working quickly, add potatoes to hot oil. Use tongs to turn potatoes to coat in oil, then return roasting pan to oven.
  5. Roast potatoes for 40 minutes. Turn and roast for a further 30 minutes or until golden and crisp. Season with salt. Serve immediately.

Learning new things: Javascript and Node.js

We’ve used Node.js (specifically with a framework called Sails.js) at work for a number of projects but I never really felt I properly understood one of Node’s fundamental concepts, that of the callback. It’s absolutely pervasive throughout Node and I was able to muddle on through at work without totally grasping it, but it wasn’t ideal.

Back at the end of January I decided to try rewriting my website using Node.js (it’s currently written in Perl using the Mojolicious framework) as a learning experience. It’s now almost two months later and my site is actually completely rewritten with Node/Sails (sans tests, which are currently being written; I know about test-driven development but I wasn’t about to start bashing my head against failing to understand how to get the tests to do what I wanted on top of learning a whole new language :P) with all the same functionality of my Perl one, and although I’m still far from an expert I actually feel like I have a proper handle on what’s going on.

The problem I found when trying to find examples was that they were all very contrived; I felt like they were missing fundamental underlying parts that apparently everybody else was able to understand but I couldn’t. For me, the “ah ha” moment was this post on Stack Overflow about using callbacks in your own functions. It didn’t assume anything or use an example of some module that apparently everyone is already familiar with (the most common one was fs.read() to read data from the filesystem). Once I had that straight, it was full steam ahead. It’s also significantly easier to deal with Javascript objects compared to Perl’s array/hash references.

My actual live website at virtualwolf.org is still on the old Perl version, but I don’t want to put the Node one up until I’ve actually got it properly covered with tests. Speaking of tests, I’m using a thing called Istanbul for code coverage, the reports it generates look like this, and it’s really satisfying having the numbers and bars go up as your coverage increases. It’s basically gamification of tests, really!

All in all, I’m pretty pleased!

Comforts

There’s certain things that are comforting by virtue of having done or seen them many times before and thus you know exactly what to expect.

Diablo III is like that for me, specifically the story mode. The original Diablo III had you go through the (very linear) story multiple times, once for each difficulty level, and by the time you hit the maximum level of 60 you’d generally done it at least three or four times. Combine that with the five different classes you can choose, and that’s a lot of repeats. When the Reaper of Souls expansion came out, Blizzard did away with that whole repeat-the-story-many-times in favour of Adventure Mode. It’s varied and certainly less repetitive than the old story mode, but I do enjoy going back every now and again and starting a new character to play through the story from the beginning. It’s definitely comfortable.

Zombies, Run!

A couple of years ago—a couple of months after we moved into our house, actually—I bought C25K, a couch-to-5K training app. It took me longer than I’d have liked, but I eventually got up to the full 5km, though only at 7km/h which for me is barely faster than brisk walking (a comfortable brisk walk for me is 6.5km/h; I know these speeds because I do my running on a treadmill at home). I stopped for a while and decided to start it again but from the beginning at a faster pace for the running sections, but fairly quickly lost interest.

Then the subject came up at work, and a colleague mentioned a thing called Zombies, Run! that combines an actual story with your running. It turned out they also made a couch-to-5K version and it’s really well done. There’s an audio story that plays, almost like an old radio drama, that puts you in the position of a new runner to an outpost that’s survived the zombie apocalypse and they need you to train up in order to be able to complete reconnaissance missions and to go get supplies from other bases and so on. It’s done in the form of periodic radio transmissions to you, but in between you can have your own music playing. The voice actors they’ve used are extremely good; there’s none of the cringe-worthy voice acting you’d often get in the video games of yesteryear. I’ll freely admit that I’ve been fairly slack, I only completed Week 4 yesterday despite having begun the program in May last year, but it is very neat and definitely a more interesting alternative to your bog-standard couch-to-5K program!

Introducing the LiveJournal XML Importer

Continuing on from my previous post about my LiveJournal to WordPress experience, and how the importer managed to miss a bunch of entries, it turns out I didn’t have every notification email still around. The ones prior to February of 2004 I’d apparently deleted so sadly there’s no recovering Kristina’s really early comments from the missing posts, but from what I could see there weren’t too many of those anyway, thankfully.

However, I’m happy to say that I’ve been able to hack the importer to import all the entries and comments from an ljdump archive! I’ve put the code up on GitHub, I’m sure there’s bugs and edge-cases and things that don’t work properly, but it worked perfectly for me. I’ve changed it from the original importer to still import comments from journals that have been deleted so the threading remains intact and you don’t end up with weird comments seemingly replying to nothing. They’re easily identified by the fact that the date on the comment is set to the time you performed the import, so they show up at the top of the Comments section in WordPress’ admin.

Lily’s visit

Lily normally lives with her mother up near Nimbin (which is an almost-nine-hour drive north from our house), but they came down for Christmas this year. We saw Lily last in October of 2014, and she didn’t really like Beanie at that point because he kept trying to lick her, and Beanie himself wasn’t quite sure what to make of her as she was running all around and generally being fairly hyper. This time was a night-and-day difference, Lily was really good with him and knew how to pet him properly, and was significantly calmer than last time. When I would read to Lily before bed, Beanie would come in and curl up on the floor while we’re doing so. The only slight issue was that Beanie kept trying to hump Lily! We’re not entirely sure why (we get the occasional hump attempt, but nothing like this), the only conclusion we came to was that it was because she’s so much smaller than us.

My sister and her husband and two girls (Scarlett, 5, and Arya, 1-and-a-bit) live three hours away in the opposite direction from Nimbin, but they were up for Christmas this year too, which we had at my parents’ place.

New bears

"THAT WAY!"

Reading with Nanna Ruth

We had Lily for a week all up, from Christmas Day to New Year’s Day. Kristina pointed out that it’s so much easier to have a proper conversation with Lily now, she actually listens to what you’re saying rather than just talking at everyone all the time and not really caring if anyone else is listening. There were a few pretty hilarious exchanges, too. My favourites were these two:

Lily: Why are we doing this?
Me: Better safe than sorry.
Lily: I don’t understand.
Me: It means it’s better to do something and perhaps not have needed to, than to not do it and then have regrets later.
Lily (in a very matter-of-fact tone): You are speaking absolute nonsense. I don’t understand a word you’ve just said.

It was a serious struggle not to burst out laughing. The other one was when we were reading before bed, Lily still has some of her older learn-to-read books around, and one of them had a thing on the front:

Lily (reading the cover of the book): “Synthetic phonics.”
Me: Synthetic! That’s a big word!
Lily (again very matter-of-fact): That’s not a big word. “Intergalactic” is a big word.

Lily’s reading level is pretty impressive. Mum bought her The Animal Book, which is a big book about all sorts of different animals and plants, and Lily was just zooming through it, even reading the proper scientific names of things with barely a pause. She’d also had the rite of passage that I’m pretty sure all children go through (me included), which is where a relative buys you a big book of REALLY CORNY jokes. We were inundated with jokes the whole time, but I think the most impressive thing was how well she was remembering all the different jokes.

I’d forgotten that I had Minecraft on the Xbox One, so we had a pretty good chunk of time playing that as well, either Lily by herself or the both of us with splitscreen. She’d only played on her grandma’s iPad previously but she picked up how to control everything with a twin-stick controller really quickly! She was fairly obsessed with it and would tell me even when she wasn’t playing about what she wanted to build next or how she wanted to modify her various houses she’d created.

Scarlett, my niece, was staying with my parents for five days while everyone else went back down to Nowra, so we were able to go over and catch up a couple more times. We went to Featherdale Wildlife Park on Sunday, which mum and dad hadn’t ever been to before (naturally, photos were taken), and on Tuesday we went down to Collaroy Beach to generally chill out (where I put suncream on but managed to burn both kneecaps and the shin on one leg. >_<).

We also realised that we didn’t have any recent photos of all of us, so I set up all my crazy lighting equipment yesterday and took some really quite nice photos of us.

Family photo!

All in all, it was just a really nice time. She’s such a good kid, her manners (both regular and table) are great, and she’s extremely bright. It sucks that we don’t see her more often, but c’est la vie.

Missing history

So it turns out that the LiveJournal to WordPress Importer didn’t actually import everything. I’d been going through and updating links to old entries to point to their relevant entry here in WordPress, and there were several pages that didn’t actually make it across (it also imported every single comment back to around 2010 twice, so I had to go through and delete all of those duplicates; prior to 2010 it was fine, for some weird reason). That wouldn’t have been too bad, but in between my having imported my LiveJournal originally and me discovering this, Kristina’s old LiveJournal account was deleted, which also meant that every single comment of hers on my LiveJournal was now gone, and any fresh import I did directly from the LiveJournal API wouldn’t have them at all. 🙁 Looking back through my old entries was kind of sad, with just “Deleted comment” everywhere in place of Kristina’s actual comments.

I wanted to have the complete history of my LiveJournal here in WordPress, but I also didn’t want to have all of Kristina’s comments missing. I figured SOMETHING had to be able to be done!

I used ljdump to hit LiveJournal’s API and download each entry there into a raw XML file, and that had grabbed all the journal entries, so clearly something had fucked up in the WordPress import part.

The situation was this:

  • I had most but not all of my old LiveJournal entries imported into WordPress
  • Those entries that made it across did have Kristina’s old comments on them
  • I had all of my own entries downloaded to raw XML
  • ljdump also grabbed all the comments for each entry as well (sans Kristina’s, obviously)

I manually went through and compared the entries in WordPress to those on my LiveJournal month-by-month, and found that there were 68 missing ones in total. I hacked at the LiveJournal to WordPress Importer plugin until I was able to get it to read the raw XML files that’d come directly from ljdump, then spun up a new temporary WordPress install and was able to import just those missing entries. Next, I erased that temporary instance, imported the full backup from this blog, then ran the importer again to bring in just those 68 missing entries from XML, and it worked a treat.

Unfortunately there were a handful of those entries that also had had Kristina’s comments on them previously, so they were still missing. Thankfully, me being the digital hoarder that I am, I still had all of the email notifications that LiveJournal had sent me for each and every comment on my journal, and the LiveJournal API actually shows even deleted comments in their properly threaded state, just with no body or detail beyond the username who posted it. So I was able to copy the content and timestamp for each comment of Kristina’s that’d been on those missing entries that weren’t imported, and update the raw comment XML with that detail!

This is still a work in progress and my next step is to hack at the importer further to read the comments directly from XML (currently it’s reading the journal entries from the XML files, but the comments are still pulled from LiveJournal’s API directly). It’ll definitely be do-able, it may just take a little while because everything related to WordPress is in PHP and I’ve not done any PHPing for quite a number of years now!

This may seem a bit odd, but given I have 13 years of history in LiveJournal, and it’s where Kristina and I initially started chatting a lot more before she visited and we got together, I didn’t want to have these weird entries where Kristina was just essentially erased from my blog history.

I might even put my modifications to the plugin up on Bitbucket if it seems to be working well, given the current LiveJournal to WordPress Importer is a bit shit.

Tasmania, and LiveJournal to WordPress

So I’ve finally ditched LiveJournal and migrated it all over to my own WordPress site. I wanted to keep blogging things, but where people might actually read them. 😛

We decided to scale down our ambitions with the five-year anniversary trip, as Kristina is currently contracting and so would be losing out on quite a lot of money, and so went to Tasmania for five days instead. We’d heard great things but had never actually been before.

We stayed in the Hobart Art House, which was fantastically close to everything. We rented a car from Budget, and specified “Suzuki Swift or similar”. A Swift would have been fine, but instead we got a 1.2-litre Nissan Micra. It’s the most gutless car I’ve ever driven, and to get up the hill to where we were staying I had to put my foot all the way down to the firewall, and even then it still struggled.

At the bar

Day 1 was wandering around the Salamanca Markets, burgers for lunch at Jack Greene (their beef and blue cheese burger is amazing), then some more wandering until dinner at Monty’s on Montpelier. Their website says “fine dining” and my god, if anything that’s underselling it. The food there is absolutely incredible.

Cliffs

Day 2 started with a three-hour boat ride around the south-eastern coast with Pennicott Wilderness Journeys. According to the tour guide it was one of the calmest days they’d seen this year, but unfortunately that didn’t help because about half-way through the trip I got horribly sea-sick, though thankfully (barely) not to the point of throwing up, but it meant I was mostly sitting there feeling absolutely miserable and not even being able to look at any of the scenery. Kristina was suffering as well, so in hindsight the whole boat trip was a terrible idea. It redeemed itself with lunch at Port Arthur Lavender (again, incredible food), then dinner at Rockwall. One of their specials was a coffee-rubbed wagyu, and it was definitely the best steak I’ve ever had. It was pretty much falling apart in my mouth, it was so tender.

The other side of the lookout

Day 3, we went to Bruny Island. The “attractions” there were rather over-hyped, but the scenery is absolutely stunning. It’s like a tropical island in places, the clarity of the water is unbelievable. The afternoon was spent walking around the streets around where we were staying and checking out all the old houses, then dinner was at Mures. Mine was quite tasty, but Kristina’s risotto was crunchy, and even after she sent it back and got a new one, it wasn’t brilliant. She got a cold seafood tasting platter afterwards though, which was REALLY good.

Looking down to Hobart

Day 4, the last full day we had, we wandered around the botanical gardens, had lunch at Blue Eye (in keeping with everything else, the food was excellent), then went for a drive up to Mount Wellington. On the drive up, there was someone in a van who was doing 40km/h in a 70km/h zone and kept drifting over entirely into the other lane. It was terrifying! Thankfully there was a solid section of broken line on the road where I could overtake them, and the little Micra managed it surprisingly well. Mount Wellington itself was absolutely incredible, I’ve never been up a mountain where there’s alpine foliage before, and it felt like an entirely different world. Dinner was at Don Camillo, which was quite tasty but not on the level of some of the other restaurants.

On the last day we walked around Salamanca Place again, and had lunch back at Rockwall before heading to the airport. Our own car is only a 1.8-litre 2000 Corolla, but it was pretty funny feeling like it had a whole lot of grunt driving back from Sydney airport after having spent five days driving the Micra around. 😛

All in all, Tasmania was excellent and we’ll definitely be back!