Stubbing services in other services with Sails.js

With all my Javascript learnings going on, I’ve also been learning about testing it. Most of my website consists of pulling in data from other places—Flickr, Tumblr, Last.fm, and my Ninja Block—and doing something with it, and when testing I don’t want to be making actual HTTP calls to each service (for one thing, Last.fm has a rate limit and it’s very easy to run into that when running a bunch of tests in quick succession which then causes your tests to all fail).

When someone looks at a page containing (say) my photos, the flow looks like this:

Request for page → PhotosController → PhotosService → jsonService → pull data from Flickr’s API

PhotosController is just a very thin wrapper that then talks to the PhotoService which is what calls jsonService to actually fetch the data from Flickr and then subsequently formats it all and sends it back to the controller, to go back to the browser. PhotosService is what needs the most tests due to it doing the most, but as mentioned above I don’t want it to actually make HTTP requests via jsonService. I read a bunch of stuff about mocks and stubs and a Javascript module called Sinon, such but didn’t find one single place that clearly explained how to get all this going when using Sails.js. I figured I’d write up what I did here, both for my future reference and for anyone else who runs into the same problem! This uses Mocha for running the tests and Chai for assertions, plus Sinon for stubbing.

Continue reading “Stubbing services in other services with Sails.js”

And now for something completely different: roast potatoes!

Growing up, we’d frequently go over to my grandma’s house on a Sunday evening for a full roast dinner. Roast beef, roast veggies, gravy, the works. Her roast potatoes were always amazing, they were wonderfully crispy on the outside and I never knew how she did it. Whatever my parents’ method is pales in comparison as they never get them at all crispy.

I made roast potatoes tonight following a recipe from Taste.com.au entitled The Best Roast Potatoes, and my god, they’re not joking. They’re almost identical to how my grandma’s used to come out! If you’re a fan of roast potatoes, you should definitely give that recipe a go.

  1. Preheat oven to 200°C or 180C fan-forced. Peel potatoes and cut into 5cm chunks. Place into a large saucepan. Cover with cold water.
  2. Bring to the boil over medium-high heat. Reduce heat to medium. Simmer for 5-10 minutes or until partially cooked (potatoes should be only just tender when pierced with a skewer).
  3. Drain potatoes well. Return to saucepan over low heat until any remaining water evaporates. Shake saucepan vigorously to rough up surface of potatoes (this will make them crunchy when roasted). Alternatively, scrape surfaces of potatoes with a fork to create a rough texture.
  4. Pour oil into a large roasting pan. Place roasting pan into oven for 5 minutes or until oil is hot. Working quickly, add potatoes to hot oil. Use tongs to turn potatoes to coat in oil, then return roasting pan to oven.
  5. Roast potatoes for 40 minutes. Turn and roast for a further 30 minutes or until golden and crisp. Season with salt. Serve immediately.

Learning new things: Javascript and Node.js

We’ve used Node.js (specifically with a framework called Sails.js) at work for a number of projects but I never really felt I properly understood one of Node’s fundamental concepts, that of the callback. It’s absolutely pervasive throughout Node and I was able to muddle on through at work without totally grasping it, but it wasn’t ideal.

Back at the end of January I decided to try rewriting my website using Node.js (it’s currently written in Perl using the Mojolicious framework) as a learning experience. It’s now almost two months later and my site is actually completely rewritten with Node/Sails (sans tests, which are currently being written; I know about test-driven development but I wasn’t about to start bashing my head against failing to understand how to get the tests to do what I wanted on top of learning a whole new language :P) with all the same functionality of my Perl one, and although I’m still far from an expert I actually feel like I have a proper handle on what’s going on.

The problem I found when trying to find examples was that they were all very contrived; I felt like they were missing fundamental underlying parts that apparently everybody else was able to understand but I couldn’t. For me, the “ah ha” moment was this post on Stack Overflow about using callbacks in your own functions. It didn’t assume anything or use an example of some module that apparently everyone is already familiar with (the most common one was fs.read() to read data from the filesystem). Once I had that straight, it was full steam ahead. It’s also significantly easier to deal with Javascript objects compared to Perl’s array/hash references.

My actual live website at virtualwolf.org is still on the old Perl version, but I don’t want to put the Node one up until I’ve actually got it properly covered with tests. Speaking of tests, I’m using a thing called Istanbul for code coverage, the reports it generates look like this, and it’s really satisfying having the numbers and bars go up as your coverage increases. It’s basically gamification of tests, really!

All in all, I’m pretty pleased!

Comforts

There’s certain things that are comforting by virtue of having done or seen them many times before and thus you know exactly what to expect.

Diablo III is like that for me, specifically the story mode. The original Diablo III had you go through the (very linear) story multiple times, once for each difficulty level, and by the time you hit the maximum level of 60 you’d generally done it at least three or four times. Combine that with the five different classes you can choose, and that’s a lot of repeats. When the Reaper of Souls expansion came out, Blizzard did away with that whole repeat-the-story-many-times in favour of Adventure Mode. It’s varied and certainly less repetitive than the old story mode, but I do enjoy going back every now and again and starting a new character to play through the story from the beginning. It’s definitely comfortable.

Zombies, Run!

A couple of years ago—a couple of months after we moved into our house, actually—I bought C25K, a couch-to-5K training app. It took me longer than I’d have liked, but I eventually got up to the full 5km, though only at 7km/h which for me is barely faster than brisk walking (a comfortable brisk walk for me is 6.5km/h; I know these speeds because I do my running on a treadmill at home). I stopped for a while and decided to start it again but from the beginning at a faster pace for the running sections, but fairly quickly lost interest.

Then the subject came up at work, and a colleague mentioned a thing called Zombies, Run! that combines an actual story with your running. It turned out they also made a couch-to-5K version and it’s really well done. There’s an audio story that plays, almost like an old radio drama, that puts you in the position of a new runner to an outpost that’s survived the zombie apocalypse and they need you to train up in order to be able to complete reconnaissance missions and to go get supplies from other bases and so on. It’s done in the form of periodic radio transmissions to you, but in between you can have your own music playing. The voice actors they’ve used are extremely good; there’s none of the cringe-worthy voice acting you’d often get in the video games of yesteryear. I’ll freely admit that I’ve been fairly slack, I only completed Week 4 yesterday despite having begun the program in May last year, but it is very neat and definitely a more interesting alternative to your bog-standard couch-to-5K program!

Introducing the LiveJournal XML Importer

Continuing on from my previous post about my LiveJournal to WordPress experience, and how the importer managed to miss a bunch of entries, it turns out I didn’t have every notification email still around. The ones prior to February of 2004 I’d apparently deleted so sadly there’s no recovering Kristina’s really early comments from the missing posts, but from what I could see there weren’t too many of those anyway, thankfully.

However, I’m happy to say that I’ve been able to hack the importer to import all the entries and comments from an ljdump archive! I’ve put the code up on GitHub, I’m sure there’s bugs and edge-cases and things that don’t work properly, but it worked perfectly for me. I’ve changed it from the original importer to still import comments from journals that have been deleted so the threading remains intact and you don’t end up with weird comments seemingly replying to nothing. They’re easily identified by the fact that the date on the comment is set to the time you performed the import, so they show up at the top of the Comments section in WordPress’ admin.

Lily’s visit

Lily normally lives with her mother up near Nimbin (which is an almost-nine-hour drive north from our house), but they came down for Christmas this year. We saw Lily last in October of 2014, and she didn’t really like Beanie at that point because he kept trying to lick her, and Beanie himself wasn’t quite sure what to make of her as she was running all around and generally being fairly hyper. This time was a night-and-day difference, Lily was really good with him and knew how to pet him properly, and was significantly calmer than last time. When I would read to Lily before bed, Beanie would come in and curl up on the floor while we’re doing so. The only slight issue was that Beanie kept trying to hump Lily! We’re not entirely sure why (we get the occasional hump attempt, but nothing like this), the only conclusion we came to was that it was because she’s so much smaller than us.

My sister and her husband and two girls (Scarlett, 5, and Arya, 1-and-a-bit) live three hours away in the opposite direction from Nimbin, but they were up for Christmas this year too, which we had at my parents’ place.

New bears

"THAT WAY!"

Reading with Nanna Ruth

We had Lily for a week all up, from Christmas Day to New Year’s Day. Kristina pointed out that it’s so much easier to have a proper conversation with Lily now, she actually listens to what you’re saying rather than just talking at everyone all the time and not really caring if anyone else is listening. There were a few pretty hilarious exchanges, too. My favourites were these two:

Lily: Why are we doing this?
Me: Better safe than sorry.
Lily: I don’t understand.
Me: It means it’s better to do something and perhaps not have needed to, than to not do it and then have regrets later.
Lily (in a very matter-of-fact tone): You are speaking absolute nonsense. I don’t understand a word you’ve just said.

It was a serious struggle not to burst out laughing. The other one was when we were reading before bed, Lily still has some of her older learn-to-read books around, and one of them had a thing on the front:

Lily (reading the cover of the book): “Synthetic phonics.”
Me: Synthetic! That’s a big word!
Lily (again very matter-of-fact): That’s not a big word. “Intergalactic” is a big word.

Lily’s reading level is pretty impressive. Mum bought her The Animal Book, which is a big book about all sorts of different animals and plants, and Lily was just zooming through it, even reading the proper scientific names of things with barely a pause. She’d also had the rite of passage that I’m pretty sure all children go through (me included), which is where a relative buys you a big book of REALLY CORNY jokes. We were inundated with jokes the whole time, but I think the most impressive thing was how well she was remembering all the different jokes.

I’d forgotten that I had Minecraft on the Xbox One, so we had a pretty good chunk of time playing that as well, either Lily by herself or the both of us with splitscreen. She’d only played on her grandma’s iPad previously but she picked up how to control everything with a twin-stick controller really quickly! She was fairly obsessed with it and would tell me even when she wasn’t playing about what she wanted to build next or how she wanted to modify her various houses she’d created.

Scarlett, my niece, was staying with my parents for five days while everyone else went back down to Nowra, so we were able to go over and catch up a couple more times. We went to Featherdale Wildlife Park on Sunday, which mum and dad hadn’t ever been to before (naturally, photos were taken), and on Tuesday we went down to Collaroy Beach to generally chill out (where I put suncream on but managed to burn both kneecaps and the shin on one leg. >_<).

We also realised that we didn’t have any recent photos of all of us, so I set up all my crazy lighting equipment yesterday and took some really quite nice photos of us.

Family photo!

All in all, it was just a really nice time. She’s such a good kid, her manners (both regular and table) are great, and she’s extremely bright. It sucks that we don’t see her more often, but c’est la vie.

Missing history

So it turns out that the LiveJournal to WordPress Importer didn’t actually import everything. I’d been going through and updating links to old entries to point to their relevant entry here in WordPress, and there were several pages that didn’t actually make it across (it also imported every single comment back to around 2010 twice, so I had to go through and delete all of those duplicates; prior to 2010 it was fine, for some weird reason). That wouldn’t have been too bad, but in between my having imported my LiveJournal originally and me discovering this, Kristina’s old LiveJournal account was deleted, which also meant that every single comment of hers on my LiveJournal was now gone, and any fresh import I did directly from the LiveJournal API wouldn’t have them at all. 🙁 Looking back through my old entries was kind of sad, with just “Deleted comment” everywhere in place of Kristina’s actual comments.

I wanted to have the complete history of my LiveJournal here in WordPress, but I also didn’t want to have all of Kristina’s comments missing. I figured SOMETHING had to be able to be done!

I used ljdump to hit LiveJournal’s API and download each entry there into a raw XML file, and that had grabbed all the journal entries, so clearly something had fucked up in the WordPress import part.

The situation was this:

  • I had most but not all of my old LiveJournal entries imported into WordPress
  • Those entries that made it across did have Kristina’s old comments on them
  • I had all of my own entries downloaded to raw XML
  • ljdump also grabbed all the comments for each entry as well (sans Kristina’s, obviously)

I manually went through and compared the entries in WordPress to those on my LiveJournal month-by-month, and found that there were 68 missing ones in total. I hacked at the LiveJournal to WordPress Importer plugin until I was able to get it to read the raw XML files that’d come directly from ljdump, then spun up a new temporary WordPress install and was able to import just those missing entries. Next, I erased that temporary instance, imported the full backup from this blog, then ran the importer again to bring in just those 68 missing entries from XML, and it worked a treat.

Unfortunately there were a handful of those entries that also had had Kristina’s comments on them previously, so they were still missing. Thankfully, me being the digital hoarder that I am, I still had all of the email notifications that LiveJournal had sent me for each and every comment on my journal, and the LiveJournal API actually shows even deleted comments in their properly threaded state, just with no body or detail beyond the username who posted it. So I was able to copy the content and timestamp for each comment of Kristina’s that’d been on those missing entries that weren’t imported, and update the raw comment XML with that detail!

This is still a work in progress and my next step is to hack at the importer further to read the comments directly from XML (currently it’s reading the journal entries from the XML files, but the comments are still pulled from LiveJournal’s API directly). It’ll definitely be do-able, it may just take a little while because everything related to WordPress is in PHP and I’ve not done any PHPing for quite a number of years now!

This may seem a bit odd, but given I have 13 years of history in LiveJournal, and it’s where Kristina and I initially started chatting a lot more before she visited and we got together, I didn’t want to have these weird entries where Kristina was just essentially erased from my blog history.

I might even put my modifications to the plugin up on Bitbucket if it seems to be working well, given the current LiveJournal to WordPress Importer is a bit shit.

Tasmania, and LiveJournal to WordPress

So I’ve finally ditched LiveJournal and migrated it all over to my own WordPress site. I wanted to keep blogging things, but where people might actually read them. 😛

We decided to scale down our ambitions with the five-year anniversary trip, as Kristina is currently contracting and so would be losing out on quite a lot of money, and so went to Tasmania for five days instead. We’d heard great things but had never actually been before.

We stayed in the Hobart Art House, which was fantastically close to everything. We rented a car from Budget, and specified “Suzuki Swift or similar”. A Swift would have been fine, but instead we got a 1.2-litre Nissan Micra. It’s the most gutless car I’ve ever driven, and to get up the hill to where we were staying I had to put my foot all the way down to the firewall, and even then it still struggled.

At the bar

Day 1 was wandering around the Salamanca Markets, burgers for lunch at Jack Greene (their beef and blue cheese burger is amazing), then some more wandering until dinner at Monty’s on Montpelier. Their website says “fine dining” and my god, if anything that’s underselling it. The food there is absolutely incredible.

Cliffs

Day 2 started with a three-hour boat ride around the south-eastern coast with Pennicott Wilderness Journeys. According to the tour guide it was one of the calmest days they’d seen this year, but unfortunately that didn’t help because about half-way through the trip I got horribly sea-sick, though thankfully (barely) not to the point of throwing up, but it meant I was mostly sitting there feeling absolutely miserable and not even being able to look at any of the scenery. Kristina was suffering as well, so in hindsight the whole boat trip was a terrible idea. It redeemed itself with lunch at Port Arthur Lavender (again, incredible food), then dinner at Rockwall. One of their specials was a coffee-rubbed wagyu, and it was definitely the best steak I’ve ever had. It was pretty much falling apart in my mouth, it was so tender.

The other side of the lookout

Day 3, we went to Bruny Island. The “attractions” there were rather over-hyped, but the scenery is absolutely stunning. It’s like a tropical island in places, the clarity of the water is unbelievable. The afternoon was spent walking around the streets around where we were staying and checking out all the old houses, then dinner was at Mures. Mine was quite tasty, but Kristina’s risotto was crunchy, and even after she sent it back and got a new one, it wasn’t brilliant. She got a cold seafood tasting platter afterwards though, which was REALLY good.

Looking down to Hobart

Day 4, the last full day we had, we wandered around the botanical gardens, had lunch at Blue Eye (in keeping with everything else, the food was excellent), then went for a drive up to Mount Wellington. On the drive up, there was someone in a van who was doing 40km/h in a 70km/h zone and kept drifting over entirely into the other lane. It was terrifying! Thankfully there was a solid section of broken line on the road where I could overtake them, and the little Micra managed it surprisingly well. Mount Wellington itself was absolutely incredible, I’ve never been up a mountain where there’s alpine foliage before, and it felt like an entirely different world. Dinner was at Don Camillo, which was quite tasty but not on the level of some of the other restaurants.

On the last day we walked around Salamanca Place again, and had lunch back at Rockwall before heading to the airport. Our own car is only a 1.8-litre 2000 Corolla, but it was pretty funny feeling like it had a whole lot of grunt driving back from Sydney airport after having spent five days driving the Micra around. 😛

All in all, Tasmania was excellent and we’ll definitely be back!

New roof

We discovered earlier in the year that the roof on our house was fucked. We’d started to get leaks in the kitchen and dining area, had a guy come out to fix them, and it turned out that the tiles themselves were completely brittle and were cracking just from being walked on. D: The guy who came out said he’d never seen anything like it, and he guessed that it’d been a bad batch of tiles right from the beginning.

We ended up getting an entirely new metal roof, in dark grey, and it’s SO NICE. They put a layer of 50mm thick insulation directly underneath the metal so it’s not a standard “The rain comes and you can’t hear yourself think”-type of metal roof. I’m quite interested to see how much of a difference it makes in summer. It’s also steel rather than tin. It took four days and cost just shy of $14k, but there wasn’t really much of a choice. 🙁 At least the weather was nice while it was being done, anyway!

I took a couple of photos while it was being done.

And then the finished product!

We also went to Featherdale Wildlife Park for kungfupolarbear‘s birthday last week, to take photos and generally visit the animals which was pretty great as always.