Monkey see, monkey do; occasionally monkey learn.
TwitterRSS Feed
A weekend at Random Hacks of Kindness

A weekend at Random Hacks of Kindness

Random Hacks of Kindness (or RHoK for short) is a hacking event that I've been involved with on and off over the past four or five years. Since we have just had a recent RHoK event in Melbourne (Nov 2017) I thought I should write down what my own personal involvement was this time so that others may get a feel of what it is like to become involved in RHoK or other similar hacking events.

randomhacks

RHoK is not a competitive hacking event with big prizes up for grabs but instead aims to be a collaborative hacking event where some of the hackers will switch teams when they cease to be of use to their current team or feel they can provide more value on another team. Rather than trying to use some company's new API and do something cool with it we instead look at trying to solve real-world problems for the changemakers who usually come from a charity, community group or social-enterprise and who have a real-world problem that they would like help with i.e. usually idea-rich, cash-poor looking for a kick-start. Another feature of RHoK, is the emphasis on continuing involvement in the same project between RHoK events through what has become known as RHolls (Rock 'n' Roll - get it?) that take place every 4-6 weeks; and even using your own spare time if you have any to spare. There is no actual commitment to be involved with a project after a hack event has finished but it is really encouraged and it is up to each individual to decide on their continuing level of involvement.

So who gets involved in RHoK? Well, it isn't just software developers or coders as these projects require people with a wide range of skills and backgrounds so that we can be effective and deliver the right thing. My friend Samara tweeted this picture

The hack weekend

Now that the stage has been set I'll continue to describe my involvement with the most recent Melbourne RHoK event.

Day 1.

The Changemaker Pitches

This was the final chance for each of worthy changemakers to make their cause known and to persuade any final non-committed hackers to join their team. In no particular order (mainly because I forgot) the changemakers on the day were.

  • Berry Street - Berry Street are back with their mission to help children in protection stay connected to their families and other important people in their lives. This changemaker, like several changemakers before them, are proof that you can get a team to engage in several RHoK and RHoLL events to get that project across the line.
  • Go Jane Give - Hoping that RHoK can help them extend the reach of the grassroots campaign that allows you to turn your talents and interests into a fundraiser for your favourite causes.
  • Carers Couch - Needs help to provide online support to carers of people with cancer. Martina of Carers Couch has introduced a new term "cabbage salad" that we are sure is to replace the word "spaghetti" when dealing with awful/complex code.
  • PollyannR - Wants to improve the way that regional creatives can get the funding they need.
  • Start Broadband - Helping disadvantaged Aussie families get online.
  • Shifra - Looking to use technology to increase/improve sexual and reproductive health resources for mobile populations, such as refugees.
  • Care to Compare - An enterprise where consumers will get to compare and choose health products with 100% of the profits going to charity.
  • Hitnet - Co-creates and distributes health information and services to marginalised communities. They need help in maintaining and managing the content for their 70+ (and growing) information hubs that are provided to local communities across the country.
Team forming and storming

Hackers who had already been to the information night held earlier in the month had already picked their cause and started to form around the respective changemakers, others quickly joined the teams that they felt they could contribute to. I joined up with Carers Couch as I had some previous interaction with them on the run-up to the hack event as I was asked to be their RHoK mentor and to help them prepare for the weekend. The team started off by introducing themselves and getting down to the task of determining what the needs were and what could be tackled in that weekend. As the team was large enough that there was even a possibility that several tasks could be tackled in the same weekend.

The Carers Couch team

During this time another RHoK mentor came over and asked if we had any UI/UX people willing to jump ship to help another team that really needed one. Once we had a lull in our session I went over to the Hitnet team to get more details about what they needed and it looked they had already found a UI person and had now picked their goldilocks problem (not too big or too small, just right) and were stuck with another issue involving XML. After a quick discussion I realised I could probably contribute more to this team rather than to the initial Carers Couch team I was currently involved in, and so after a quick apology to the team, I moved tables.

After another round of introductions, we started to carve up the problem into a potential solution and assign some tasks to ourselves.

The problem

Dan Laush has written a wonderful write-up of the project itself so I'll only summarise here.

The Hitnet team need to update an XML configuration file for each hub on their network to control what content is to be displayed on each and they currently do this by hand for over 70+ hubs each time new content is supplied to them.

A sample XML hub file - well the part we need to manipulate

The approach

A strawman design

The team identified we had just the right skills to split into three to tackle each part of the intended solution of our simple multi-tier architecture. As I had already decided to jump onto this team to help them tackle the XML issue I decided to concentrate on the worker process that would take XML files that were used for hub configuration and update them based on data pulled from an API that was to be built by other members of the team. By breaking up the project this way kept the concerns separate and allowed individual teams to largely work independently of each other, mocking the data where required until the other team(s) were ready to serve us our data.

The worker process

Hitnet were already using Google Cloud Platform (GCP) to host their servers and so it seemed only right that we put the worker there right next to the servers/files it will need to interact with. An added bonus is that I had never used GCP before so now was a chance for a learning experience as well; a big plus in my opinion.

The worker process had the task of taking an XML configuration file for a hub and updating it with data retrieved from an API call. The data itself was the description of the modules, i.e. content, that are to be displayed on that hub. We quickly fleshed out a number of tasks needed to meet the objectives and check if the worker was a feasible choice.

  1. How easy is it to create a Function on GCP?
  2. How do we read and write files from/to storage?
  3. How will we manipulate the XML files?
  4. How will we make the API calls?

Assumptions

  1. As we did not have direct access to Hitnet's GCP account (and since we are new to this it is right we should be kept well away) we made the assumption that the XML files were available via a GCP storage bucket.

Creating a Function on GCP is really easy and in minutes, I had created an account and got access to $300 of free credits to use over the next year. Node.js appears to be the default language for writing Functions on GCP and Google has also provided a number of examples; these examples had pretty well answered how to do steps (1) and (2) which was a win. The worker process itself is triggered by a simple HTTP(s) request and it seems it could also be triggered by a message placed on a queue but we have no need for that at the moment. There was no mechanism for a timed trigger like with have with Azure Functions but we could use a cron job or a service like Zapier.

Manipulating the XML, proved to be harder than it should have been and I eventually settled on using the xml2js NPM package as it used the xmlbuilder NPM package that would allow us to save the manipulated data back to XML again. We finally got to the stage where we had the ability to pull, manipulate and push back a single XML file. Finished the day by adding code to iterate each file in the target storage bucket and update them.

End of the day

At the end of the day, each member of the Hitnet team had something successful to declare:
UI team - had a React app up and running and created their first forms.
API team - had spun up an API using nginx and a Postgres database.
Worker team - had a worker process that could do all the parts required for it.

Day 2.

Coming in by train

I had a look at the code I helped create the day before whilst heading back to the event and after all the hacking and tweaking the day before it was a mix of callbacks and promises and it felt dirty. However, this is not a difficult thing to resolve and so I used the time to clean up the code.

Putting it all together

After the success of the day before we were all ready to connect everything up and this really put the pressure on the API team to create the APIs we needed. Whilst I waited for the API team to create the many endpoints needed by the front-end team I spent some time adding some error handling, improving the detail of the console logging and, generally tidying up the code.

The Hitnet team

Once the API team were ready for me, we worked together to build the query that returns the required content modules for each hub. To consume the API I decided to use the node-fetch NPM package to make the API call and, because I dislike building URLs using string concatenation is this always seems to cause problems for someone later down the line, I decided to construct any URLs using the url-assembler NPM package.

The marketplace

At previous RHoK events we stopped at 3pm for presentations held in a lecture theatre but this year the organisers decided to use "The Marketplace". During the time allocated each team can spend time presenting and demoing to the other hackers and the judges and hopefully persuade those with remaining tokens to vote for them for the People's Choice awards (and yes you could vote for yourself if you wanted to but I don't know of anyone that did).

The future

There are still things to do with this function to make it production ready and we hope to deal with these in the next few weeks through the RHoll events that we have already booked up.

  • Security - we need some, not only to make sure only trusted users can use the new UI but to also make sure the API is not called incorrectly and to finally ensure the worker is not constantly triggered and use up our precious credits on GCP.
  • Integration - we need to integrate with Hitnet's GCP and see if we can access their files via the cloud storage that we assumed we could.
  • Process - improve the build, test, deployment process of the worker function. We are looking to hand this back to Hitnet at a later date so they can maintain this going forward.

Summary

I definitely enjoyed the weekend and I definitely learnt something new, even better when it is doing something that would be of actual value to someone. The code for the worker (like everything produced at RHoK) can be found on their github account i.e. hitnet-worker.

I believe everyone in the team tried/learnt something new, I saw this tweet from Liz who also joined the Hitnet team to work with Dan on the front-end.

and my understanding is that Ankit and Peter, who made up the API team, were also in somewhat personally unexplored territory when building the API using nginx and Postgres.